One area of testing that often gets neglected are the various ways that bugs can be ferreted out by considering how they “hide.” Testing an application is sometimes like walking a mine field, where you have to trigger the mines in order to see where they are hidden. Another analogy might be a virus hunter, who has to find signs of an outbreak by testing the population, some of whom are hiding to cover up the fact that they have a virus. Here I want to present some information about why testers (and developers!) should consider a technique known as hidden-fault analysis and, more importantly, show how the notion of hidden faults can skew test results.
Month: December 2011
Welcome Mutants to Your Testing Strategy
Have you ever come across the term “mutation testing”? My experience is that some people might have heard about the concept of mutation testing — perhaps in passing — but rarely have they heard too much substantive about it. Even when I talk with developers and testers, I was initially surprised at how few of them readily accommodated this sort of technique. On the other hand, if there isn’t a lot out there about it, my surprise is unwarranted. That’s even more so the case if the material out there does not talk about how mutation testing can actually be practical.
Cautionary Tales on Metrics
Some testers — and most managers — like to talk about metrics. One thing that often doesn’t get discussed is what I call metric dissociation. Here’s my thought: metrics should be easily or directly interpretable in terms of product or process features. If they’re not, I think they are dissociated from the actual work that’s being done and thus there’s great danger of the metrics being misleading, at best, or outright inaccurate, at worst.
Thinking and Testing with Executable Specs – Part 5
This is the fifth post in my series of using a tool that supports writing executable specifications. If this is the first post in the series you’re reading, then you are also reading the last post in the series. Which means this post will be almost totally meaningless to you. To catch up you can read parts one, two, three, and four. (Whether those make this post less meaningless is then up to you!)
Continue reading Thinking and Testing with Executable Specs – Part 5
Thinking and Testing with Executable Specs – Part 4
This post is the fourth in my series regarding approaching testing when you are using an executable specification type tool. The first, second and third posts are pretty much necessary reading for this one to make any sense to you at all.
Continue reading Thinking and Testing with Executable Specs – Part 4
Thinking and Testing with Executable Specs – Part 3
This post is the third in my series regarding approaching testing when you are using an executable specification type tool. This one will following on directly from the activities done in the first two posts. (I recommend reading part one and part two if you want to follow along, since these really are interconnected threads.)
Continue reading Thinking and Testing with Executable Specs – Part 3
Thinking and Testing with Executable Specs – Part 2
This post is the second in my series regarding approaching testing when you are using an executable specification type tool. Assuming you want to follow along, the first post is fairly critical to understanding how the project is currently set up.
Continue reading Thinking and Testing with Executable Specs – Part 2
Thinking and Testing with Executable Specs – Part 1
I’ve been doing some presentations and training sessions on using tools like Cucumber. What’s interesting about these is that in many cases people have tried to play around with the tool but have conceptual difficulties with figuring out how to get started in terms of making decisions continuing to using the tool.
As a tester, when introducing yourself to this area of practice, you might read that You’re Cuking It Wrong or that You’re Cuking It Right. It still may not be clear which you are doing. You might read about being imperative versus declarative and wonder which is for you. (Can’t I do both?) You might even come to the conclusion that Cucumber sucks.
Continue reading Thinking and Testing with Executable Specs – Part 1