Symbiont is a little further along, enough so that it makes sense to look at the basic idea of what it means to provide a façade over two tools. The focus here is really just to show you some tool construction from the ground up. This may help you understand tools like page-object or watir-page-helper a little bit or they may spur on your own creative efforts as you build your own tools.
Evolving the Symbiont
In my previous post about the start of my Symbiont project, I gave a few examples of what I can do now with libraries like Selenium and Watir. It probably wasn’t entirely clear what direction Symbiont was going to go, given what I described there. Here I’ll to provide a bit more of a roadmap for what Symbiont will need to accomplish.
Rise of the Symbiont
Lately I have been developing lots of test solutions, most of them having to do with automated testing. I originally developed a library that I called Lucid. This was in turn based on a library I was originally calling Terminus. Now I’m trying to distill a lot of what have learned into a library that I’m calling Symbiont.
I’ll be perfectly honest: what I’m writing here is largely for my own benefit as I get my thoughts in order, but I do have hopes for Symbiont to be something useful. So I will document its development path here and if someone besides me finds anything of interest in it, that’s great.
Tests as Specifications
In Testing’s Brave New World, I ended up talking about BDD and the concepts that testing, acting as a design activity, has to work within. That was a post fairly heavy on nomenclature concerns. I sort of skirted around the issue that it’s the tools we use in our BDD type activities that are often forcing us to consider this terminology. Even if you are not using a specific tool, but rather a technique like spec workshops — which is a “BDD activity” — you are still wrestling with these terms.
So let’s put this in some context.
Test Writers Step in Early
In my last post about what tests are for I talked a bit about the ability of test writers to be nimble in terms of getting quality-focused information — usually as a result collaborative discussions — encoded into readable tests that focus on intent rather than implementation. All of that’s easy to say but test writers sometimes have a hard time figuring out how to exactly do it in the context of the collaboration possibilities in their environment. Here I want to talk a little about the core ideas, backing them up with an example.
What Are Your Tests For?
A key question to ask is what you want to get out of your tests. I’ve already talked about how I think the purpose of testing is to find information and then communicate that information. That’s great and all … but … what do you actually want to get out of the tests themselves? The way you answer that question will shape your testing function to a large extent, particularly when it comes to picking supporting tools, such as for test management or test automation.
Test Teams Need Inventors: Balancing Creativity
In a previous post I talked about inventors as people who see and think differently. I also brought the idea of inventors having to institute cultural change in some cases. What I didn’t do is how an inventor utilizes their skill-set and mind-set to get that change going. So let’s talk about that. Hopefully I won’t make too much of a fool of myself.
Continue reading Test Teams Need Inventors: Balancing Creativity
Test Teams Need Inventors: Seeing and Thinking Differently
In various posts I’ve tried to show how I believe testers can invent solutions to problems they are encountering. These solutions do not always have to be tool-based in nature. Sometimes you are presenting a new way of thinking about processes, sometimes you are reframing problems, sometimes you a providing a hopeful vision, sometimes you are in fact coming up with ad-hoc tools, and sometimes you are coming up with new techniques. At the core of this, however, is thinking about problems and thinking about solutions. It’s about being an inventor.
Continue reading Test Teams Need Inventors: Seeing and Thinking Differently
Cars, Shapes, and Tests
I think the future of testing relies less on a sole focus on test execution and more on the ability to write tests in the language of the business domain, effectively tying together the role of business analyst and tester. I think that future also includes the ability of testers to instrument certain artifacts, like requirements documents, so that the requirements, acting as tests, can be converted to different formats, most notably automated tests. There’s a lot of contention out there around these ideas from “purist” testers: those who feel that these attempts provide a single-source test format is getting away from testing and more into development.
While I agree in part that this is putting some focus on development activities, I disagree that this gets away from testing. Effective testing, to me, is about putting testing where it will do the most good in the most responsible fashion. Sometimes that testing is finding new bugs, sometimes it’s providing confidence that bugs haven’t come back. Either way, testing is all about communication.
Epistemological Angst (or When Does “Before” Lose Meaning?)
I find myself in a philosophical mood today and it’s based on some experiences with testers that simply don’t ask questions. As testers we have to ask questions. Lots of questions. We also have to recognize when we are getting answers. Sometimes, however, we have to realize that sometimes answers aren’t that easy to come by. And here I’m not just talking about the idea that no one has the answers. What about when answers just aren’t possible at all? Can that happen? Of course it can. It depends on the types of questions being asked and the context in which answers are expected. (See my time travel post for one example of this.)
So — just bear with me here — I want every tester out there to consider a question: what existed before the big bang?
Continue reading Epistemological Angst (or When Does “Before” Lose Meaning?)
Testing’s Brave New World
As a tester you will often encounter environments that desire to practice some form of Behavior-Driven Development (BDD) and testing practices must fit within that. The term “BDD” may be used more or less vaguely. One thing that’s often somewhat common is that people will start to promote so-called “BDD tools.” Examples here would be tools like Cucumber, Spinach, Turnip, SpecFlow, Behat, Lettuce, and so forth. These tools all ask you to adhere to some sort of organization based on terms like “feature”, “story”, “scenario” and so on. What this often ends up doing is totally missing the point.
But what is the point?
Finding Those Hidden Bugs
One area of testing that often gets neglected are the various ways that bugs can be ferreted out by considering how they “hide.” Testing an application is sometimes like walking a mine field, where you have to trigger the mines in order to see where they are hidden. Another analogy might be a virus hunter, who has to find signs of an outbreak by testing the population, some of whom are hiding to cover up the fact that they have a virus. Here I want to present some information about why testers (and developers!) should consider a technique known as hidden-fault analysis and, more importantly, show how the notion of hidden faults can skew test results.
Welcome Mutants to Your Testing Strategy
Have you ever come across the term “mutation testing”? My experience is that some people might have heard about the concept of mutation testing — perhaps in passing — but rarely have they heard too much substantive about it. Even when I talk with developers and testers, I was initially surprised at how few of them readily accommodated this sort of technique. On the other hand, if there isn’t a lot out there about it, my surprise is unwarranted. That’s even more so the case if the material out there does not talk about how mutation testing can actually be practical.
Cautionary Tales on Metrics
Some testers — and most managers — like to talk about metrics. One thing that often doesn’t get discussed is what I call metric dissociation. Here’s my thought: metrics should be easily or directly interpretable in terms of product or process features. If they’re not, I think they are dissociated from the actual work that’s being done and thus there’s great danger of the metrics being misleading, at best, or outright inaccurate, at worst.
Thinking and Testing with Executable Specs – Part 5
This is the fifth post in my series of using a tool that supports writing executable specifications. If this is the first post in the series you’re reading, then you are also reading the last post in the series. Which means this post will be almost totally meaningless to you. To catch up you can read parts one, two, three, and four. (Whether those make this post less meaningless is then up to you!)
Continue reading Thinking and Testing with Executable Specs – Part 5
Thinking and Testing with Executable Specs – Part 4
This post is the fourth in my series regarding approaching testing when you are using an executable specification type tool. The first, second and third posts are pretty much necessary reading for this one to make any sense to you at all.
Continue reading Thinking and Testing with Executable Specs – Part 4
Thinking and Testing with Executable Specs – Part 3
This post is the third in my series regarding approaching testing when you are using an executable specification type tool. This one will following on directly from the activities done in the first two posts. (I recommend reading part one and part two if you want to follow along, since these really are interconnected threads.)
Continue reading Thinking and Testing with Executable Specs – Part 3
Thinking and Testing with Executable Specs – Part 2
This post is the second in my series regarding approaching testing when you are using an executable specification type tool. Assuming you want to follow along, the first post is fairly critical to understanding how the project is currently set up.
Continue reading Thinking and Testing with Executable Specs – Part 2
Thinking and Testing with Executable Specs – Part 1
I’ve been doing some presentations and training sessions on using tools like Cucumber. What’s interesting about these is that in many cases people have tried to play around with the tool but have conceptual difficulties with figuring out how to get started in terms of making decisions continuing to using the tool.
As a tester, when introducing yourself to this area of practice, you might read that You’re Cuking It Wrong or that You’re Cuking It Right. It still may not be clear which you are doing. You might read about being imperative versus declarative and wonder which is for you. (Can’t I do both?) You might even come to the conclusion that Cucumber sucks.
Continue reading Thinking and Testing with Executable Specs – Part 1
Testing: Art and Science
I’ve heard a lot of people in the software industry say that “testing is more art than science.” My opinion is that people who say this most likely know little about art and probably know even less about science.