Are We Defocusing Our Development and Test Practices?

As a tester, you’ll hear about behavior-driven development (BDD). Sometimes this goes by acceptance test-driven development (ATDD) or maybe even acceptance test-driven planning (ATDP). You may hear that these are related to test-driven development (TDD). After a while you might start to wonder if these are just different ways of saying the exact same thing.

My take is that, for the most part, they are. I think this plethora of names and the lack of operational specificity around them lead people to construct other names that then make their way into the mind space of developers and testers. As an example, I’d prefer if BDD referred to Business-Driven Development since that’s putting the focus where it ultimately needs to be. I say that because most of these acronyms are referring to techniques where the business is involved to some degree in writing tests or guiding development.

In truth, of course, business has always guided development since releasing a product is a business decision, not a technical one. (It may be a business decision backed up with technical input but it’s still a business decision.) Approaches like BDD are trying to make that process a little more visible and a little more sustained while putting the focus on collaboration via conversation. So, wait, should we have Collaboration-Driven Development (CDD)? Hmmm. Well, anyway, I like the approach that BDD seems to promote. What I’m worried about is that the profusion of terms can cause cognitive friction, semantic nests, and ambiguity where none need be.

One of the big concerns I have with BDD is not with the approach itself but rather the way it’s approached by some practitioners. Consider why the term BDD was used: to get away from the term “test” in TDD. You’ll hear various reasons as to why this shift was supposedly necessary. A great source would be to consider Dan North, who seems to have coined the term. Consider Introducing BDD. You’ll read this:

It suddenly occurred to me that people’s misunderstandings about TDD almost always came back to the word “test”. … I started using the word “behaviour” in place of “test” in my dealings with TDD and found that not only did it seem to fit but also that a whole category of coaching questions magically dissolved. … I found the shift from thinking in tests to thinking in behaviour so profound that I started to refer to TDD as BDD, or behaviour-driven development.

Okay, let’s grant that this verbal change worked. Lots of problems “magically dissolved” by using the word “behavior” rather than “test.” What I think this really shows is that people weren’t thinking about tests correctly since tests do express behavior and the verifications of the results of that behavior. Either way, even in Dan’s formulation, you’re still doing testing. I’m worried that this “distinction” (if such it is) leads to a possibly harmful division of thought and a confusion of terms. But okay, fine: people needed this shift in nomenclature. So even if I’m way off base on my worry, I would then argue that BDD doesn’t really distinguish itself since many terms out there seem to be referring to the same thing, but not always obviously so. This can lead to a defocusing of what’s actually being done and that can foster process and practice ambiguity.

As a tester you’ll probably come across phrases like “we don’t test, we specify behavior” or “we have scenario and examples rather than tests.” You’ll most often hear those from BDD practitioners. These viewpoints are somewhat dangerous in my view because it can get people out of the habit of thinking about testing. And while there may be debate about who does the testing, there should be no debate that testing will be done.

While I supported BDD as an approach, I kept having people — both testers and developers — bring up various other terms like TDD, ATDD, and many others. What I’ll charitably call “debates” would swirl around the “correct” way to be doing development and, by extension, testing. This was particularly the case in environments where testers and developers worked closely and influenced each other’s thinking. What I found is that people did seem to be treating these terms differently but in some vaguely undefined way. My arguments that these debates seemed a bit fruitless since we were all talking about the same thing tended to fall on deaf ears.

As I was having lots of these thoughts I came across Specification by Example by Gojko Adzic and, somewhat unexpectedly, I found a mirror for some of what I was thinking. In the Preface to the book, I read this:

Different teams use different names for their ways of dealing with specifications and tests, yet they all share a common set of core principles and ideas, which I hold to be essentially the same thing. Some of the names that the teams used for these practices are …

… and then he lists the following: agile acceptance testing, Acceptance Test-Driven Development, Example-Driven Development, story testing, Behavior-Driven Development, and Specification by Example. Gojko’s viewpoint is that all of these things are “essentially the same thing” and I happen to agree. About this situation, I then read this:

The fact that the same practices have so many names reflects the huge amount of innovation in this field at the moment.

That’s one way to look at it and there’s no doubt some truth to that. It’s also possible to say that there’s a moderate amount of confusion in the field at this moment, with people often using different names to mean the same thing or, perhaps worse, using the same names to refer to different things. This is what happens when you don’t use operational definitions. Referring to the practice of specification by example in particular, Gojko says:

The community has a dozen names for the practice as a whole and twice as many for various elements of it.

So for those who want to keep sticking with the distinctions of DDD, BDD, TDD, ATDD, ADTP, EDD, PDD and actively promoting those distinctions, I think what has to be done is that those people need to talk about what the expected outputs are of using one practice over another. Practitioners need to know how those outputs (and the means by which they are reached) serve as distinguishing factors so that those of us wading in the acronym soup can tell the difference regarding which process we’re following. This is rarely something I see done, hence why I believe it’s possible for there to be a defocusing effect.

Of that list of acronyms, EDD (Example-Driven Development) and PDD (Persona-Driven Development) could arguably fit within the context of BDD (Behavior-Driven Development). You’re just encapsulating the behavior as examples executed by a certain persona. If those examples are used as tests, then they feed into TDD (Test-Driven Development) in some fashion. Presumably DDD (Domain-Driven Design) would have to come in there somewhere as well to make sure everything is in the language of the business domain. Which brings up a good point: could we have the term “Development” replaced by “Design” in TDD, BDD, and so on? Would that be a different thing entirely? Or just a variation on a theme? See how this can get a little defocused? At the very least, there’s lots of wiggle room.

Going back to my main point, Gojko also says:

Specification by Example, agile acceptance testing, Behavior-Driven Development, and all the alternative names for the same thing solve these problems.

(The emphasis there is my own.) Here “these problems” refers to “poor quality, rework, and missed customer expectations” but, once again, we see the idea of similar — if not identical concepts — being named different things. I don’t mean to point out Gojko’s words as if they were indicative of the field as a whole and I’m certainly not criticizing his viewpoints in particular, especially since I happen to agree with them. I’m just using this as a particular data point regarding how others do seem to see a conflation of terms.

So I did some checking around, both past and present. Check out BDD, TDD, and the other Double D’s from 2007. Right at the start, you’ll see:

…I see BDD as an evolution of TDD with a better syntax and mechanics for expressing the desired functionality with tests/specifications.

That actually sounds pretty good for me. It does beg the question why people didn’t just update TDD to allow for “better syntax and mechanics for expressing the desired functionality.” In other words, why did we need a new term? (According to Dan Norht, it’s because people were too confused or too put off by the word “Test” in TDD.) Later in that post you’ll see how the author breaks out what he calls the “double D’s.” As you can see, some people really like to break out those “driven-development” concepts. I’m not here to say these viewpoints are wrong or right, particularly since they are opinions and nothing more. What I’m curious about is whether people think we really and truly need this many different terms.

Have things improved since 2007? Well, check out BDD, TDD, Acceptance Testing and Executable Specification from 2010. One of the comments has this to say:

BDD doesn’t replace TDD but is one way of practising the acceptance testing part of it.

Another comment says:

My gut feel is that BDD doesn’t replace TDD, but the BDD advocates seem to imply it does.

I tend to agree with both of those sentiments. Again, though, you can see how it’s quite possible for ambiguity to creep in. You may also notice that the people in the trenches who work with these concepts often have a clearer focus than those who come up with them or actively promote them.

Finally, check out TDD vs BDD from 2011. One of the comments to the article says something interesting:

I think you might be missing a key aspect of BDD. BDD is many times used for the bigger kind of tests. i.e. E@E / Acceptance tests. I don’t think that its TDD Vs BDD.

So, taking those ideas and adding in those with the post from 2007, if BDD is an evolution of TDD and if BDD is putting focus on acceptance tests, then does it follow that Acceptance Test Driven Development (ATDD) is a further evolution of BDD? Should that be what we refer to now?

On that score, check out BDD: ATDD done well?. Here we read:

[BDD] changed the language commonly used in TDD, using the vocabulary of examples of behavior in place of tests.

Okay … so is that why we have Example-Driven Development (EDD)? Is that the evolution of BDD? If it’s just a part of BDD, why do we have a separate name for it? The article makes a great point, though:

The boundaries of BDD are still unclear, and the community are revisiting the old question and asking, “What makes BDD different?”

To me, I think this kind of discussion would have been better had when deciding whether or not to introduce a new name in the first place. The comment above shows that the idea came first and now people are trying to figure out what makes it a different idea. This is as opposed to recognizing the need for a new idea, finding out how it will be different, and giving it a name. That said, Dan North did have what seemed to be a lot of arguments in terms of why he felt BDD was different in some sense. So, again, even the promoters of BDD don’t really seem to be on the same page many times.

Lest I seem a bit too focused on names and splitting semantic hairs, I will say that I’m much more concerned about whether people are engaging in contextual good practices. As long as people are clear what they’re actually doing, I really don’t care what they call it too much. So if people want to call what they do BDD one moment and TDD another — well, I suppose that’s their prerogative. What I’m less okay with is when it’s not recognized that all those terms have some form of testing as their basis or when the terms are used solely to avoid having to say the words “test” or “testing.” Read that last quote again: “using the vocabulary of examples of behavior in place of tests.” To me, “examples of behavior”, when agreed upon as being accurate, are tests. Why do many BDD practitioners want examples in place of or instead of tests?

Along these lines, and perhaps as an example of what you’re likely to come across, check out the post TDD, BDD, and XXDD. The author states:

By the way, I HATE saying TDD is “not about the testing”. I have to say it now and then because of people not realizing it’s about designing, rather than testing. The fact that the word “test” is in the name just helps confuse the issue.

I think this makes an excellent point in expanding the notion of a “test” so that people realize that testing is a design activity first and foremost. In fact, it always has been but that’s gotten lost in the profusion of terms and concepts that say everything is “driven” by development. I disagree, however, that just having the word “test” in there is enough to confuse the issue. Most people I’ve read on this subject can’t quite tell me why the word “test” was so confusing. Incidentally, regarding that last post, you might also note that we have yet another acronym there: Executable Example Driven Design (XXDD). Again, it’s really easy for these terms to make their way through the thought space.

So here’s my final thoughts on this.

  • There are ways to do things when you are trying to get a business product out to customers who sustain the business by their purchase or support of the product.
  • There are various thinking skills, encapsulated in techniques that I use as part of that goal.

So if I have to adhere to a term, I guess I’ll go with BDD but in my mind at least it’s going to be Business-Driven Development. That will be my “way of doing things.” The techniques I will utilize are collaborative discussion, writing in domain language, usage scenario writing, illustrating scenarios with examples, and so forth. I don’t need to refer to any of my techniques as Persona-Driven Development, Example-Driven Development, and so on. I can just say that using a persona is a technique. Writing up examples is a technique. Using domain language in all artifacts is a technique.

As a Quality Assurance and Test professional I’ve learned not to worry overly much about the nomenclature that developers like to keep tossing out and I’ve learned to steer testers away from blindly following all these distinctions and repeating them as if they were writ in stone. Rather, I’m encouraging testers to try to think about one overall approach, call it some name, and then think up techniques that will be used in that approach. Don’t conflate your approach with your techniques. Don’t name every technique as if it were an approach. I also want testers to think about these issues solely in terms of their usefulness. I want testers to make sure that their approach and its techniques lead to practical benefits in terms of testing and to make sure that the business needs are reflected in tests.

Ultimately testing is all about communication. That’s a much better starting point, I think, then all this worry over whether I’m doing TDD, BDD, EDD, ATDD, ADTP, or whatever other “driven-development” term comes out next.


This article was written by Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.