In my post Are We Defocusing Our Development and Test Practices? I was talking about the various “driven-development” terms. I had brought up how Behavior Driven Development (BDD) seems to have largely come about because some people felt that the term “test” was a bit too confusing for some people or, at the very least, off putting for some reasons that are not explained very well. There was thus apparently a shift to “examples” that talked about “behavior” and this was somehow different than a test. And this somehow made everything a bit better. But did it? What was it about the terms “test” and “testing” that were just not working? I’d rather people deal with that problem before thinking we need to come up with new acronyms. I say that because, ultimately, the concepts of a “test” and the activity of “testing” is still very much there.

In the aforementioned post, I had said:

What I’m less okay with is when it’s not recognized that all those terms have some form of testing as their basis or when the terms are used solely to avoid having to say the words “test” or “testing.”

I felt that the most active BDD promoters, although not always the practitioners, whether intentionally or not, often seem to try to cover up any notion of testing or tests. By itself, that might not be so bad, particularly if the day-to-day practitioners by and large don’t take the same tack as the BDD promoters. But then we started getting all these others terms — EDD, PDD, ATDD — and those were conflated, contrasted with, or compared to existing terms, like TDD. I then mentioned a particular resource I found helpful: Specification by Example by Gojko Adzic. In Gojko’s book of the same name, you’ll read this:

I realized that the practitioners of Specification by Example, myself included, have traditionally been guilty of using technical terms to confuse both ourselves and everyone else who tries to implement these practices …. If we want to get business users more involved, which is one of the key goals of these practices, we have to use the right names for the right things and stop confusing people.

The trick, of course, is that the “right names” for the “right things” are things that people still disagree on — which is probably part of the reason why we have BDD, TDD, ATDD, EDD, ADTP, and so on. As an example of this process in action, however, Gojko says:

Until I started using specification workshop as the name for a collaborative meeting about acceptance tests, it was difficult to convince business users to participate. But a simple change in naming made the problem go away.

The key here is that he didn’t change the fact that the output of this process was tests — specifically, acceptance tests. What he did was make the process of creating those artifacts more palatable by calling the process by which tests were created a more friendly name. Gojko could have called these “collaboration workshops.” He could have said this is “part of BDD.” He could have said “this is how you do ADTP.” Any of those may have been correct. But pick a name and then stick with it. I believe that’s what your teams have to do — and knowing that many of the names out there refer to the same exact thing is one way to stop worrying about what you’re actually doing and get down to actually doing it.

Actually, I’d like to know why Gojko’s audience of business users found it so difficult to participate when acceptance tests were referred to. I routinely involve business users in writing acceptance tests and I have yet to find a situation where the notion of a test scared anyone off. In fact, tests were things most people could understand and get behind. Gojko then goes on to say:

I also want to avoid using the word test too much. Many managers and business users unfortunately consider testing as a technical supplementary activity, not something that they want to get involved in. After all, they have dedicated testers to handle that.

This, to me, is a classic case of not recognizing (or at least teaching others to recognize) that “testing” is a broad term, that is applicable across the product and application development process. I personally can’t abide the idea of avoiding the use of the word “test.” I don’t think you can make it clear enough that, ultimately, testing is what you are doing.

People should not fear the word “test.” It’s not scary. Really.

I can see the concern that putting the word “test” or “testing” in something might suggest you’re forcing everyone to do testing. There is, however, a difference between specifying tests and executing tests. If you aren’t making that clear to your business users, they may equate the two in their mind. Even putting that to one side for a minute, if there is this fear of forcing people to do things that are not their job, then why do concepts like BDD, TDD, EDD, and so on lack that same concern about having the word “development” in there? Is it because ultimately everyone is involved in developing to some degree? Perhaps. But if that’s the case, then everyone is equally involved in testing. Or they should be, anyway.

Here’s another example. Gojko says:

When a team uses a piece of information to guide development, support the system, or estimate the impact of business changes, it’s misleading to call that information a “test.” Tests aren’t used to support and evolve our systems; documentation is.

Why is that misleading? Good tests can help estimate because the nature of a proposed business change can be reflected in the number of tests that will have to be added, reworked, or removed. Tests most certainly can be used to support a system. A system can evolve based on the understanding that is encoded in the tests, either those that are pre-existing or those that are created in order to guide further development in a given area. Further, if tests are a form of documentation then this notion that tests don’t support or evolve development but documentation does is itself a bit misleading. Ironically the above quote comes in a section in the book called “Tests can be good documentation” so I was quite confused by this. Again, this just shows me how people sometimes use the term “test” in limited ways.

A test is any objective means of critical evaluation. This means tests are a multi-faceted work product. As such they can absolutely help us support and evolve a system. The only thing I can guess here is that Gojko is simply saying rather than use the word “test”, he’d prefer to call it “documentation.” That’s fine, I suppose, but I think it’s actually a bit more misleading in that sense since the term “test” has a more focused connotation than “document” and I’ve seen many more people fear the term “documentation” than they do “tests.”

Another interesting statement from the book:

The result of this refinement is at the same time a specification, a target for development, an objective way to check acceptance, and a functional regression test for later. I don’t want to call this an acceptance test because it makes it difficult to justify why this document needs to stay in domain language, be readable, and be easily accessible.

Huh?!? You would have difficulty justifying why an acceptance test (serving as a form of document) needs to be in the domain language, readable, and accessible? Really? I guess that’s just so totally outside my experience that it’s hard for me to understand where that sentiment even comes from. Attaching the qualifier “acceptance” to the word “test” can connote an extra layer of detail for people. Yes, you may have to explain it. But that’s okay. There’s nothing wrong with educating people on the word “test” and its potential broad applicability.

I do think Gojko takes a more practical approach than coming up with some methodology (like, BDD) by focusing on what things are called (regardless of methodology) while still making sure that the high-level terms map without ambiguity to testing. So “test first” becomes “specifying collaboratively” and “writing functional tests” becomes “illustrating using examples.” Very simple. No need to worry about whether I’m using a “second-generation, outside-in, pull-based, multiple stakeholder, multiple scale, high automation agile methodology.” (See Dan North’s slides on BDD, from where I got that. You can also see a presentation of these concepts.)

You might also check out Liz Keogh’s article ATDD vs. BDD, and a potted history of some related stuff. At one point in there you’ll see it mentioned that the focus is “the language of examples and behaviour, rather than tests and acceptance criteria.” It’s the “rather than” that gets me. To me, the “language of examples and behavior” is the same thing as “tests and acceptance criteria.” I don’t see the difference between the two. I realize Liz was talking about the alleged difference between BDD and ATDD but I don’t see the difference there either, as I mentioned in my previous post.

I’m not saying there isn’t a difference to some people because clearly there is. I just don’t see why there needs to be. I feel all this throwing around of acronyms and diluted distinctions between concepts really does nothing to help the practice of testing. Having trained lots of testers, I can say that many of them (rightfully) find all these distinctions less than illuminating when it comes to how to include the business in effective and efficient testing and use that testing as an input to development rather than an output of it. Having worked with many developers, I can say some of them are a little hazy on the exact details as well. (Again, see my previous post for some examples of that.) Incidentally, the “ATDD vs. BDD” post ends with this:

The difference is that one is called Behaviour Driven Development – and some people find that wording useful – and one (or two) is called (Acceptance) Test Driven Development – and some people find that wording useful in a different way.

So, basically, it’s whatever you want to call it that you find most helpful. Great. As a critical reader, when I see that I wonder if that applies to all the other “driven-development” stuff out there. Contrasting Liz’s post, Gojko for his part does seem to make a distinction between ATDD and BDD:

The acceptance-testing-centric model (often called acceptance test-driven development, ATDD, or A-TDD) focuses on the automated tests that are part of the Specification by Example process. In this model, the key benefits are clearer targets for development and preventing functional regression.

The system-behavior-specification-centric model (often called behavior-driven development or BDD) focuses on the process of specifying scenarios of system behavior. It centers on building shared understanding between stakeholders and delivery teams through collaboration and clarification of specifications. Preventing functional regression through test automation is also considered important.

Again, to me “specifying scenarios of system behavior” (what he refers to as BDD) should be the same thing as gathering acceptance tests and writing them down (which he refers to as ATDD). Whether or not those tests are automated is of secondary concern to me since automation is a technique. At least, however, Gojko is making an operational distinction and it’s clearly one that could be adhered to in a team. This is the kind of thing that I think is needed if people are going to make sense of these terms. I would much rather hear something like the above then just hearing “Well, BDD and ATDD may mean the same things but they may not. It really all just depends.” For example, Gojko says of these two models:

The acceptance-testing-centric model is more useful for initial adoption if a team has many functional quality issues. When things are running smoothly, the behavior-specification-centric model is useful for explaining the activities of short-term and mid-term software delivery.

Now, again, you may not agree with these, but you can’t deny that at least there is some specificity about how the terms are being presented.

Another example came to mind when I looked into Domain Driven Design (DDD). Let’s consider a quote from Eric Evans, who wrote the very interesting book Domain-Driven Design. Eric has said that the term “agile” has largely lost all meaning because just about anything can be called “agile” these days. It’s become a catch-all term that, in many cases, simply means “not waterfall.” In Eric’s keynote for DDD Exchange 2010, titled DDD Emerging Themes 2010, he said:

Anything can be called agile, anything can be called SOA — when anything can be called that it’s no longer a useful term.

I’ve been advocating that BDD (and other members of the acronym soup) should have practitioners that work at defining some operational specificity about what their approach actually is, how it differs from other approaches, and what the expected outputs are of the approach. Since I mentioned, DDD and Eric Evan’s thoughts about ‘agile’, I will say that the conference mentioned was a way to define the driving principles behind DDD and what it means to be “doing DDD.” You might also check out the presentation Folding Together DDD and Agile.

Regarding “anything can be called agile,” I fear the same is true (or becoming true) with approaches like BDD. The goal is conversation and that’s great. But what that conversation should lead to is valuable artifacts. Examples and scenarios may be valuable artifacts. To me, they are valuable if they can be turned into or are used as tests. That works when people are encouraged to treat tests as being effective to the extent that the tests make sure actions and output agree with a shared notion of quality. That shared notion of quality is encoded by things like scenarios, examples and so on. Those latter elements serve the purpose of testing; they are not a different name for tests. They are simply one among many ways of writing tests and thus performing testing. Examples and scenarios are techniques.

Tests are an objective evaluation criterion. That should be the definition of a test. An example or a scenario can be that. But until the example or scenario is defined as a test, it’s not necessarily that. And if you say an example or a scenario must be that, then why not just stick with the word “test”? Scenarios and examples can feature a lot of detail that doesn’t need to be in place for a test. Scenarios and examples can contain a lot of surplus detail. They become tests when that detail is refined down. What you need is a concrete (objective) and precise context for what’s going to happen and how you know it happened correctly.

An example may show you a calculation that should occur as part of your application, where the example says “for these inputs, a calculation will be done that leads to this output.” But a full test will indicate how you validate that calculation by itself or as part of a UI or perhaps in terms of how it appears on different reports that your application generates. In some cases, test validation is best done at different levels.

• You’ll have unit tests to check on the actual methods that build up the calculations.
• You’ll have integration tests that make sure a full calculation path results in a the correct output for a given input.
• You’ll have system (or end-to-end) tests to make sure that the calculation results actually show up in an web-based user interface.
• You’ll have web service tests to make sure that the calculation results are accessible via an interface (API calls).

The example is definitely a good guide, but it’s not a substitute name for a test. Rather, I would say it’s a way to conceptualize tests. It’s a technique for writing tests. Now consider a possible scenario:

1. Create a transaction on an account.
2. This creates a system generated journal entry in the general ledger.
3. The user then checks a transaction report, a GL report, and income/expenses report.

This scenario provides a lot of possible tests, including tests of validations on the form fields for creating a transaction. Like the example before, the scenario is definitely a good guide, but it’s not a substitute name for a test. Again, I would say it’s a way to conceptualize tests.

My main goal here was really just to continue my theme that “test” and “testing” are not terms to be avoided or replaced with other terms. The terms that some people use in place of “tests”, such as “examples” or “scenarios” are, to my way of thinking, just different ways to write some aspects of a test. They are techniques; they are ways of doing things. I don’t have a problem with the concepts of BDD (or TDD or ATDD or whatever) except to the extent that I believe the promoters of these concepts often haven’t made enough operational distinctions to say why one is different than another or why it would matter. Combined with the apparent concern about the words “test” or “testing”, I want to make sure testers think critically about these concepts, particularly when the promoters of the concepts often seem a little hazy on the details and distinctions themselves.