Regarding my previous post on this subject, a tester I work with asked me a great question regarding the readability of the TDL and the ability to discern the actual requirements from it. This post is essentially what my answer was. Whether my answer is good or not is up to the individual reader.

The tester had looked at these examples from my previous post:

Scenario: No Discount For Active Customer
  GIVEN an Active customer
  WHEN  the customer makes a purchase less than $10.00
  THEN  no discount is applied
  WHEN  the customer makes a purchase of exactly $10.00
  THEN  no discount is applied

Scenario: Discount for Active Customer
  GIVEN an Active customer
  WHEN  the customer makes a purchase of $10.50
  THEN  a discount of 1% is applied

Upon doing so she felt she had a “fundamental misunderstanding of, at the very least, what the scenario title is supposed to convey.” Specifically, from her viewpoint the scenario title was the requirement.

Are Scenarios Requirements?

When she said that to me, I realized something immediately. If, with the above examples, you treated the scenario titles as requirements, you have a contradiction. Here’s what you have if you just look at the scenario titles:

Scenario: No Discount For Active Customer
Scenario: Discount for Active Customer

The tester said she would have written the scenario titles as this:

Scenario: No discount for active customers when purchase is less than or equal to $10.00
Scenario: Discount for active customers when purchase is greater than $10.00

Keeping in mind that the scenario title is generally a “non-executable” portion of a test spec, what this can do is lead to repetition. Consider those new titles along with the steps:

Scenario: No discount for active customers when purchase is less than or equal to $10.00
  GIVEN an Active customer
  WHEN  the customer makes a purchase less than $10.00
  THEN  no discount is applied
  WHEN  the customer makes a purchase of exactly $10.00
  THEN  no discount is applied

Scenario: Discount for active customers when purchase is greater than $10.00
  GIVEN an Active customer
  WHEN  the customer makes a purchase of $10.50
  THEN  a discount of 1% is applied

The issue for me there is the duplication. I restate the conditions in two places: in the scenario title and in the steps themselves. At the very least, this allows the possibility that I might update one without updating the other. But assuming I have discipline enough to not do that, doesn’t the above make sense? Aren’t those scenario titles supposed to be requirements, after all?

Conditions, Not Requirements?

Well, as I see it, the distinction here is that scenarios in a feature file are not statements of requirement, per se. They are conditions upon or of the requirements. The feature file had a title and the scenarios were ways of testing that feature. Looking at what I had originally, we have this:

Feature: Apply a Discount for Weekend Sale Purchases
  Scenario: No Discount For Active Customer
  Scenario: Discount for Active Customer

All this being said, it didn’t take me too long to realize that this can seem a bit confusing, particularly if testers are coming from a world where their test case summary, for example, is meant to a statement that reflects upon, or even restates, some requirement. Further, the tester said this about my original example: “My concern is that in the situations [you provide] one must read through the steps to know the requirement.” The argument thus being that the requirement was easier to read with the longer form scenario title. And that can be a valid argument, right?

So let’s talk about this for a moment. The key question really is: “Where are the requirements in a TDL format?”

From my perspective, the conceptual issue here is the idea of a test spec, emphasis on that last word. A test specification is really a requirements+test specification. All of this is predicated upon the idea that testing is a design activity and that, at a certain level of approximation, requirements and tests merge into the same type of communication mechanism. (The old Möebius idea.)

But that can get tricky, right? After all, consider back in the day when lively “debates” — if such they can be called — swirled around the distinction between a “requirement” and a “requirement statement.” When BDD first started coming on the scene, lots of conversations focused on the idea of distinguishing between “business need” and “business understanding.” That was tied up with the notion of distinguishing between a “business rule” and a “requirement.” And then, of course, you got into what is “acceptance criteria” versus what is an “acceptance test.” I probably even jumped into this fray myself with Testing’s Brave New World.

Whether these debates were needed at all is not not the point. Rather it’s just to show that conflating terms can lead to some cognitive friction, particularly when people have years of experience with certain terms.

What Kind of Specification?

When I first started down this path, I preferred the phrase “test specification.” The reason for this is that the phrase “test specification” (or “test spec” to make it sound more lively) is because I like to call attention to the fact that we have a “specification” (so people start hearing ‘requirements’) that is encoded as a “test”. In my mind, the notion of a requirement sort of changes in that the “requirements” are the combined set of the scenario titles plus the steps. Yet it’s sort of a new way of looking at it: by the conditions. Or, put another way, each scenario is one view into the overall “set of requirements.”

This is why the Given, When, and Then statements themselves should be relatively short and as declarative as possible. I say that because it should not take a long time to parse them to understand the specific scenario. So let’s go back to my example:

Scenario: No Discount For Active Customer
  GIVEN an Active customer
  WHEN  the customer makes a purchase less than $10.00
  THEN  no discount is applied
  WHEN  the customer makes a purchase of exactly $10.00
  THEN  no discount is applied

That tells me that there is a condition wherein there is no discount for an Active customer. So now I know that condition is a business rule, effectively. Getting into specifics, the “no discount” condition holds when a customer that is Active makes a purchase that is less than or equal to $10.00.

All of that what I just said is one requirement.

We have a statement of intent (“there is no discount for active customers”) and the specific implementation which says specifically what that means (“purchases of less than or equal to $10.00”).

And, in fact, in the interests of quick parsing this is one area where the combination of the Given into the When can help:

Scenario: No Discount For Active Customer
  WHEN  an Active customer makes a purchase less than $10.00
  THEN  no discount is applied
  WHEN  an Active customer makes a purchase of exactly $10.00
  THEN  no discount is applied

But, being even more reductionist, then that can really become:

Scenario: No Discount For Active Customer
  * no discount is applied when an Active customer makes a purchase less than $10.00
  * no discount is applied when an Active customer makes a purchase of exactly $10.00

And then I suppose that could become this:

Scenario: No Discount For Active Customer
  * no discount is applied when an Active customer makes a purchase less than or equal to $10.00

Largely that (or something close to it) is probably how a business analyst is initially going to state the situation. The question is where to stop trimming down. Personally, with the asterisk examples, I still like a line for each specific data condition (“less than $10.00”, “exactly $10.00”).

Does This Impact Automation?

Wrapped up in this, of course, is that if you are using a TDL like this, you are probably also considering automating these tests. So you do have to consider how you will be able to automate the above phrases, particularly as you reduce them down. Clearly the more you reduce them down, the more specific your statement gets. Whether that makes it easier or harder to automate really depends on your framework for establishing data context and so forth.

I don’t want to turn this into an “how to automate phrases” post, but just so it’s clear I do think about these things, let’s consider two variations from above:

Variation 1

  GIVEN an Active customer
  WHEN  the customer makes a purchase less than $10.00
  THEN  no discount is applied

Variation 2

  WHEN  an Active customer makes a purchase less than $10.00
  THEN  no discount is applied

The only difference is that in the second one I wrap the context (GIVEN) with the action (WHEN). Is that valid? A purist would perhaps say no. My heuristic is usually that if the context is simple enough that it can be stated as part of the key test action without making the phrase cumbersome, it might be worth doing this. Some people disagree with this entirely. I still go back and forth on this myself, to be honest. The point here is that I would have no trouble automating either phrase set above. What I’m most concerned about is how readable these phrases are and how quickly they can be parsed to gather meaning.

This notion of “requirements as tests” (which, to me, is another way of saying “specification by example”) means a rethinking of a “requirement” or, at least, how requirements are situated relative to tests. A requirement is something that is (obviously) required. But more specifically, a requirement is a specific behavior that we say must be exhibited by the application. However, that requirement is made executable by the test itself. Thus the tests and the requirements stop being distinct entities, at least to some extent.

Whether or not this is a “new way of thinking” really depends on the relationship you previously considered between requirements and tests. Further, regardless of whether you agree with my viewpoint on this, I hope you can see that it is important to at least consider what it means to “read a test spec”, particularly from the viewpoint of different audiences.