As a tester you will often encounter environments that desire to practice some form of Behavior-Driven Development (BDD) and testing practices must fit within that. The term “BDD” may be used more or less vaguely. One thing that’s often somewhat common is that people will start to promote so-called “BDD tools.” Examples here would be tools like Cucumber, Spinach, Turnip, SpecFlow, Behat, Lettuce, and so forth. These tools all ask you to adhere to some sort of organization based on terms like “feature”, “story”, “scenario” and so on. What this often ends up doing is totally missing the point.

But what is the point?

Testing is an empirical, technical investigation based on questions that you ask. These are questions that you can ask an application in a sense, such as by trying things out with it. But more importantly these are questions that you ask people, not just about the application but how the application is supportive of and responsive to the business domain that it operates within.

You perform testing in order to get answers to relevant questions. Testing is thus done to provide quality-related information so that the business can make informed decisions. Testing is thus ultimately all about communication.

That is the point.

So with a “BDD approach” there’s usually a focus on concepts like “feature”, “scenario”, and so forth. That focus is often placed on how those terms are used within a tool. First, let me point you to what I think is a fantastic article: step away from the tools. What I get out of that article is simply this: don’t make test automation the end goal. That’s putting the focus on tools. Whereas the focus should be on the continuous goal of communication and collaboration. The act of collaborative discovery gets you and your teams to a shared notion of quality. That shared notion of quality can then be tested for. Automation is just one technique in your arsenal of testing.

When I talk to people about this they often can conceptually get behind the idea of not focusing on a tool. But then when they talk about the concepts of “features”, “scenarios”, “stories” and “cases” they start to go back to how the tools seem to use those terms rather than how they and their teams would like to use those terms.

Tools work for you… The tools you use should serve you and what you want to do. You don’t choose your terminology based on what some tool is willing to accommodate on your behalf.

So let’s talk about those terms. While I do this, keep in mind that I’m giving you my view of the world and how I tend to conceptualize everything. I don’t promote this as the right way. I promote it as a way; one that has worked well for me. What I will try to show you is some of the thinking that led me to how I currently use terms, and why I made the decisions I did. Knowing my rationale may better help you see why you agree or why you disagree.

I’ll start with what many people consider to be the highest-level concept: a feature. Let’s agree to say that a feature is a piece of system functionality that delivers value to people who will actually use or benefit from the application or service being developed.

A feature can be thought of as defining a “unit” of user value. In other words, this feature as an atomic unit is beneficial to users. It gives them certain benefits or helps them avoid certain consequences.

One problem I often had was that a lot of times I seemed to be in meetings discussing something that didn’t quite feel like a “feature” but rather more a “capability” within a feature or an ability of the system to do something. Let’s consider a simple but common example. “Export to PDF” and “Export to Excel” may seem like features, but I could probably argue they are capabilities of an overall feature which is “Ability to Export Information” or, perhaps even better, “Ability to Share Information.”

So I’m wondering if “feature” versus “capability” matters. Now I’ll say that some people like to do one better and even question whether ability and capability should be different terms. A “capability” is usually defined as the feature of being capable of something. Not too informative, huh? I like the idea that a capability is a practical ability. As such, I personally don’t see the need to operationally separate these terms. Instead the question for me is this: does feature = capability? Well, what if we think about it this way:

• A capability is a specific action that the application or service can do or provide.
• A feature is a concept that an application or service supports.

So, again, a feature of the product is, say, the ability to share reports in various formats. One particular way provided to accommodate that is the ability to “Export to PDF.”

Did you notice? In that last sentence I defined the feature as a whole as an ability (“share reports”) as well as an aspect of the feature as an ability (“Export to PDF”). So, really, the distinction must not matter too much to me. I could have changed my wording but, really, is it that important? Probably not. I’m perfectly willing to accept the idea that a “feature” is a more or less granular concept based on the context and I’m apparently willing to call just about anything an ability.

Let’s see if this holds up. Taking another example, an application or service handles attempts at unauthorized access to the system. That’s a feature of the system. To actually provide that feature, the application or service has the ability to recognize various types of invalid login attempts such as:

• Generates error for bad username
• Generates error for bad password
• Generates error for valid login/password but with inactive account
• Denies access when an authorization token that has expired is used
• Denies reset password functionality when security question cannot be answered

Okay, now stop for a minute. Given what I just said, do you feel that the way I define feature versus capability should be reversed? Could these examples have been conceptualized differently? Most certainly they could have and if you wanted to reverse the terms I would say do it! Use terms in a way that make sense to you. But also keep in mind this point: does the above distinction of feature vs. capability even matter all that much to you?

Let’s put that on hold for now. What about stories then?

A story is usually said to be a slice of system functionality that is deliverable independently from other work. The idea is that each story should be able to specify the value it — and it alone — brings to people who will presumably benefit from the functionality that will be provided to make the story something real. In the modern business analysis world, the structural motif used for stories tends to be something along the lines of “As a … I want … so that” or “In order to … as a … I want.” So:

As a manager who reports data for my company
I want the ability to export reports to different formats
so that I can better handle distribution of information to my clients.

Or, worded in the other common motif:

In order to better handle distribution of information to my clients,
as a manager who reports data,
I want the ability to export reports to different formats

I personally prefer the first one; I just think it reads better. You can see that this format really gets into talking about business needs:

• As a {who},
• I want {what},
• so that {I get something / I avoid something}.

The challenge here is that a story can often impact several features. Yet a single feature might be delivered through a series of stories. For example, the above story talks about the benefit of not just “export reports” but also the benefit of “different formats.” Maybe, upon discussion, it’s determined that a few stories are needed: Export to PDF, Export to Excel, Export to CSV, Export to HTML. So somebody may actually want to break the above out in that way:

As a manager who reports data for my company
I want the ability to export reports to PDF
so that I can better handle distribution of information to my clients.

As a manager who reports data for my company
I want the ability to export reports to Excel
so that I can better handle distribution of information to my clients.

...

And so on. Do you see how that could potentially get a little cumbersome? After all, the stories are all pretty much saying the exact same thing except for one part. Yet that “one part” is pretty darn important isn’t it? So assuming you go with the slightly more generic user story, how do you make sure you capture the relevant details? One answer is that the various ways of satisfying the business need can be covered by scenarios.

A scenario is an aspect or facet of story framed as a test that is used to help people think through the various ways a feature can be used. What that means is that a scenario captures one key example for a feature. In other words, a scenario represents a way that the system delivers value to the person using it. Put yet another way, each scenario exploits an ability that the application or service claims to have in support of the feature. A scenario is thus something very much like a test case.

And I just know some testers out there went “Whaaaaaat?” when they read that last bit. A scenario is like a test case? Utterly ridiculous. A test scenario isn’t a test case. A test scenario is a path through the system; it’s a path that may involve many different test cases. Isn’t it? Well, maybe; let’s not worry about that for now.

You can essentially take each statement of capability/ability and break it down:

Ability {of whom/what?} to {do/see something} {on/with} {something}

Filling that in for one example:

Ability of data manager to export to PDF

Since the story indicates the who, I can really just say this for my scenario title:

Export to PDF

So here’s what the whole thing would look like if written down in some nice, concise format:

As a manager who reports data for my company
I want the ability to export reports to different formats
so that I can better handle distribution of information to my clients.

Scenario: Export to PDF
Scenario: Export to Excel
Scenario: Export to CSV
Scenario: Export to HTML

The above is what would be produced in a simple specification workshop.

Spec workshop? What’s that? Spec workshops are a wonderful concept but not one I’ll be going into too much here. Suffice it to say that I hope you can see how the above could quite easily be the output of a discussion with business analysts and developers and, further, be used as input to later development and testing activities. If you can see that, you’re well on your way to understanding spec workshops.

Some people refer to a story as something that expresses one very specific need that a user has. By this definition, either “Export Reports” as a whole could be a story or the individual “Export to {something}” would be a set of stories. I think most agile developers would prefer the latter because that kind of granular approach is better subject to estimation in terms of level of work. Yet there are some people that would say that the story is “Export Reports” and the “Export to {something}” would be a use case. Uh oh. Another term. Do we need it? Let’s see.

A use case, by this logic, would have to be pretty similar to a user story. The difference that your average business analyst will bring up is that a use case describes one specific interaction between the user and the application or service. If you think about it, though, that’s pretty much what the scenario is doing here as well. (Agree? Disagree?) Personally, I like the idea that use case = scenario = test case.

Here’s how I like to think about it. User stories are about needs. When you write a user story, what you’?re describing is a “raw” user need. It’s something that the user needs to do in his day-to-day job. If you never build any software for that user, that need will still exist. This perhaps makes it clear why I think the feature and capability/ability thus is not that much of a distinguishing characteristic. The capability that the user needs as part of their job (“sharing information”) and the idea of a specific aspect of that (“sharing reports (via PDF)”) can all sort of be thought as a feature. The business need has been stated.

Use cases are about the behavior you’ll build into the application or service to meet those needs. A developer who needs to build working software should be able to read a use case and get a good sense of what the software needs to do. It typically has a lot of detail, and describes everything that the developer needs to build in order to meet the users’ need. That’s why it needs to have a lot more detail, and be clear and unambiguous. And, hey, what luck! That’s exactly what testers need as well.

Going back to the previous concepts, this is why you take the “As a … I want … so that” and break it down into scenarios that say “here is how what you want works.” The “so that” part speaks to value. Why are we doing this? Is it really valuable? That’s important because breaking down stories can be hard. It’s hard because you have to make sure that those stories still deliver value at each point. In other words, maybe “Export to Excel” and “Export to CSV” is crucial because the manager actually likes to work with the data a bit, perhaps do some pivots, and so forth. This is not something that could be done with PDF or HTML — at least not as easily. So the Excel and CSV stories deliver more value. Which means the user story should change to reflect that, such that perhaps the only use cases/scenarios/test cases being considered are export to Excel and CSV.

I mentioned that scenarios can be thought of as test cases. Okay, so what about test conditions? Do they come into play here? Some testers live and die by the distinction between test condition and test case. Others don’t. I’ll go on the assumption that while you may not live and die by them, you do care about the distincion and I’ll say this: the way that a scenario is broken out into steps could be considered exercising a condition. Here’s how I like to think about it.

• A list of conditions are input to test case design.
• So a given set of conditions defines a class of cases.
• Particular conditions then define individual cases.

A condition should be a specific aspect of a test; it should not be a condition within a test.

So three conditions here might be:

Do "Export PDF/Export XLS"
  on "Basic/Enterprise Product"
    with "Admin/Power User".

Each quoted element is a condition pair. There will thus be 2 × 2 × 2 = 8 test cases. A single case (or scenario) would be “export PDF” on “basic” with “admin user.” Other conditions that might get factored in could be the browser and/or operating system that is being tested on. Think of the condition like this:

IF I am a power user,
  AND IF I am using the Basic Product
    AND IF I export to PDF
      ...

Here you can see how the conditions break out a bit. The overall test case — scenario — encompasses these conditions.

And that finally brings us back to that whole test case vs. scenario distinction. Some testers really don’t like this definition of scenario because they feel that a scenario is more all encompassing. In this view, a scenario might be something more along these lines:

"A user logs in, generates a report, validates the information on the report and then exports the report to PDF."

I would agree that is definitely a user story and, yes, you could most certainly argue that it’s a user scenario. But let’s think about this for a second. Stories describe goals and desired outcomes. Here the actual goal is a report that is accurate. The fact that it can be distributed as PDF is probably secondary since if the report is totally inaccurate, being able to distribute it easily is of much less concern.

To me scenarios don’t so much “describe goals and desired outcomes” as they do provide a specific example of how the goal is achieved and the specific nature of the outcome. So if you agree to treat a scenario as a test case, then a scenario should be some specific action — or set of actions — that lead to one very specific observable that can be specified in very concrete terms. That’s as opposed to the more overarching idea of a scenario as a single path of user interaction through the system to accomplish a specific high-level goal.

Perhaps a better way to say this is that the above story, while being a user scenario, is not necessarily a test scenario simply because there are tons of things that could also be tested during the actions of that user scenario.

And that brings up a good point and perhaps a bit of a digression. You might check out what James Bach had to say regarding BDD vs. testing. Specifically note the idea of the “BDD scenario turned into testing.” I think James’ point is that people make a mistake when they equate BDD work with testing work. I personally try not get into too many of these debates. I agree that BDD isn’t testing. What I do think is that BDD is ultimately about design and communication. I also think that’s what testing is about as well. In my view testing is (or should be) a design activity.

This is probably the place where I can go back to the tools but it’s also the place where I think I’ll stop for this post. That last point — that testing is a design activity — is an important one. As part of that activity, you’ll use terms like scenario, feature, user story, use case, test case, and probably a few others that I didn’t bring up. I kept this post fairly tool neutral. I want to come back to this topic and address some actual scenarios and tests but I need to get my own thoughts in order first. As I stated earlier, treat the above as my opinion based on what I find useful and workable as a set of operational terms. Your mileage — along with your opinion — may vary. If it does, I’d love to hear from you.