There are many references out there that discuss Gherkin, which is a structuring element for your test description language (TDL). So what I’ll be offering here is really nothing new. This post is simply a relatively concise look at the TDL pieces and parts in terms of how these elements can be used in the context of Lucid or related tools.
Test Description Language (TDL)
A test description language is the language you use to describe, and thus specify, your requirements as tests. Or your tests as requirements, if you prefer. This is what allows testing to be a design activity. What makes a style of writing a TDL is adherence to a structuring element and a set of principles and patterns that are used to guide expression.
You will generally be using your TDL in the context of a test specification or test spec. Your TDL can be parsed by Lucid (or other similar such tools) and can be executed either by a human being or by an automated testing tool.
Specifications
Specs, somewhat like user stories, are vertical slices of system functionality that are (in theory, anyway) deliverable independently. Stories are generally chunks of work that tend to be small. This is to support their use as a description of scope as well as a means of frequent delivery. This means a single feature can be provided through numerous stories.
As a note, some tools that do similar things to Lucid call specs by the name of “feature files.” The problem is that a user story, or a spec, can impact several features. Sometimes a spec is a feature file; but sometimes it is more of an ability or a business need that impacts multiple features. That means your spec can be describing an entire feature or just part of the feature. What this does is allow you a certain amount of flexibility as to what you consider to be a spec file. What should be a constant, however, is the following:
A spec file should describe some aspect of behavior with representative examples of expected outcomes.
I’ll cover how you make sure you are adhering to this guideline when I talk about the spec template.
With many testing tools, you test to validate some system functionality against its requirements (or specifications). With a tool like Lucid and adhering to a TDL, the requirements and the tests are the same thing. A test specification is a set of requirements that are written up in a testable format. With this, you have traceability built in. You also have acceptance built in because the TDL can be written by developers, testers, or business analysts. Ideally you have them all collaborating as specs are written. The TDL can even be written by customers.
Another point to consider is the following:
Each spec should specify the direct value it brings to users of regarding the behavioral functionality the spec is describing.
A good way to make sure you are adhering to this guideline is to realize that the people that the functionality will provide value for should be able to write acceptance criteria for it. If they cannot, then it’s likely not bringing them direct value.
Test Spec Template
Below is a template of what a test spec is composed of. Anything in italics indicates specific material you would enter. Colored words are keywords that are specific to the TDL.
Feature: Title Description / Narrative Background: Steps Scenario: Title Steps Scenario: Title Steps Scenario: Title Steps
Feature
Every test spec will have a descriptive keyword at the top. Above you can see this keyword is Feature. You can also use Ability or Business Need. That keyword will be followed by a name or a phrase, which you can think of as the title. This title is a concise statement about what aspect of the application is being specified in this particular test spec. This title helps a reader understand what the spec file is about in general.
A good technique for deciding on a title is to think about what you would type into a search engine to look for this spec file if it were online.
A narrative or description can follow the title. This can be as short or as long as you like; it is free-form text that describes the intention of the behavior (feature, ability, business need) covered in the spec file.
This paragraph, regardless of how long, should give the reader a context to understand the rest of the file. What you are doing here is describing the feature. Generally, to describe a feature, should clearly state who cares about some functionality, why the functionality is important to them, and what the functionality actually delivers.
A good technique for the description is to use one of the standard feature injection templates.
When you have an effective description, you have enough material for meaningful discussion on how to specify this story. A flow of activities, or scenarios, that provide end-to-end value is what you are looking to achieve.
Scenarios
Each spec should be illustrated with key examples. The examples show the expected outcomes in specific representative cases with very precise information.
A scenario is the mechanism for capturing examples for a feature in the spec file. Each scenario will have the Scenario keyword. That keyword will be followed by a title. The scenario title is a concise statement about one aspect of the feature.
A scenario title provides the reader with the context required to understand the key example described by the scenario.
A good technique to use when deciding on a scenario title is to try to summarize the intent of the example covered by the scenario to another person. Try to reduce your summary to the fewest words possible.
Since the scenario represents an example of usage, that means the scenario represents a way that the feature delivers some value to someone who will use it. Most features will be made up of multiple scenarios. A few things to consider in this regard:
- Each scenario describes a different example of how that feature should behave in different circumstances. This allows scenarios to be used to explore different paths through a feature.
- Each scenario is a single concrete example of how the system should behave in a particular situation. If you add together the behavior defined by all of the scenarios, that’s the expected behavior of the feature itself.
From a testing standpoint, it would not be inaccurate to think of a scenario as being equivalent to a test case. The scenario is the test case title.
Considering the scenarios may provide assistance with something mentioned earlier regarding the feature description. Specifically:
A good technique for describing a feature is to write the scenario titles first, and then summarize them.
Regarding the idea of scenarios being examples, it leads you to ask what kinds of examples? How many? The idea is you want to focus on key examples. I say that because one danger of using all examples is that people tend to write every type of test in TDL. The TDL is effectively speaking to acceptance-driven and system-level based tests. There are many other tests, however, that are much better done at a unit test level. So that’s why you focus on key examples. But even that leaves open what that means. Some general guidelines are:
- At least one representative example illustrating each important aspect of business functionality. This means so-called “happy path” workflows, wherein you allow for different data conditions.
- Some examples illustrating each important edge case. You do not describe every edge case, so using techniques like equivalence classing boundary conditions.
- Some examples illustrating any particularly troublesome areas of the expected implementation. This could be known areas of fragility in the application, test or data conditions that caused bugs in the past, elements that rely on legacy code, and so forth.
Steps
Scenarios follow a pattern. Specifically, every scenario will be made up of, or described by, steps. Steps are domain language phrases that we can combine to write scenarios. These steps indicate how the specific scenario can be executed, either by a human or by an automated test tool.
Each scenario tells a little story describing something that the application should be able to do. To keep things simple, steps either refer to the context of a scenario, an action that the scenario describes or a way to validate the action.
Scenario: Title
GIVEN some context
WHEN some action
THEN some observable
To help structure scenarios, the steps follow a common structure, using the keywords Given, When, and Then. The statement following the Given keyword will be the context. The statement following the When keyword will be the key action of the scenario (the test condition). The statement following the Then keyword will be the observable that occurs as a result of executing the specified action in the specified context.
Keep in mind that these steps should be the minimum relevant steps to carry out the intent of the test. The intent is indicated by the scenario title. A few other points are relevant:
- Each scenario must make sense and be able to be executed independently of any other scenario.
- All scenarios should directly relate back to the feature title. The scenarios are effectively a working example of the feature.
You can have multiple step keywords.
Scenario: Title
GIVEN some context
GIVEN some other bit of context
WHEN some action
THEN some observable
WHEN some other action
THEN some other observable
You can use connector keywords in scenarios if you feel it makes it more readable. Connector words are And and But.
Scenario: Title
GIVEN some context
WHEN some action
AND some other action
THEN some observable
BUT some other action
THEN some other observable
You do not have to place a Given in your scenarios. Sometimes the context can be wrapped up in the action negating the need for a Given step.
Scenario: Title
WHEN some action
THEN some observable
You can use a generic step if the context, action, and observable are all simple to state and manifestly apparently from a single phrase.
Scenario: Title
* some action takes place in a context and leads to an observable
Backgrounds
One of the elements you see in the above earlier test spec template is a something called a Background. A Background is entirely optional but if you have a common set of actions that you want for every scenario in a test spec, then you would put those steps under a Background keyword.
In all respects a Background acts like a Scenario, but a background section in a test spec allows you to specify a set of steps that are common to every scenario in the file. Instead of having to repeat those steps over and over for each scenario, you move them up into a Background section.
A Background will generally only consist of a context, meaning a Given step.
You should use a Background if you find it improves the readability of your features by removing repetitive steps from individual scenarios.
A test spec can only contain one Background section. The reason for that is, again, a background is meant to be a common context for all scenarios in the spec. If you find yourself wanting more than one background, more than likely that means you simply have scenarios with different context or you perhaps have scenarios that belong in a different spec file, under a different feature title.
Effective Test Specs
There can be many audiences for a test spec. Think about your specs as documentation that will be read much more than they will be changed. Consider that other people will need to read and understand your specs months and perhaps even years after you write it.
Certainly one audience for test specs is to serve as a target for development. This target is one that should prevent misunderstanding and ambiguities because of the fact that scenarios are specified as examples. This means scenarios in a spec file should be:
- Written with precisely testable conditions.
- Written as specifications, not scripts.
- Written in domain language.
- Focused on varying data conditions.
- Focused on intent, not implementation.
- Talk about business functionality, not software design.
There are definitely heuristics (made up of patterns and anti-patterns) to writing test specs. Some examples:
- Very long files or scenarios are an anti-pattern. The longer they get, the harder they are to reference and the harder they are to understand. Work to break scenarios down so that they focus on a key action (test condition). If there start to be too many scenarios, perhaps describe the feature or ability from a higher level of abstraction.
- Generic values in scenario steps. A step like “Given a bank account with a valid number” sounds convenient, but “valid” is not precisely defined and is thus ambiguous. A step like “Then the correct withdrawal should be made” seems convenient but you don’t know what ‘correct’ is in this case. The point of an example is that it should be realistic and very precise. Use an example of a valid number (and specify another scenario that illustrates an invalid number). Define what ‘correct’ means by illustrating it with the precise output.
- Inconsistent language. As the project evolves, the domain language might change, whether expanded, reduced or amended. You should be working to use key domain phrases that indicate task or workflow actions that users will actually take when performing an example. If something is called a “Customer”, don’t call it a “User” in some scenarios and a “Person” in others.
- Several actions or mixed validations and actions. Although a scenario that checks several cases makes a perfectly good regression check, it can be hard to understand. Each scenario should be independent, focused on a single example and perform a single key action. It is fine for related observables, as the result of a key action, to be included in one scenario.
A Useful TDL Example
Here is an example from a hypothetical application that uses some of the above concepts. This is presented solely to show you some of the concepts in a realistic context.
Feature: Support a timeout period for liquidating redemption transactions
Background:
Given an account with a transaction with a valid date of notification
Scenario: Default value for Timeout Period
* the default value for Timeout Period is blank
Scenario: Valid Timeout Period values can be saved
* the transaction can be saved with a Timeout Period of 2
Scenario: Timeout Period can be 0 for soft-lock accounts
* the transaction can be saved with a Timeout Period of 0
Scenario: Invalid Timeout Period values cannot be saved
When the transaction is saved with a blank Washout Period
Then an alert will be displayed.
Scenario: Timeout Period can be configured regardless of value
* configuration options will be shown when attempting to configure
| a blank Timeout Period |
| a Timeout Period of 10 weeks |
| a Timeout Period of 20 weeks |
| a monthly Timeout Period |
| an annual Timeout Period |
Something to note here: I was able to use a Background because all of my scenarios were going to use an account that had a transaction with a valid date of notification. The background saved me from specifying that context in every scenario.
However, if any of my scenarios were going to require a different context, such as an invalid date of notification or no such date at all, then I could not use the Background. I could do one of two things in that situation:
- Specify the appropriate context in scenario (with a Given step).
- Make the feature more specific.
Regarding the second point, I might create two test specs, with the following feature titles:
- Feature: Support a timeout period for liquidating redemption transactions with valid notification dates
- Feature: Support a timeout period for liquidating redemption transactions with invalid notification dates
In that case only scenarios relevant to those feature titles would go in each test spec.
Other TDL Elements
There are some supporting elements that you can use as part of your TDL. I’ll cover those here.
Test Tables
Sometimes steps in a scenario need to describe data that doesn’t easily fit on a single line of Given, When, or Then. So you can place these details in a table right underneath the step. Test tables give you a way to extend a step beyond a single line to include a larger piece of data.
Given the following data
| Name | Affiliation |
| Spider-Man | Marvel |
| Superman | DC |
| Batman | DC |
The table starts on the line immediately following the step, and its cells are separated using the pipe character: |.
The table above shows using a heading for each column in the table. You have the freedom to specify data in different ways, such as putting the headings down the side:
Then the following characteristics will be returned
| weapon | light saber |
| powers | use of the Dark Side |
| disposition | hates everyone |
You could also just specify a list:
When the following Ninjago characters are selected
| Sensei Wu |
| Garmadon |
| Great Devourer |
Knowing which format to use depends entirely on how your step is written.
You can have multiple tables in a scenario.
Given a user has posted the following messages:
| I am making dinner |
| kids just woke up |
| I am going to cry |
When a query on "I am" is executed
Then the results should be:
| I am making dinner |
| I am going to cry |
Scenario Outline
Sometimes you have several scenarios that follow exactly the same pattern of steps, just with different input values or expected outcomes. You can use a scenario outline to specify the steps once and then play multiple sets of values through them.
Scenario Outline: Game Packs
Given the price of a game pack is $10.00
When <number> of game packs are bought
Then the price should be <total>
Examples:
| number | total |
| 1 | $10.00 |
| 2 | $20.00 |
You indicate placeholders within the scenario outline using angle brackets. These brackets indicate where you want real values to be substituted. The scenario outline itself is useless without a table of Examples, which lists rows of values to be substituted for each placeholder.
You can have any number of Scenario Outline elements in a feature and any number of Examples tables under each scenario outline. You can also create a Scenarios table if you feel that reads better.
Scenario Outline: Withdraw fixed amount
Given <Balance> in an account
When withdrawing the fixed amount of <Withdrawal>
Then <Outcome>
And the balance of my account should be <Remaining>
Scenarios: Successful withdrawal
| Balance | Withdrawal | Outcome | Remaining |
| $500 | $50 | receive $50 cash | $450 |
| $500 | $100 | receive $100 cash | $400 |
Scenarios: Attempt to withdraw too much
| Balance | Withdrawal | Outcome | Remaining |
| $100 | $200 | see error message | $100 |
| $0 | $50 | see error message | $0 |
Notice here how Scenario tables (and Example tables) can have their own title.
Extended Strings
You can specify short strings as part of your regular steps. For example, you can do this easily enough:
Then an alert message should say "Don't ever do that again."
Extended strings allow you to specify a larger piece of text than you could fit on a single line like this. For example, you might need to describe the precise content of an email message.
Scenario: Ban Griefers
When a player griefs with TNT
Then an email should be sent containing:
"""
Dear Horrible Player,
Your account privileges have been revoked due to your
unscrupulous behavior recorded on:
31 January 2012, CSR Ticket 12345
Enjoy your lifetime ban.
"""
And the users account should be locked
The entire string between the triple quotes is executed in the context of the step above it. The indentation of the opening “”” is not important but the indentation inside the triple quotes is significant. Imagine the left margin running down from the start of the first “””. If you want to include indentation within your string, you need to indent it within this margin, as you can see in the above example.
Extended strings can useful for email messages but also for longer alert messages or help content text. You could use extended strings to specify snippets of JSON or XML if you are testing web service API responses.
The Structure is Not the TDL
A final note here is that the structuring element of Given-When-Then is just one of many that you could use. The TDL is actually the part that is worded in your domain language. This is entirely free-form in that you can say whatever you want to say. The trick is that, ultimately, these phrases must be used as unambiguous statements for testing purposes, both manual and automated. That means you are going to want to slowly build up a list of accepted TDL phrases that you can reuse in a variety of contexts. Rather than go into more detail on that in this post, you can see other posts in the TDL category that provide more contextual examples.
