Lately I’ve been writing a lot of specifications and by that I mean test specifications. And by that I mean the specifications you tend to write in tools like Cucumber, SpecFlow, Lettuce and so on. What’s been interesting is deciding at what level of intent to write at.

In my world there tend to be two major focus points for testing. One focus point is the business rule. Here any test is demonstrating a business rule by exercising a high-level statement of what is expected to happen. This rule can, and in fact should, exist separate from the details of the user interface.

Another focus point is the workflow rule, for lack of a better term. Here any test is utilizing the user interface in order to carry out the demonstration of the business rule. However, the specifics of exactly how the user interface is used are not necessarily part of the workflow.

As an example, I may have a system that has an algorithm engine behind it. These algorithms ensure that business rules are adhered to. For example, maybe our system has a maximum cap on the average grant amount that is allocated to various universities. But perhaps this maximum differs depending on the resources allocated to a project at the university. So business rules might be stated like this:

The maximum cap for an average grant amount for any university is $400,000.
If a university has allocated between 1 and 9 projects, the average grant amount respects the maximum cap.
If a university has allocated between 10 and 20 projects, the average grant maximum cap is raised to $900,000.

All of this can be stated outside of any particular user interface and even outside of a particular workflow. Now here’s how the workflow might be stated:

A university creates a new study.
The university associates one project with the study.
The university applies for a grant amount of 200,000 for the study.
The university associates another project with the study.
The university applies for a grant amount of 200,000 for the study.
The average grant amount is 100,000.

Note how this workflow is separate from any particular specifies of a user interface. Exactly how a new study is created or how a project is associated with the study is not stated. I consider the above to be a series of activities that the user would engage in as part of their workflow. So we have a domain-specific activity of “associate a project with a study” and another of “apply for a grant amount.”

Now, all that stuff I’m not stating is yet another focus point of tests: the action level. This is where you indicate the exact steps that are needed to exercise the functionality. This is how someone would carry out the activity. Here is where I might describe what specific page I am going to in order to create the new study, how to associate a project with the study, where to check the average grant amount, and so on.

So what does all of this tell us? I think a couple of things and I bet they won’t come as revelations.

• The business rule focus isn’t tied to any particular user interface design, worflow, or specific set of actions. This means the business rule level should remain stable and unchanged during changes to the user interface. This is the case regardless of whether those changes are layout and/or modifications.
• The workflow (or activity) level is tied to the business rules but not necessarily to any specific actions. So if the set of actions for a particular workflow changes you should need to rewrite only that workflow. The business rules may not have changed at all and the activities themselves may or may not have changed.
• The action level is tied to the layout and structure of the application and the fields available for interaction. This means that when the layout and/or fields change you should need to rewrite only the implementation of particular action steps affected by that. Ideally you do not need to change the test at the business or the workflow level.

If I’m working in so-called BDD mode, the stability in my test specifications comes from the fact that business rules generally don’t change as much as technical implementations of those rules. This tells me that the closer my tests are to the business rules, the more stable they will tend to be. Yet it also means they can be harder to execute insofar as they tell you very little about how to go about testing. Or, rather, they foster the idea of cultural memory about how to actually carry out the tests.

And yet one of the biggest issues with tests described at the action level (such as click there, type this, and so forth) is that they can become fragile and even hard to understand, since the details of what I’m doing is mixed in with the how. Action-level information is also coupled tightly to the actual implementation so if a lot of tests repeat the same sets of actions, all of those tests have to be updated when changes occur.

So business-level may be too high (but still useful) and action-level may be too low (but still necessary). That leaves the workflow or activity-level. Is this middle ground the place to be when writing tests? Workflows are really sets of activities taking place over a specific area of an application or a set of areas of an application. In my view, this maps to how I think about testing because I’m normally thinking of “when I follow this specific workflow, then I should see a specific observable.” The “specific workflow” refers to a data condition — which can be complex — being exercised as part of a test condition.

This means my tests can be composed from various workflows. In turn, my workflows are composed of specific actions or sets of actions. A set of logically related or grouped actions — an action set, as it were — can be considered an activity.

Let’s summarize a bit here:

• Workflows are an activity set.
• These workflows are composed of statements, each of which defines a particular activity.
• Each activity is composed of an action set.
• An action set is a logically related group of actions.
• Actions can be combined to form activity sets that allow a specific activity to take place.
• Actions can be reused to create different activity sets.
• Different activity sets mean different workflows.

So then I come to the Given-When-Then format and figuring out how to word these things. Or, rather, how exactly to convey the above in the Given-When-Then format. And that’s always where the problems start to crop up in my experience and why many testers give up on tools like Cucumber. I read up a lot on this and there are a lot of differing opinions. For example, I read Why I Don’t Use Given-When-Then and the salient point I took from that was:

I want to read sentences. When possible, I want to read short sentences. Given When Then generates a matrix of sentences to be parsed. GWT also saves some space in the report, but I gladly give that space back to have behavioral sentences that are easier to read quickly.

From Contemplating Given-When-Then I took this to heart:

If there are other things that need to be initialized for this test that are not mentioned in the Given setup, then it’s time to pause to reflect. Could this indicate a mismatch between the world envisioned by the business and the implementation in the system? That could be something to correct. Or might it be something that is often implicit in business conversation, but needs to be explicit in the test.

Then in Refactoring Given-When-Then, I responded positively to this:

My rule of thumb is: if the context is unexpected then we need to mention it, but if the context can be inferred from the other parts of the scenario, or from common sense, then leave it out. There’s a feeling that adding context will remove ambiguity, but all it actually does is complicate the example.

Then I read the polygot tester and came across this:

There is value in brevity. There is elegance in sparseness. Elaborate wordy structures are not always the best approach.

I agree with that sentiment but I actually found that what people meant by “brevity” could differ quite a bit and what someone considers an “elaborate wordy structure” could differ. Some testers would worry about being too wordy over and above making sure they were conveying intent. Some testers would strive for so much brevity that they let ambiguity creep in the door.

But, overall, these various thoughts and opinions started to crystallize around my own thoughts regarding how to express activity-level (or workflow) statements. I’m going to give you an example of what I did in terms of a style that seems to work for me, even though I’ve been told this is NOT a “good way to do it” by some BDD practitioners. I won’t go too much into the details of the application I’m testing but rather just give you some scenarios and show you how I evolved them as my thinking evolved.

Here were some scenarios I started out with:

Given the Location tab of a plan associated with a Phase IV no IND study
When  choosing locations
Then  the default MOH/FDA Delay for all locations and regions is zero

Given the Location tab of a plan associated with a Phase IV no IND study
When  a location is selected
Then  MOH/FDA Delay is "0"
And   Total/Avg for MOH/FDA Delay is "0"

I don’t like using first-person phrases on business-facing scenarios. So you’ll see my Given here is not “I am on the Location tab of a plan …” and my When is not “I am choosing locations”.

As I looked at these, my When felt a little sparse. The Given was saying where I was supposed to be at the start of the test and the When was saying the action to take. However, going with what I said earlier, people tend to think in terms of a workflow and that usually means that, in their mind, they wrap up “the action I’m taking” and “the place I’m taking that action” in the same context. So my above scenarios became this:

When  choosing locations on a plan associated with a Phase IV no IND study
Then  the default MOH/FDA Delay for all locations and regions is zero

When  viewing selected regions on a plan associated with a Phase IV no IND study
Then  the default MOH/FDA Delay value is "0"
And   the total for MOH/FDA Delay is "0"

Here “choosing locations” and “viewing selected regions” were essentially what someone was doing. It was a domain-specific activity within our application. This activity was taking place within a specific context: in this case on a system entity called a plan. Not only that, but this was a plan that was associated with another particular system entity, a study. Not only that, but the study was of a very specific type: a Phase IV no IND study. (For those curious, this is a type of clinical trial and the IND means Investigative New Drug.) Further, the very fact that I’m “choosing locations” or “selecting regions” or “selecting locations” means I’m on the Location tab of a plan. That information is encoded within the very action we are talking about.

But doesn’t that hide a lot? After all, a tester totally unfamiliar with our application would not be able to test this functionality. True, but I argue I don’t want a tester who is totally unfamiliar with our application doing the testing! Why would I want that? If my application is being tested, I want it being done by people who have some idea of what they are doing in the context of the application domain.

Anyway, I really like this format because my scenario is now a high-level set of activities but that are removed from some specific implementation (action) details. So, going with the first scenario, those activity statements are then matched behind the scenes as such:

Notice how here I start to get into implementation specifics and here I word things in a first-person context. I do that because the activity at the intent level breaks down into specific actions that someone would take. If that someone was a tester, the above statements could be what goes into a test case or what goes into the “steps to reproduce” for a bug report. That led me to some heuristics:

• If I’m writing an activity, it’s at an intent level and is viewpoint-neutral.
• If I’m writing an action, it’s closer to the implementation level and in the first-person viewpoint.

To put this in a simpler context, consider the example given in Polygot Tester regarding the juke box. For me, combining the context (Given) and the action (When) means I could end up with this:

When coins are deposited in a juke box showing 0 credits

The “0” part could be a regular expression such that the phrase can be reused for any credits. In fact, you could make that entire phrase after the word “juke box” a regular expression so that you could also allow for:

When coins are deposited in a juke box that is out of order

I see this as a way to clearly state business intent without putting a lot of specific data in a intent-level statement. In other words, I can use specific data conditions (“showing 0 credits”) or specific test conditions (“that is out of order”). Here what “out of order” means is implicit in the same way that “being on the Location tab” was for my “choosing locations” activity.

I guess what I’m saying is there are two layers here both in natural language. The activity level is stated in high-level workflows that only tell you what you are doing (“choosing locations”; “depositing coins”) on what sort of entity (“on a plan associated with a Phase IV no IND study”; “an out of order juke box”). The action level is stated in lower-level terms that tell you the sequence of steps that allow the activity to be carried out.

What happens when you get even more complicated than these examples, though? You definitely run into some challenges. Here’s another example that I was working on. This was a much more complicated financial application that had to do with the monetary flow into accounts.

When  a sale transaction's effective date is prior to the most recent final balance
And   cash will be received during the month prior to the start of the window
Then  the starting balance should not change
And   the cash value will change

When  a sale transaction's effective date is after the most recent final balance
And   cash will be received during the month prior to the start of the window
Then  the starting balance will change
And   the cash value will change

When  a sale transaction's effective date is prior to the most recent final balance
And   cash will be received after the start of the window
Then  the starting balance should not change
And   the value of cash and accounts receivable will change

When  a sale transaction's effective date is after the most recent final balance
And   cash will be received after the start of the window
Then  the starting balance will change
And   the value of cash and accounts receivable will change

There is a lot of information here and these were statements that our business people could get behind because it was worded exactly how they thought. Looking at those activities, one thing would not have been clear to you which is that all of this takes place on an account. Where that was made clear, however, was in the narrative of the test specification. Another thing you can’t fail to notice is the utter lack of specific data. Does that matter?

Well, the above captured the business rule and the workflow as part of an activity. The details of how to do the workflow were behind the scenes. But what about the details of what data to use? Well, the good thing about understanding the business rule is that it normalizes the data in that as long as I have a data condition that complements the stated test conditions, the details of the data probably don’t matter. Consider a much simpler example:

When the data collection method for a plan is set to "DC - 3rd Party"
Then the DC 3rd Party Costs are non-zero.

When the number of subjects are increased for a plan
Then the DC 3rd Party Costs should increase.

Now there’s a trick there. The first scenario bleeds into the other or, rather, the second sort of relies on the first. Yet this was initially how business folks wanted to talk about these things. But I ended up modifying the above such that the first one became this:

When the data collection method for a plan is initially set to "DC - 3rd Party"
Then the DC 3rd Party Costs are non-zero.

The second became:

When the number of subjects are increased for a plan that was set to "DC - 3rd party"
Then the DC 3rd Party Costs should increase.

But some felt that this wasn’t stated very business like. Here the business logic was as such: If people using our software (who would be vendors) did not want our software to calculate labor fees related to the process of capturing data, then those vendors would want to create pass-through costs. (These are costs incurred by the vendor but passed on to the client.) No vendors of the plan are responsible for “DC – 3rd Party” data collection management, which is why it’s considered third party. So what this led is to some more activity-based (not action-based) wording:

When none of the providers for a plan are responsible for data collection
Then there must be DC 3rd Party Costs

When one of the providers for a plan are responsible for data collection
And  the number of subjects in the plan are increased
Then DC 3rd Party Costs must increase

As with my other examples, a lot of the implementation logic is hidden behind the scenes. Likewise, what the exact costs are in the first scenario or what they increase by in the second scenario is less important than those statements because, ultimately, those statements hold true for any specific numbers.

This post is getting quite long so I’ll stop here but the main points I was going for here was:

• I believe writing activity (workflow) level statements in BDD tools can be a powerful way to convey business rules.
• This seems to work best when I conflate the Given and When.
• This seems to work best when I chain first-person action steps behind the scenes.
• I believe that writing generalized test conditions rather than specific data conditions can be an effective way to make sure the focus is on the business logic being fulfilled.

That said, my thoughts are still evolving on this and I’m working on a lot of different examples to see if I can utilize all that I talked about here in simple and complex scenarios. What you see here is a very early snapshot of my thoughts in progress.