Some testers have problems conceptualizing acceptance tests as I described them in my post on writing tests from stories. The problem seems to be one of seeing how the concept of an acceptance test applies or translates to concepts they already know, such as test conditions and/or test cases.

The good news is that “acceptance”, as applied to a test, is a qualifier on the test as opposed to a type of testing. In other words, any test can be an acceptance test depending upon how it is used. So here I’ll talk a little about how I’ve found testers can make the mental transition from test concepts they know to the qualifier acceptance which they might not be as familiar with.

Some testers only think about test cases. That’s what they were taught. That’s just about the only term they use. Other testers are a little more nuanced and recognize that there can be test conditions and test cases. Does the distinction matter? Well, the distinction between test condition and test case can be important if you use the distinction as a structuring or organizing principle for your tests. Below is a conceptual hierarchy of how these concepts are often considered:

• Condition
• Case
• Step
• Step
• Step
• Case
• Step
• Step
• Step

A test condition is defined as the objective of a test or a set of tests. This is similar to the way in which a requirement is defined as the condition or capability needed by a user to solve a problem or achieve an objective. The element of an “objective” is what ties together a test condition and a requirement. A test case is a particular variation on how the objective can be met.

You will find definitions differ in the testing world regarding what a test condition is and what a test case is. What I have provided above are definitions that are internally consistent and that, for the most part, match the salient aspects of various recognized definitions, such as those provided by the QAI, ISTQB and IEEE 829.

The test steps are what break out the case and say exactly how to execute the test case.

The reason these definitions matter is because they serve as the basis for action. I say that because a key driver for an effective test environment is that tests cannot cost more in maintenance than they return in value. So with that in mind, an efficient operating guideline is that you should not couple your test conditions and test case titles too closely to the implementation details of the product you are testing.

Important Point! Test conditions and test case titles should specify the behavior and describe the functionality.

Notice I refer to a test case title there. This is just what it sounds like: the title of the test case, without all of the steps. The title of a test case should tell you what the test is going to be looking for, similar to how a ticket summary in most bug tracking systems gives you a concise idea of what the ticket is about.

Here’s how I think about it: my tests, at a condition level, focus on the activities, not the tasks. That’s the basis of the level of granularity between a condition + case title versus a set of test steps. The activities encapsulate the behavior and that means condition-level tests are behavioral tests and are based on intent, thus being more static. Tests derived from the conditions — i.e., test cases broken down into steps — are functional tests and are based on implementation, thus potentially being more fluid. It’s those functional tests that can potentially cost more in maintenance than they return in value.

What about preconditions? What about specifying test data? To the extent that this is possible and not confusing, the test conditions should be able to specify these elements. As an example, below is one way that you could write test conditions and test case titles for specific functionality:

• An individual can add a benefit package.
• HSA
• FSA
• HRA

• An individual can swap their benefit package.
• HRA -> FSA – Allowed.
• FSA -> HRA – Allowed.
• HSA -> FRA – Not Allowed.

In the first test condition, we have the notion of adding a benefit package to an individual’s account. The test case titles very concisely indicate what possible packages can be added. The second test condition is different: it’s a swap of one benefit package for another. Note the “very concisely” part. That first test condition could have been written like this:

• An individual can add a benefit package.
• Add an HSA package.
• Add an FSA package.
• Add an HRA package.

It’s up to you as to which way you prefer. It may even differ based on the complexity of the feature you are testing or your ability to cover the overall functionality in the test condition, such that even concise test case titles are clear enough.

With these examples, you might notice that the test case titles are annotated with the result of the action. This is still high-level. With the second condition, it’s not said what “not allowed” actually means for example. But, at a glance, it should be obvious to a client and to us what is considered the acceptance criteria for adding and swapping a benefit package.

Whoa! Notice how I just slipped in that phrase “acceptance criteria” there? Here we have a set of test conditions and test case titles that are acting as acceptance criteria. Why is that? Because those test artifacts could have been derived from conversation with a business person. The tests are thus encoding the rules of what the system should be doing. Likewise, those tests can be used for further conversation as case titles are added. The point here is that test artifacts that any tester should be familiar with can be used as acceptance tests.

Now, if you live in a Behavior-Driven Development (BDD) world, you might be told by some people that the above are not acceptance tests because they’re not written in the “correct” way for an acceptance test. Well, first off, that’s not true. As I just said, those tests — assuming they are the result of or the driver of conversations about what the system should do — are acceptance tests. However, there is benefit to considering what people mean by the “correct” way to write an acceptance test. Usually you’ll hear this talked about in terms of specific structures, like Given/When/Then. So let’s consider how we can flesh out this example as acceptance tests while also introducing some key words or key phrases:

GIVEN an individual who is enrolled

AND who has a current benefit package,

WHEN they swap their benefit package,

THEN that individual only has the swapped benefit package.




Current Benefit Package
Swapped Benefit Package
Result


FSA
HSA
HSA


HRA
FSA
FSA

Notice how the key words and phrases are mirrored in the table to make sure those elements are specified. The italicized words are matched in the table. The bolded words indicate important elements as well: the context of of user doing the action (a data condition) and the action itself.

The above is an example of a generalized condition with data specified in a table. Here are a few more examples of how these constructs can be specified, where the data conditions are specified as part of the test conditions. The pre- and post-data conditions are italicized. The action states (either as pre-conditions or current actions) are bolded.

GIVEN an individual who is enrolled
AND who has an FSA,
WHEN they swap their benefit to HSA,
THEN that individual now only has an HSA.

GIVEN an individual who is enrolled
AND who has an FSA,
WHEN they add an HSA to their benefits,
THEN that individual has an HSA and FSA.

GIVEN an individual who is enrolled
AND who has an FSA,
WHEN they swap their benefit to an HSA
AND they add an HRA to their benefits,
THEN that individual has an HSA and HRA.

What these collective examples show you is a couple of ways to approach things. One way is that you can focus on concrete examples from which generalizations can either be derived or decomposed. Those examples can be provided as separate tables, as just one possible implementation. Alternatively, you can provide the examples as part of the conditions themselves.

You’ll notice, however, that the format really doesn’t matter. What makes them an acceptance test, or set of acceptance tests, is whether all of that information (1) can be talked about with business users and developers without getting into implementation details and (2) can be executed against a working application.

Let’s consider another example, but with a slightly different type of functionality. Assume we have the start of our acceptance criteria in the form of a business need:

The system must allow someone to enroll with an SSN but has to protect against identity theft and mistaken benefit applications which could happen with multiple SSNs.

Now let’s flesh out that business need (requirement, if you prefer) into a series of tests.

Test Condition: Allow one active and unique SSN per employer.
Case Title: Enroll user with new/unique SSN; allow enrollment.
Case Title: Enroll same user with same SSN in same employer; do not allow enrollment.
Case Title: Enroll same user with same SSN in two different employers; allow enrollment.

Test Condition: Disallow multiple active SSNs for the same employer.
Case Title: Enroll user with duplicate SSN, same employer, status is inactive; allow enrollment.
Case Title: Enroll user with duplicate SSN, same employer, status is active; do not allow enrollment.
Case Title: Enroll user with duplicate SSN, different employer, status is inactive; allow enrollment.
Case Title: Enroll user with duplicate SSN, different employer, status is active; allow enrollment.

Test Condition: Prevent duplicate enrollments from setting up multiple accounts.

Notice how that last test condition doesn’t have test cases? Why not? Because it isn’t quite enough for me to write tests, at least with certainty. Does this apply to single employers only? Does the active/inactive status of the account matter? It sounds like this test condition will allow duplicate enrollments, just not duplicate accounts. But if duplicate enrollments are allowed then what happens with those tests above that do not allow duplicate enrollment? Which rule kicks in first? There is ambiguity there.

All of those questions would have to be discussed, answers discovered, and then encoded in tests. That is how these tests become acceptance tests.

To the extent that the conditions and cases can be tied immediately and directly to the business need and/or requirement, the need for traceability goes down to some extent because test reviews mean looking at the requirement and making sure the tests are in line with it. Are those acceptance tests? They can be. Just because they’re not written in Given/When/Then format does not rule them out as acceptance tests. Conversely, just because something is written in Given/When/Then format does not automatically make it an acceptance test.

Important Point! What determines an acceptance test is how it is derived and how it is used, not how it is structured.

Clearly, I could have written some of the above conditions plus case titles like this:

GIVEN an individual,
WHEN that individual is enrolling
AND their SSN is unique in the system,
THEN allow enrollment.

GIVEN an individual,
WHEN that individual is enrolling
AND their SSN is not unique in the system,
THEN do not allow enrollment.

The question now is: what’s most useful? What’s most useful is whatever works for you. When I’m choosing the basis of making my tests into acceptance tests, I’m not asking about their structure so much as I’m following an operating principle I have. My operating principle (with any testing that ties itself into acceptance) boils down to this: what’s a way that I can write tests such that I adhere to a single source test design pattern.

What does that mean? It means that information should be stored in one place and one place only. One key operational question of this is asking “Is this information already captured elsewhere that I could simply reference?” Think about how most testers are in environments where they have requirements of some form and then take the information from the requirements and make tests out of those. That’s two artifacts, each repeating some of the information. That violates my operating principle single source information.

But if you have my tests acting as requirements specifications, then I have a single source design pattern. In that case, it doesn’t matter what format my tests take, it simply matters whether they are acting as something else — requirements — as well as tests.

So are my tests fit to be called acceptance tests? Well, to answer that I’m always asking myself: What keeps the test most associated with the requirements but also keeps the requirements themselves readable? The reason that matters is because the requirements should be the result of (or further driver of) conversations about what the application should do. Those requirements should drive the design of a specific implementation. That’s exactly what my tests should also be doing if they are acceptance tests.