In my last post about what tests are for I talked a bit about the ability of test writers to be nimble in terms of getting quality-focused information — usually as a result collaborative discussions — encoded into readable tests that focus on intent rather than implementation. All of that’s easy to say but test writers sometimes have a hard time figuring out how to exactly do it in the context of the collaboration possibilities in their environment. Here I want to talk a little about the core ideas, backing them up with an example.

First, let’s consider the business goal. While many test writers are not necessarily involved in writing such goals, they most certainly should be involved in knowing what they are. Beyond that, I would actually argue test writers should be involved in writing such goals if testing is being treated as a design activity. Note I’m not saying that test writers will be responsible for such goals; simply that they be involved in the process.

A business goal is the underlying reason for a project. It’s usually the guiding vision, if you will, that gets the business stakeholders to decide to invest money in software development for a particular feature. This is basically the first place where “Hey, I’ve got a cool idea” gets translated into something that the business can get excited about. The business — whomever or whatever that is taken to be — should be able to clearly see how a given goal can either earn, save, or protect money. Ideally, a goal should be measurable, so that the specifics of the goal can guide the implementation. A measurable goal makes it possible to ascertain whether the project succeeded, to track progress, and to prioritize better, such as deciding whether to work on this goal or that goal.

The very fact that a business goal presumably has quality-related parameters and given that this goal must ultimately be implemented in some fashion, this means that test writing is almost a given, correct? After all, if testing is to help communicate information about quality related decisions and the results of those decisions, then the most responsible moment to get some sort of testing involved is when this information and these decisions are first being made.

Here’s a really important point! Testing is a Design Activity. The above points are really important to me. Testing is a broad term. It is a design-related activity acting within the context of an engineering discipline. Testing is ultimately about being responsible. And that means doing things at the earliest responsible times. When business goals — and thus feature direction — is being set down is one of those responsible times. But to see this requires seeing testing as a communication-focused activity that fosters collaborative discussion and the encoding of quality related decisions in a form that can initially guide — and later verify — implementation.

So let’s start with a goal.

"Increase ability of trial administrators to make clinical decisions by allowing them to see all documents associated with their study."

This goal can certainly be stated in terms of how these abilities will increase sales or keep existing customers. Another goal might be:

"Increase security of the system by making sure trial administrators can only see documents associated with their study."

You can see how you might have a lot of variations on this. As you start to get more specific, you’re probably starting to get into implementation and scope. Instead, as a test writer, let’s see if can derive the implementation scope from the business goals. The implementation team and the business sponsors can come up with ideas that can be broken down into deliverable software chunks. The test writer can make sure that everyone agrees on what the chunks are and what observables will be in place to make sure the chunks were developed as intended.

Incidentally, this is probably where a spec workshop would come in. I won’t be talking about the specifics of a spec workshop here but what I’m going to be discussing certainly would fit within the context of one.

So the setup here is you have a meeting where the test writer is with the developer, a business representative, and a business analyst. There could be more people involved but, at this stage, there really shouldn’t be less.

In this meeting a theme is identified for a “content visibility system” that can be broken down into two feature sets: “document content visibility” features and “page content visibility” features. Business wants to focus on building the document content visibility first. The idea is that trial administrators will be able to see a list of documents on a page. Further, those administrators will only be able to see documents that they should be allowed to see. Here’s the scope for this example:

Scope: User stories for a document content visibility system.

• In order to allow trial administrators to see documents related to a study, as a study sponsor I want users to be able to see document blocks.
• In order to prevent clinicians from seeing confidential documents related to a study, as a study sponsor I want users to only see document blocks they have permission to see.
• In order to see a full picture of my study, as a trial administrator I want the ability to see all documents related to my study in a document block.

A key thing to note here — and that a test writer should have an eye on — is that each defined piece of this feature set has a specific user or user role associated with the business goal. Above you see “trial administrator”, “study sponsor”, and “clinician.” It’s important to have specific user roles.

So what do we do with that information? In some places, that information is just given to developers and they get to go to town implementing something that should hopefully satisfy those goals. From a testing perspective, certainly tester could produce test specifications for the appropriate scope once the development team starts implementing a particular feature. But instead of that, we could do that test specification part right away — before any development begins.

That detailed specification can be written in the form of tests and those tests can serve as input to development. For example, when we start working on the first item of the feature, “document blocks” must be defined. So during collaboration, it’s decided that the system will have pages that are made up of either trial blocks — that contain textual content about the clinical trial — or document blocks — that contain lists of documents related to studies for the clinical trial.

It’s also decided by business that each content visibility block should have its own permissions. Visibility of each block can be restricted to users who are members of specified clinical groups. All users who are members of that group will be able to see that content block, whether it be a trial block or a document block.

So, as a test writer, who is interested in getting some information encoded as tests in the form of a specification, we can identify some key examples with our audience:

• If a document block is permissioned to be limited to a given group, and if a trial administrator signs in who is part of that group, then that trial administrator should see that document block.

• If a document block is permissioned to be limited to a given group, and if a trial administrator signs in who is not part of that group, then that trial administrator should not see that document block.

We know that a document block will be displaying documents. But which documents? We know that it should show documents related to a study. But is there any categorization of documents? Should it show all documents? Or just some? What business decides, after discussion, is that the system will have two concerns: permissioning a document block and filtering a document block. Permissions are for restricting who can view the document block at all. A document block can be permissioned based on a group, which were the examples above. Now it’s been decided that a document block can also be filtered based on a study. That brings up another example:

• If the document block is permissioned to be limited to a given group and if the document block is filtered to show documents from a given study, then a trial administrator who is part of that group, but who is not associated with that study, will see the document block and the documents in it.

“Whoa, whoa, whoa!,” the business user exclaims. “A trial administrator should NOT be able to see documents that are part of a study that they are not associated with.” Okay, so the document table has to be permissioned on a study as well and not just on a group. We just learned something pretty darn important that has to be encoded as a test. We need to make it clear that the filter just restricts what shows up in a document block, regardless of permissions. So a few more key examples:

• If the document table is permissioned to be limited to a given group and if the document block is filtered to show documents from a given study, then a trial administrator who is part of that group, but not associated with that study, will see the document block but will not see the documents in it.

• If the document block is permissioned to be limited to a given group and the document block is filtered to show documents from a given study, then a trial administrator who is part of that group and who is associated with that study, will see the document block and will see the documents in it.

Do you notice something interesting about the first item above? If the document block will be shown, but no documents show up in it — should we even show the document block at all? If so, it’s going to be empty so should we just allow that to be the case? Or should some message appear in the block indicating that no documents could be shown? Should this be a different message than if the document block is simply empty because there are no documents?

Again, all of this is important to know. This is where the test writer can start asking questions because ultimately the answers to those questions will become implementation details and thus will need to be tested. Now, granted, a business analyst could have asked these questions. They could have written up a “requirements specification” and then developers could have taken that and written a “design specification” and then testers could have taken that and written up a “test specification.”

Or we could just say that the requirements are, effectively, the tests and merge the requirements and test specifications together. Since those test specifications, as we have been writing them above, show the intent of the feature, they should show enough of the desired design that developers can take those specifications and begin implementation. Why does that latter part work? Because the developers are basically being told that whatever implementation they come up with most satisfy those tests.

There are two points lurking here. One thing test writers ask is: “Okay, but then what about a technical spec? Don’t the developers have to write that? What about the specific implementation details they decide on, such as that document blocks are in an iframe and that filters will be handled by a drop-down box?” The second point that test writers ask is: “Are we sure this is really enough for developers to begin work?”

In terms of the first point, the reason developers are in spec workshops in the first place is so that they can bring up technology-focused ideas and constraints. Developers can tell you if a given design will be hard to satisfy or not. That most definitely matters from a business perspective; but the actual details — at least at this stage — do not have to be encoded in test specifications. After all, we’re focusing on the intent here.

In terms of the second point, would I would usually recommend is that the spec workshop crew refine the specification from the key examples that were generated. This refining process can, as just one example, break down the high-level examples, into concrete examples. This will allow you to create a specification document that’s self-explanatory and formatted in a way that will make it easy to (1) guide the implementation, (2) manually test the implementation, and (3) perhaps automate the validation.

Here’s an example of what you might come up with:

These examples could be done in the same spec workshop where you put the generic statements to the test, as it were, making sure that everyone agrees that the example set does adequately cover the desired business need and thus the feature. These examples, if you think about it, are really just statements of what you are going to test. In fact, with the above you’ve stated the requirements, the data conditions, and the test conditions in one artifact: a test specification.

This test specification — a self-explanatory document for multiple parties — can be used as a target for implementation and as a driver for a manual test so we can objectively measure when the implementation is done. It would ideally be stored in a repository of specifications, to become part of the living documentation of the overall product. An example would be a FitNesse wiki system or a directory structure of Cucumber feature files.

Were you to settle on a standard format for these kinds of specifications, then the specification could be used as a driver for automated testing as well. Regarding the automation, I’m not focusing on that here too much because my point was more the need for test writers to look at where their test writing activity best fits in. That’s what matters here: the test writing. That will be the input to any automated test tool. But since the tool can’t think for you, it’s best to do all that thinking beforehand.

Put another way, my focus here is on nimble test writers creating specifications as a top-level project artifact in environments that are agile or otherwise. The specification should be viable whether or not it is put into an automated testing tool. First and foremost it’s a specification. It can become an executable specification but there’s nothing that says it has to. If executable specifications are part of your testing strategy, then you will want to consider how to write specifications such that they are amenable to both manual execution and automated execution.

There’s a lot of ideas swirling around in the above material. I aimed for a certain amount of brevity which means I no doubt skimped on what some people would consider necessary detail. My hope is that the quasi-example I used gave at least the start of an idea for how a test specification of the sort I’m describing can be written incrementally as part of a collaboration exercise.

One last point I want to make is that this test specification should be encoding the shared notion of quality for this feature that everyone is agreeing to. If you agree that’s the case, then here’s a few points you can ponder:

1. You have now done a true quality assurance — as opposed to just testing — function.
2. Your “QA Team” was made up of everyone who participated in the spec workshop.
3. Your testing function was broken up among various people, not just “testers.”

That second point goes to something I brought up in a previous post regarding whether or not a separate “QA Team” — composed usually just of testers — even makes sense. The third point is a necessary outcome of the idea that testing is a business-focused, design-guiding, quality-oriented activity.