Early on I talked about business needs becoming specs that become code. More recently, in my modern testing posts, I talked about the idea of production code being the specification of behavior. I wasn’t necessarily very descriptive in all of that, however. Let’s see if I can do better here.
So here’s a contention I’ll throw out there right away: the only specification that has the most empirical and demonstrable meaning is the application itself. We can all talk about creating “executable specifications” or “living documentation” or whatever else. Yet it is beyond doubt that the application, with its running source code, is the executable specification.
The code is the means by which the application does what it does and thus is the specification of functionality. Any “feature files” or “test cases” you have are simply tests of that behavior. Or, rather, checks of that behavior if adhere to a strict test-vs-check distinction.
Let’s Take an Example
Rather than come up with my own example here, I’m going to borrow one from the book Rails 4 Test Prescriptions. I’m doing this for a few reasons. In part I do this because I want to promote what I believe is an excellent book. This is a book that many who don’t work with Ruby would never read. Even those working in Ruby, but not with Rails, might not read this book.
I also do this because I think it’s important for testers to read books like these and glean what they can from it. This book showcases something interesting, which is that while the book does a good job of describing some test-focused aspects, it “dilutes” that a bit in also discussing the mechanics of tools like RSpec or Rails itself. I say “dilutes” in quotes because in fact the book is very true to its focus all the way through.
Beyond that, I wanted to present some code for this post and my desire was to use a language like Ruby because of how clean it is. As a clean language, it’s going to help me make a few points. That said, anything I talk about here can apply to any language. Also, you don’t need Ruby knowledge to understand this. All examples will be explained.
So let’s say we’ve got a meeting going with a business person, a developer and a tester. The business starts off the discussion:
“So our new application is going to be made up of projects and tasks. Users will be able to create a project and add tasks to it.”
- As a tester you are no doubt thinking: What’s the simplest possible thing I could test right now?
- As a developer you are no doubt thinking: What’s the simplest possible thing I could build right now?
Or, if you are a test-first sort of developer, you are thinking the exact same thing as the tester, interleaving testing with building. And that’s kind of an important point here. It’s a point that I find is often not considered, even in this current era of “technical testers” or “SDETs”.
Both the tester and developer are (hopefully) basically thinking this: “I want to be able to create a project.” And since both know that tasks can be added to a project, as per the business statement, a logical question from both developer and tester might be: “Are there any default tasks?”
The business answers that: “Any newly created project would have no tasks.”
So, tester and developer ask: “What can I do with this empty project beyond adding tasks to it?”
Business: “Nothing. If there are no outstanding tasks, then there’s nothing to do.”
But What’s the Point?
Realistically the questions might not be this staccato but I’m doing this to illustrate a point which is that you want to make sure at some point fairly early that you have an answer to this: “Why would this — all these projects and tasks — have relevance at all?”
The business might say: “Well, given a project with a set of tasks — where some of the tasks are done and some are not — we want to use the rate at which tasks are being completed to estimate the project’s end date. Then we want to compare that projected date to a deadline to determine if the project is on time.”
Ah. Interesting. But, still, what are we doing here? What problem are we solving?
Business: “We’re basically forecasting a project’s progress. Our goal is to be able to predict the end date of a project and determine whether that project is on schedule or not.”
This is pretty critical stuff, right? We need to know why we are doing something because that will help us better understand what questions to ask. It will also help us avoid going down too many paths that are not value-adding.
So … Let’s Design?
Well, let’s consider what we ended up with in terms of statements from business:
- “Any newly created project would have no tasks.”
- “If there are no outstanding tasks, then there’s nothing to do.”
But the developer and tester have also both been told a lot about possible features. It is VERY easy to start designing those features right now if you are a developer. As a tester it’s VERY easy to start asking all sorts of questions about projects, tasks, schedules, time, etc when, in fact, you are not even sure of the basics yet.
A key point there: both testers and developers can fall into this exact same trap.
TDD (and BDD?) Comes In
So as a developer and a tester you might be thinking: “Well, what does this thing look like? What does initializing a project object look like?
What could I test for to know that the project is created and initialized correctly?”
Notice I’m mixing a bit of development and test language there and that is on purpose.
As the Test Prescriptions book states, initializing objects can be a good starting place for a TDD process. In other words, you specify the initialization state of the objects under test. This makes sense both from a “pure” development standpoint as well as a testing standpoint.
There is another approach which is to design the “happy path” first. This would include initializations but also include a representative example (and usually just one) of the problem-free version of what you are dealing with. Put another way, this is what you want a successful interaction of the feature to look like.
Both are viable. Which approach you take may depend upon how complicated the feature is. If you know nothing about the feature, beyond the tidbits you got so far, you can almost assume the complication is infinite. Because, in effect, it might be until you start to nail down specifics. So is there a heuristic here? If the feature is sufficiently complex — yes, “sufficiently” is up for interpretation — you can start with the initial state and move to the happy path.
I will say here, but not prove, that this is a conversation that can be very muddied if you go the “pure” BDD route. I would argue that what we’re doing so far, and what I’ll be showing you, is BDD.
Put Pressure on the Design
So let’s answer that question: “What could I test for to know that the project is created and initialized correctly?”
Business: “Any project with nothing left to do in it is done.”
Okay, so the initial state, then, is a project with no tasks. And such a project can be checked to see if it’s “done.”
Here’s what an initial test spec might look like:
RSpec.describe 'a project' do
it 'that has no tasks is done' do
project = Project.new
Notice how the test code is now a spec along with the production code. Yes, even at the “unit” level. In fact, realistically, I don’t want to get hung up on whether we’re doing “unit tests” or something else right now.
A pure BDD approach might argue with coming up with a feature file first. Something like this:
SCENARIO: A project that has no tasks is done GIVEN the application WHEN a new project is created THEN the project is "done"
A contention I have is that you can often achieve the same goal with a programming language that favors a syntax that is close to natural language as possible. And, if that’s not possible given your language of choice, you can put a focus on crafting a DSL in any language.
Here’s what this small bit of logic has told us in terms of our design:
- There is a domain concept called Project.
- You can query instances of that domain model as to whether they are done.
- A brand new instance of a Project qualifies as done.
This also guides questions or ideas. Maybe someone argues that an empty project can’t be “done.” Maybe the idea of “done” should mean that there must be at least one completed task. Maybe an empty project with no tasks at all should be considered “not started” or “empty”. All of that is good: that’s what you want to be discussing to make sure your domain model — and its terminology — is understood.
From a “test tool” perspective, all we’ve done here is create an object and assert an initial condition. Or, rather, provided an expectation of what should happen. The point is that we are starting to define the way parts of our domain, which our application models, communicate with each other. The tests are helping to reflect the visibility of information about that design that we currently consider important enough to specify.
So what we’ve done here is document a small part of how our Project domain concept behaves. The fact that this is encoded in an executable test means that we will find out immediately if we have broken that behavior.
A Shift in Thinking
There is a subtle conceptual shift in thinking here. You don’t want to think of your tests as simply “regression tests” that are there to detect changes in behavior. Think of them as a form of specification that describes how the product works.
This can help you get past the idea of a “regression testing phase”. It doesn’t matter if there is such a phase or not. Your test documentation tells you how the product works. When the functionality of your product changes, the tests are updated to tell you how the new functionality works.
This means that your existing documentation for the old functionality doesn’t change unless the functionality changes. That’s an important point that often gets lost. You ideally want to be in a state where every test failure means that there is an undocumented change to the system. If that change was intended, you revert it. If that change was intended, you can update the documentation — your tests — to support it.
The ultimate goal of both BDD and TDD is to make any tests we have pass in such a way that we can best discover the solution to the problem and design our code accordingly. That’s important because there are two levels of putting pressure on design: one is at the business level (the design of the idea) and the other is at the code level (the design of the implementation that supports the idea).
This is a good place to close this post and let you digest what’s here. I will have a follow-up to this post where I’ll show how the design builds up. My hope is that this will reinforce what has been said here.
Also I’ll once again recommend the book Rails 4 Test Prescriptions. I’ve cherry-picked some things here from the book to make a more specific point. That book places those points in the context of development of an actual application.