An Ode to Testability, Part 1

I talked before about whether testers should be developers and I also talked about the intersection of development and testing with a focus on testability. Here I want to make that intersection a little more actionable by considering an example over a series of posts.

Bringing Together Ideas

This will be a series of posts for developers and testers. Here I want to reinforce the idea that testers are developers, but not necessarily developers with a programmatic focus. What I mean by “developer” is any member of a team whose job it is to deliver value to users. Some of those developers will be focused on articulating business need and value; others will be focused on the ability to craft production code; yet others will be focused on the core principles of testing.

I often talk about testing as a design activity and testing as an execution activity. I’ll be the first to admit that these are fluid aspects; where design leaves off and execution begins can be a little amorphous. I tell people — particularly testers — to be okay with that. The key thing we all want, as being part of a delivery team, is to handle external and internal qualities. We want to do that by putting pressure on design at different levels of abstraction. Sometimes those levels of abstraction are fully human; other times fully tech. More often than not, they are at the intersection of human and tech.

In this series of posts, I want to try to make all of that somewhat actionable in your mind and the best way I can think of to do that is work through an example. This is an example that will be based on a business need (to be talked about in this post) and an implementation of that need (to be talked about in the future posts in this series).

As we’re going through all this together, please keep in mind the idea of design pressure and testability at all points. But also think about those things consistent with the value that is being provided. Don’t worry: I’ll remind you of this along the way.

Design Pressure: Value

Business Context (As a who…)

Our business context here will be the domain of archaeology. In this context, a site of some sort will be found. Archaeologists will want to set up a dig. Many archaeologists work for universities, but some work for museums and some are entirely independent, effectively working as freelancers. What’s common to all of them is the need for funding. This funding will be predicated on a budget that the organization supporting the dig knows can be sustained given the details of a particular archaeological site and the various teams that might be called upon to work it.

Working these sites has time and cost pressures. So teams that can minimize time for certain activities but while keeping cost thresholds down — all while being professional about how to work the site, of course — are highly prized. As an organization funding a dig, these things matter.

Business Context (I want what…)

The goal here is software that allows the head of an archaeological team — which could be the organization that funds the team — to set up their dig, composed of a series of activities to be done. Those activities will have associated costs. So this application should maintain a list of activities for the team. For each activity, it’s important to maintain a status of the activity, with the status simply being whether the activity is finished or not.

Business Context (So that why…)

The pace at which a team can do these activities matters, particularly given the costs of those activities. Teams who are quicker tend to be prized a bit more. This is simply because that tends to keep certain costs down and also means the team will then be available that much quicker for other sites. So the funding organizations are really interested in those kinds of teams.


Testers! What do you think about the above? I clearly followed a feature injection style there (“As a who, I want what, so that why”).

But you might notice how I vacillated a bit there. I talked about the software being useful for archaeological teams but I also seemed to talk about the software in the context of the organizations who fund and monitor those teams. Yet, the value to both seems to coincide. Do you agree? Do you disagree? Whether you agree or disagree, do you see a problem?

Asking Questions Is Testing

There’s not necessarily a right answer to that last question. But asking those questions is a way testing, acting as a pressure on design (here at the business level), will help teams determine if we are all talking about the scope of our activities in an effective and efficient way. All testing is ultimately derived from some definition of scope. Getting that as settled as possible as early as possible is one way that specialist testers add value.

Discovery Turns to Elaboration

What we’re doing here is getting into that sometimes nebulous realm where “discovery” (the above) starts to turn into “elaboration.” That’s where we start thinking about how our ideas can be turned into something real that can provide value. When we get into elaboration, it’s important that the “testing as design activity” take on the idea that if we care enough to elaborate it, we care enough to validate it. And in order to validate anything, it needs to be testable. And in order to be testable, we have to be able to observe it and control it.

This is where our business context turns into business intent. Note: not implementation; intent.

How tight the feedback loop is between stating intent and creating implementation really depends. It’s all up to the delivery team. Obviously the shorter the feedback loop, the more tolerant you are to mistakes because you find them about as close to the time you introduce them as you can get. But this does mean you have to work on very small bits of scope.

Business Intent

There’s a crucial point here for our software that the business wants to make sure we understand.

The rate of activities being finished really matters because that has to do with how effective a team is perceived as being. And this can definitely impact the selection process when choosing which team to send to a site. This refers to the pace that the archaeological team can work at. So we want to use the rate of activities being finished — the team’s pace — to estimate the date when the dig itself will be finished.

What this is basically asking for is logic that will forecast a dig’s progress. To forecast means to predict and here we want to be able to predict when a dig will likely end. That can help a team determine whether that dig is on schedule. Sometimes that schedule might have pressures such as being given limited time at a site or due to the fact that the nature of the site will be released to the media within a certain time frame. Thus there will be an ideal finish date for a dig.

So, given a dig and a set of activities, some of which will be finished and some of which will not, we want to use the rate at which activities are being finished to estimate the end date of the dig. That will also allow people to compare that projected date to some deadline to determine if the dig is on time or not.

Testers! I would recommend keeping some of this stuff in mind. We’re going to go through some implementations in the following posts and it might help for you keep in mind the statements of value and see how those align with correctness.

Business Understanding

Notice something subtle here as we worked on value-based design-pressure. We shifted from a business context to a business intent which provided us with business understanding. We talked about what the software would do immediately (for a team on a dig site) but we also realize this has an impact on what the software produces: data that can be used in the future — to determine how dig teams have done in the past, thus enabling decisions of what teams to consider for future dig opportunities.

The main thing to notice here is that we have consistency between our discovery and our elaboration. And we didn’t need Jira, Trello, PivotalTracker or anything else to help us determine that. We entirely stepped away from tools in this post and used our skills to talk about a problem and — crucially — reflect on how we were talking about that problem.

What we’ve done here is make sure that the value we are talking about is empirical. Put another way, it’s testable. We have forced testability into the discussion of what we are talking about and what purpose it will serve. Knowing those things gives us an understanding of why people will value it. And knowing why they will value it, we can start to think about what specifically they will value. And that, ultimately, will get us thinking about how they can value it and, thus, what we have to provide.

Project: Benchmarker

We’ll call this application of ours Benchmarker.

In excavation terms, a benchmark refers to a specific point at a known elevation; a point that can be used to measure other elevations. The word more generally means a way to indicate a standard reference point by which to compare something. Let’s take a moment to congratulate ourselves on the clever naming of our project.

And now let’s talk about the code for a second.

The Code Project

I am going to use Ruby code in the following posts. I’m doing that because, in my opinion, Ruby is still one of the best languages out there for code clarity, including most particularly in its test solutions. That said, the code will be simple enough that knowledge of any programming language at all should be more than enough to let you follow along.

And, in fact, I’m willing to bet non-coders can read the small amounts of code well enough to get the point. In fact, that is a large part of the point. We get into this notion of “technical testers” in our industry and assume that has to do with being able to program. But that’s not true. Being “technical” in a testing sense does not connote any ability to program at all necessarily. That said, I do think it’s true that part of being a technical tester is being able to reason about code.

Now, all this said, my goal is not actually to have you code along with me here, but rather to see how testing as a design activity and testing as an execution activity can intersect.

This will also let us explore test-driven development a bit, which some people — even some experienced developers or development managers who should know better — dismiss as being a waste of time. That view is usually predicated upon just thinking of testing as an execution activity rather than also viewing it as a design activity. So along the way I hope to make that distinction clear.

Coding Along With the Posts

While I don’t expect you to write code along with me, you certainly could. I do have a code repository for Benchmarker that you can look at. That shows the completed project. You can look at the tagged releases. Each tag corresponds to the post part. So tag v1.0 is this post, Part 1. Tag v2.0 will be Part 2 and so on. You could just download the respective zip file for whatever place you want to start at. Alternatively, just clone the repo and check out the tag called “v1.0”.

To do all this, you do need to have Ruby installed on your operating system of choice, along with Bundler. The Benchmarker project does have a Gemfile to get you the dependencies needed, of which there are only two.

But again, while I do absolutely want you to be able to focus on the code, I don’t necessarily need you to distract yourself with the coding. So don’t worry about all this unless you feel not coding along with me will compromise your ability to work through these posts.

What’s Next?

Here we talked about design pressure on value. In the next post, we’ll get into design pressure on correctness as we work to taking our business intent and turning it into business implementation.


This article was written by Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.