What If: The Test Apocalypse

Here’s an idea that I think can be interesting in terms of how you view testing (as an activity) and tests (as an artifact).

I should note that the basis for my current incarnation of this idea is due to having just binge-watched three seasons of The Leftovers. While doing so, I was reminded of an idea I had about handling a tool apocalypse. That idea was basically imagining that every tool we use to support testing — at least the automation and management parts of it — were destroyed. Could we still function?

I would often present this idea to teams just to learn how much we were relying on artifact crutches in the form of tooling. This wasn’t necessarily to suggest that we should not be using those tools. It was simply to make it clear how much we do, in fact, rely on them.

The Leftovers allowed me to refine that idea somewhat. No real spoilers here, but for whomever has not watched the show it’s essentially predicated upon the idea that in one second, 140 million people all over the world disappeared. So that’s between 2.3 and 2.4 percent of the human population. They all just simply “poofed.”

So now let’s imagine our test repository with its population of tests. And let’s say some (random, as far as we know) percentage of those tests underwent a “Sudden Departure” event.

The powerful question here is: How effective and efficient would our testing still be?

This actually can be an interesting experiment to try out. Particularly if you try out the scenario multiple times, having random tests be “disappeared” in each scenario. And maybe it’s not so random. Maybe you introduce something that selects tests to remove based on some criteria, such as tests that are “too short” or tagged a certain way or whatever else comes to mind.

This kind of “what if” scenario can also give you discussion points around the level of detail in tests. For example, if you have a series of workflow tests that may duplicate some aspects but tend to cover wide areas of end-to-end functionality, does this mean your test effort will “survive” better even if more of those kinds of tests are removed? Or is the opposite true? You can also consider this scenario across the scope of testing, such as unit, or integration, or whatever other delimiters you use.

I recommend you challenge yourself and your teams with this kind of idea. Beyond being instructive, it can be a lot of fun.

Well, as fun as an apocalypse can be.

Share

This article was written by Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.