Testing Is The Art Of …

The title of this post indicates a sentiment you’ll often see. “Testing is the art of …” and then fill in some word or phrase. While I get the intent behind this, the word or phrase used to complete the sentence is often a bit lacking and actually reinforces the opinion that many have of testing, which is that it’s not a distinct enough discipline.

Many years ago I wrote Testing: Art and Science. At that point I was arguing against those who treated testing as either one of the two, but not both. Here I’m talking more about how art or science are used as a framing device for testing.

An Example of the Problem

Let’s consider just one relatively common sentiment, and one that I’ve seen making the rounds on LinkedIn conversations:

“Testing is the art of asking questions and seeking answers.”

This is usually a bastardized form of a more common phrase: “Science begins by asking questions and then seeking answers.” Notice how those are two very different thoughts. One talks about what something is while the other talks about how something operates. And that’s a crucial distinction.

I’m not here to condemn the process of coming up with catchphrases that you hope resonate with people. After all, I try to do that myself quite often. But I do want to call attention to a reality which is that phrases like this get quoted by test practitioners as if they were actually meaningful when, sometimes, they’re not. It’s not that the phrases are wrong, per se. It’s simply that the simplification they present distorts the reality and that distortion is often what marginalizes the discipline.

The problem in this case is that this particular definition could apply to many things. “Interviewing is the art of asking questions and seeking answers,” for example. The same would apply to a data scientist, in many ways. They ask questions of data and seek answers from it. A good case could be made for this being the basis of much of what a lawyer does (along, of course, with persuasive argumentation).

You Might Actually Be Quoted

I do think that, as specialist testers, we need to frame what we say as if it might be quoted and even quoted widely. And further that it might be quoted divorced from any other context we had put around it, much like the above phrase was for me when I kept seeing it popping up in conversations.

Let’s take that phrase at face value for a second. It in no way invites someone to ask questions about it because it tries to wrap itself up nicely as a fully self-realized (and perhaps self-evident) phrase. But if we unpack it a bit, let’s consider what this really means.

A specialist tester has to talk to experts in their various fields, such as business and developers. This specialist tester often has to talk to those folks about the deepest, most interesting parts of their problems. This kind of “talking” is not easy, though, and requires practice and experience. The necessary skills include, at minimum, how to keep a conversation moving along and on track, how to elicit meaningful responses, how to revise questions based on responses, and how to interpret both subtle cues, vague responses as well as detailed responses.

Is all of that implied in the above catchphrase? Perhaps. But my point is that the phrasing itself doesn’t invite exploration enough to get to that point. It seems rather … well, banal, to be honest.

So What’s Your Idea?

If I had to choose somewhere to start, I would rather focus on this:

Testing is the science of applied investigation, experimentation, and exploration coupled with the art of figuring out how, when, and why humans make errors when building complex things.

Admittedly it’s not as catchy. But I think it does have the benefit of being accurate while also giving a true taste of what the specialist discipline is and how it operates, particularly in a unique manner from other disciplines. I do believe it’s interesting enough to invite people to ask a series of probing questions.

Good for you. Does this really matter?

Well, the problem is that testing often finds itself in a position of having to defend its existence. Or resist pressures to be conflated entirely into development or delegated solely to an automated tool.

What often acts as a counter-pressure to that is being able to show that what specialist testing “does” is something that is unique in its own right; it’s a discipline. It’s a discipline with a very broad lens rooted in — at minimum — science and psychology. It has a technical basis rooted in a social context and is used as a way to enable decisions.

If someone can figure out a viable catchphrase for that, I’m all for it. But, in the meantime, if we’re going to call testing an art or a science, let’s at least give people a small taste of what that might actually mean and invite further discussion.


About Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.
This entry was posted in Testing. Bookmark the permalink.

2 Responses to Testing Is The Art Of …

  1. “Testing is applied epistemology,” say Cem Kaner, James Bach, and Bret Pettichord in their Lessons Learned in Software Testing.

    • Jeff Nyman says:

      But that’s not really unique or informative, in my opinion. Science itself is applied epistemology. You could certainly make the same claim for the practice of law or, if you are feeling perverse, even the practice of politics.

      The term “applied epistemology” does have a long history. It basically means “the study of whether systems of investigation that purport to be seeking the truth are well engineered to lead to true beliefs about the world.” That comes from Larry Laudan in Truth, Error, and Criminal Law, An Essay in Legal Epistemology. Likewise, the idea of critical thinking and applied epistemology have long been considered linked, as this paper shows. And, of course, “information studies” say they are also a means of applied epistemology for understanding.

      So it’s a term that already has a lot of baggage with it and can be said of many disciplines, to greater or lesser degrees. The idea of applied epistemology is certainly an aspect of testing, to be sure. But it doesn’t cover the “art” part of things necessarily. And there is an art as well as a science. And not all of it is about “truth.” Or, rather, it’s about a flexible and morphing notion of truth. Quality, after all, is a viewpoint.

Leave a Reply

Your email address will not be published. Required fields are marked *