There’s a book I really enjoyed that I believe every tester should read. Actually, anybody who has to communicate should read it. The book is called Don’t Be Such a Scientist by Randy Olson. And it talks about the need to have effective means at communicating by framing issues in a way that is interesting and not just recitations of things that people have been doing or saying. Let’s talk about this. Consider this article called “Use this approach to catch the most bugs”. This article was making the rounds a bit. I even saw someone on LinkedIn describe this as a “new approach”. I show this not to belittle the viewpoint but rather just to show that I see this kind of sentiment quoted about a lot of tester articles — usually by people who are, nominally at least, within the testing discipline. Let’s be clear: what the article describes isn’t a “new approach.” Not even close. It’s just stating what people have been doing for decades. (Whether they’ve been doing it well or not is another issue.) But the fact that someone could even think that this is somehow a new approach was very interesting to me. I’m not critiquing the content of the article so much as I am suggesting that it’s framed in a way that doesn’t really motivate and then educate. In the words of Olson’s book, the article attempts to “arouse” — with the somewhat stilted title ‘Use this approach to catch the most bugs’ — but it (possibly) doesn’t “fulfill.” For example, the article states “Validate that all the requirements have 100% code coverage.” Beyond being bad advice, not to mention quite possibly impractical, this ignores the fact that you can easily have 100% code coverage and still have bugs with requirements. The notion of 100% code coverage doesn’t even guarantee you’ll catch “most bugs.” I tried to talk about that idea in ways that were framed around hidden bugs and mutants. These are examples where I try to arouse curiosity but then fulfill that by showing exactly what I mean, while also hopefully educating people on some ideas. The article I’m referencing also talks about the validation of requirements. But how about framing that as a problem? I tried my hand at this with Cars, Shapes, and Tests. I’m not saying my approach is better. What I am saying is we, as testers, should try to present information in a less dry fashion. We should try to avoid what might seem like the recitation of facts and instead try to engage people via examples.

Substance and Style

So here I’m not entirely questioning the substance of the article, but rather the style. Now, a very, very important caveat here: I am more than certain that if I look back at some articles I’ve written over the years, I could be just as guilty of what I’m describing with this example, and perhaps even more so. I’m probably still guilty of it from time to time. In fact, that “Cars, Shapes, and Tests” article of mine that I just referenced definitely suffers a bit from exactly what I’m taking about here. In fact, let me be as fair as possible here and consider some more of my blunders. I wrote The Value of Testing … Relative to Quality. Awful. Even the title is bad. Consider my Put Some Heuristics Between Your Strategy and Your Tactics. Or, rather, don’t consider it. It’s equally awful. What about my Connect Quantity to Quality article? You guessed it: awful.

Framing the Style

Back to the article for a bit, the fact that this is such an “old” message — and yet some people can treat it as a “new message” and promote it as such — is actually part of the problem we have with framing our activities in the testing discipline. The reframing part (i.e., “not being such a scientist”) is that this article could be summed up as “test at multiple levels of abstraction.” Huh? Exactly! That wording can get people wondering about what you mean. Breaking that down a bit more what we’re really talking about is testing at the times where we humans make the most mistakes when building complex things. Those times are when we talk about what to build and when we build it. So … if we put testing at those places where we are most likely to make mistakes, are we then likely to “catch the most bugs”? See how I frame it as a question rather than as a conclusion? That’s important. Arouse the curiosity, cause a little cognitive friction, and then explain (“fulfill”) what you are talking about. Going back to those coverage areas, I would often introduce those by: “What if I told you some bugs are hidden? And they stay hidden even if you have 100% code coverage?” And then I might follow up with: “But what if I told you mutants could help?” Then and only then do I start to get into the recitation of techniques.

Expand the Framing

It’s important to expand on the ideas that you have framed a bit to sustain the interest. As another example, the article makes a good point but doesn’t expand on it. The point it states: “Quality is a broad term, so you can apply it broadly throughout the software development life cycle.” Yes … but quality is not not just a broad term. Quality is a term that is both objective and subjective. Quality is a complex accumulation of viewpoints; viewpoints that can change based on circumstance, situation and even something as malleable as mood. Understanding that coupled with the above points is a key part of understanding that “catch the most bugs” can be a daunting proposition. So now we framed a question — “How can you catch the most bugs?” — by first talking about a known fact — “humans make mistakes” — coupled with a framing of that — “human make mistakes at key points in a project.” Then we got into quality a bit and suggested it’s a suitcase word and what that might mean for even asking how “bugs” would be recognized and the challenges therein.

Scientist Mode

As specialist testers, operating within a domain that has its own vocabulary, it’s easy for us to fall into “scientist” mode. We often don’t frame things in a way to engage people. Consider this quote from Olson’s book:

“A scientist goes out into nature, gathers data, comes back to the laboratory, and puts it together in order to present to the world a story about how things are. A filmmaker goes out into the world, shoots film, comes back to the editing suite, and puts it together in order to present to the world a story about how life is.”

Testers need to have those “filmmaker” skills as well, I believe, just as I still believe that there is a component of the fiction writer in good testers. There is the ability to “frame the scene,” as it were, and help people understand the human complexities that are going on that lead to problems in the first place. Why is that relevant to you as a tester? Consider this quote from the book:

“A backlash has developed against science, in disciplines ranging from evolution to global warming to mainstream medicine. An entire anti-science movement has emerged that truly does threaten our quality of life. Large groups of people are fighting against hard, cold, rational data-based science and clinical medicine, simply saying they don’t care what the science says.”

I don’t know that we have an “anti-testing” movement, per se, but we do often see a backlash of sorts against testing as a discipline, such that it gets conflated, marginalized or delegated to automation. We do still see many situations where test results are entirely ignored that then lead to more-or-less drastic public problems with services or products.

Doing and Communicating

We have the doing (objective) and we have the communicating (subjective). That communication breaks down into substance (objective) and style (subjective). When I read a lot of what testers have to say, I find that many testers are lacking the subjective components here, particularly when they are strong in the objective components. But beyond personal blogs and whatnot, I see this same thing happening on the job as well. Yet another point from the book:

“Science, from the beginning of time, has always consisted of two parts. First is the obvious part, the doing of science: the collecting of data, the testing of hypotheses, the running of experiments — all the standard stuff. But there is a second part that isn’t so immediately obvious, and that is the communicating of science.”

This goes a little bit back to what I talked about with Testing is the Art Of, meaning the idea that how we frame and communicate about our discipline as a whole does matter. And this very much translates into what we do as part of our day-to-day activities in our work. What this has to do with is storytelling, which as anyone who has told stories before knows is equal parts science (objective) and art (subjective). This is what takes you beyond the “arouse-and-fulfill” means of communication, which can only take you so far. As Olson states in the book:

“So the arouse-and-fulfill strategy has its limitations. For all you scientists out there, it’s kind of like the surface area to volume function in limiting organism size — you eventually reach a point where there’s not enough surface area for gas exchange and the organism can’t get any larger. At that point, the organism has to have a circulatory system. In other words, there has to be a different way of doing things. For communication, that different way, beyond the simple arouse-and-fulfill model, is storytelling.”

Storyteller Mode

I think testers need to embrace the idea of narrative more. To understand the fundamental aspects of storytelling and why this mode of communication works so well. Don’t let developers be the only ones telling interesting stories. Because, right now, they kind of are. Yes, we need to be testers. It’s just that sometimes we don’t have to be such testers.