I talked before about tradition and dogma and not too long ago, on LinkedIn, I saw someone post yet another one of those bits of dogma in our industry without considering the context. The discussion that ensued showcased exactly the problem with simply regurgitating the “received wisdom” of others. So let’s talk about this.

Before getting started here, I’ll note that awhile back I talked about the danger of the technocrat tester. What I’m talking about this time is actually a much more pernicious problem and it’s one that’s particularly dangerous in an industry that is conflating terms, roles, and disciplines. But I do think that both articles reinforce each other to some extent.

The Context

Okay, so first of all, the regurgitated quote that started the discussion:

“Quality is achieved by putting development and testing into a blender and mixing them until one is indistinguishable from the other.” How Google Tests Software

The Response

My response to this is that this is a perfect example of using a quote to distort a more nuanced reality. If you read the entire book or read Google’s blog posts from the time the book was written or subsequent to it, you realize that’s not literally what they do. Here’s one example of another quote from Google:

“At Google, this is exactly our goal: to merge development and testing so that you cannot do one without the other.”

Note that this doesn’t mean development and testing are “indistinguishable” as the first quote suggests. It means they are done concurrently and in the same context. And for something to be done concurrent with another, it must be distinguishable, by definition.

So, yeah, quotes can sound good but it’s important to realize when the context behind that quote is operative. For example, Google also says:

“Testing must be an unavoidable aspect of development, and the marriage of development and testing is where quality is achieved.”

Again, not indistinguishable aspects but, rather, interleaved aspects. A “marriage” does not make two things indistinguishable from each other; it (hopefully) puts them in cooperation (rather than opposition), allowing for mutually reinforcing strengths.

Quoting various things often means more with the weight of evidence. In other words, the more quotes you have, the more you can start to determine how someone, or some group, seems to think. So consider another quote from Google:

“We solve this problem by having two types of testing roles at Google to solve two very different testing problems.”

Yet another:

“Modern testing is about optimizing the places where developers test and where testers test. Getting that mix right means a great product.”

The point here is that Google has always recognized the separable nature as well as the fact that these functions (as opposed to roles), definitionally, are not indistinguishable. What they do say is:

“The simple solution to this conundrum is to stop treating development and test as separate disciplines.”

That’s different than saying a “separate activity” or a “separate function.” They are using this in the sense of a branch of knowledge or a field of study. That is the lack of separation they talk about.

But … The Response to What?

So why was I throwing all these quotes back at the person who posted the dogma? Well, I think the initial lesson should be: don’t just repeat quotes, especially hyperbolic quotes. Understand the context. But, more to the point, in bringing this up to the original poster, the following comment was made. And I provide this quote (accurately but anonymously) because, in various language, this is exactly the type of response that is normally given.

“I think making development and testing indistinguishable is the goal. Having development and test be separate things produces lower quality in my opinion. You don’t build something and then test it. Building it and testing it are the same activity.”

This person was using the initial Google quote as supportive of this entire belief. But think about it. By definition “building it” and “testing it” cannot be the same activity. That is the dangerous part of this view, in my opinion.

Relation should not imply conflation. Muddy thinkers routinely fall prey to that mistake. And that was my point about taking some other quotes from Google. While one quote seemed to imply the “blending”, all of their other quotes from how they do things show they recognize the danger of the conflation. What Google has done is implemented an approach whereby the relation is encapsulated in one person or one group, in this case developers. The developer acts as a developer in some cases (“build it”) and as a tester in others (“test it”). Two different things: not the same activity.

I should note that whether people agree with the developer-as-tester approach is, of course, a different issue entirely than whether or not the two roles should be “indistinguishable.”

Discussion Continued

The person in question, responding in particular to my admittedly charged use of the word ‘dangerous’, replied:

“I don’t find it dangerous at all. Instead, I find it essential. The testing and development happen simultaneously as the same activity. If one uses test driven development for their small tests then it is impossible to separate them. Also, the medium tests are built out as part of development and not as a separate activity. If you read the entire book this is what they describe. This is also my experience.”

And my response is that, ontologically, what’s being described here is not possible. And I would argue testers deal with ontology all the time; it’s one of the core skills we often have that distinguishes our thinking from that of developers. Now, being fair, what is said there is ontologically possible — if you redefine your terms entirely, of course. But then you may as well just use one term rather than two if you feel both terms describe the exact same thing.

But here’s the problem with that kind of thinking. TDD does not at all imply testing and development are indistinguishable (which is what we started with) but it does imply they are separable. Incidentally, notice how easy it was to switch from one word (indistinguishable) to another (inseparable) and treat them almost as the same? That’s what the original poster was doing in this conversation and it’s a key flaw that I see often: again, relation becomes conflation. Muddy thinking.

And, in fact, this aspect of separation with TDD is a large part of the point: it’s test DRIVEN development. It is development (an activity) driven by a test (an artifact). And therein lies the danger of the conflation. TEST is an artifact; TESTING is an activity. And testing, as an activity, extends before code is even written and even before we’re even deciding if we should write code.

Dangerous Conflation

This conflation that we’re seeing here — even with “indistinguishable” morphing into “inseparable” — leads to three errors of reasoning in our industry right now: muddy thinking, insufficient category distinction, and descriptive parallax. That is the danger I was referring to. I wasn’t so much referring to the “danger” of developers doing testing; that’s the entirely different issue I mentioned.

Finally, I should note that the the notion of simultaneity, as posited by my conversation companion, is not true in this context any more than the suggestion that simultaneity happens in the real world. It doesn’t even though it looks like it does. What you have are two aspects (development-thinking, test-thinking) taking place concurrently and interleaved to greater or lesser extents.

Testing, as activity, is always putting pressure on design. Some of that design is speaking to code. Other times it is not. Conflation often hides that fact.

Muddy Thinking? Too Harsh?

Well, I call it that because it’s important to notice the danger — but also the opportunity. If you take the responses to me (which have been reproduced accurately) they basically say this:

• “I think making development and testing indistinguishable is the goal.”
• “The testing and development happen simultaneously as the same activity.”
• “If one uses test driven development for their small tests then it is impossible to separate them.”

Bolded emphasis is mine. Notice three terms: indistinguishable, simultaneous, and inseparable (“impossible to separate”).

For something to be described as simultaneous, there have to be more than one thing so “indistinguishable” is no longer accurate. Likewise, for simultaneous to make sense, the ‘more than one thing’ you are dealing with, by definition, has to be separable. It’s only by its being separable so that you can decide if the components are simultaneous or not. Conflation leads to muddy thinking; or muddy thinking leads to conflation. Take your pick. Either way, it’s poor thinking. And it’s this kind of thinking that is occurring a lot in the overall discipline of software engineering.

That’s the problem. What’s the opportunity bit I mentioned?

The ability to point things like this out is often the skill that specialist testers are bringing over and above pure development activities. We’re clearing up muddy thinking; the inaccurate, or conflicting, use of terms; the unpacking of suitcase words; the removal of semantic nests where one word is used to describe multiple things or where multiple words are used to describe the same thing.