I recently participated in a discussion around the idea of whether testers “own quality” in some sense. The answer to me is obvious: of course not. But an interesting discussion did occur as a result of that. This discussion led to my post about what it meant to own quality across the abstraction stack. But there’s a more systemic concern with this level of thinking that I want to tackle here.

The article that started a lot of the discussion was Testing Antipattern: Avoiding Accountability. As respectfully as I can say it, this article is another example of the muddy thinking that I’ve written about before. A bit of a charged statement, obviously, and I want to make sure that my counter-argument is seen as an argument of substance as opposed to any sort of personal attack.

So what I want do here is provide some context around the comments to that article and hopefully distribute that context to a wider group of readers. This serves two purposes. One, everyone can check the source material, as it were, for what I’m talking about. This was a public discussion with all relevant details available. Two, this gives a wider group of people a chance to tell me where I might have got it wrong.

I’ll also point out an article I wrote a long time ago (Quality Assurance – Obviously) so you can see how my thinking now is relatively in line with what I’ve been saying for some time. On the other hand, if you feel I’m wrong, at least you can rest easy knowing I’ve been consistently wrong.

Does Accountability Mean Ownership?

I feel that the article makes an unfounded assertion right at the start:

“Many testers today try simply to avoid accountability.”

Well, wait. Is that unfounded? After all, I’m pretty harsh on testers in the industry and someone could argue that my own statements are unfounded. Okay, fair enough. Let’s shelve that concern for a second. The article, following on from the above statement, says this:

“Testers should own the quality of the product.”

But this is a non sequitur, at best. Being accountable does not equate to owning quality. I would argue, and there is a great deal of historical precedent behind my argument, that testers most definitely do not and should not own quality. First of all, that implies quality (an abstract) can be owned as if it were an artifact. It isn’t and thus it can’t be.

What Exactly Are We Owning?

But let’s assume that I’m wrong and that “quality” can be owned. But … what does that mean?

The article keeps talking about testers owning “quality” — but crucially — it never defines what “quality” means. This is a common problem with people who make these assertions. They want someone to own something that they themselves can’t or won’t define.

So that’s why I feel a little more comfortable about my statement regarding the unfounded assertion I made at the start. The article is about what is allegedly a “testing antipattern” (going by the title) but then conflates “accountability” with “ownership.” But that’s not ownership of testing; rather it’s about ownership of quality. So … are we conflating “testing” with “quality”?

It’s a defocused (or muddy) message, at best.

While I’m picking on this article, there is a long trend of conflating “testing” with “quality” and it’s similar to how people have often conflated with “testing” with “quality assurance.” I’ve talked about this concept before in suggesting testing shift back to its roots and the bug hunting focus. So, again, if it seems like I’m being harsh on this particular article, I have a long history of talking about these ideas and thus giving people a chance to tell me how wrong I am.

But hold on a second here. Let’s back up. I just said that the article moves from “accountability of testers” to “ownership of quality” and I suggested that this was an inappropriate conflation. But … is it? After all, isn’t testing how we determine what quality is at any given time?

Yes, testing (activity) is about that to a large extent. But that activity is different than the role (tester) who might be doing it. Let’s put that aside for now and go back to that definition thing and the need to define your terms if you’re going to make pronouncements, particularly about what someone should own.

So here’s my view and it’s one I’ve stated regularly: quality is ultimately a complex tessellation of viewpoints within the organization and without. Quality is a shifting perception of value over time. Quality is thus partly subjective. This is quite literally indisputable.

Given that, how would any one person or team own what is partly subjective? They can’t.

It is true, of course, that quality does have some objective components and it’s perhaps fair to suggest that certain people will have more ownership of certain of those components than others. But then you have to ask if that’s even the discussion to be having.

I would argue not.

Testers As One Force of Quality

I’ve talked about testing as part of project forces. So that’s another way for me to frame this.

What I suggest is that what specialist testers should do is provide a strong influencing force by which the shifting perceptions of quality — the subjective and the objective parts — are made as transparent as possible to the delivery team. Testers need to understand that there are internal qualities and external qualities. Testers need to understand that both can degrade. Thus testers have to be able to work with a delivery team to put in place mechanisms that can show where, when, and to what extent quality is degrading.

Those mechanisms are often going to be some form of test (artifact) or testing (process), even though they might not be called as such and even though a “tester” (by role) by not be the one doing anything.

So, contrary to the article’s title, I would suggest that the true anti-pattern is treating quality as an artifact. Another anti-pattern would be assuming that one role (group, team, whatever) can be accountable for it. I think the idea of a sole group having ownership of quality is a terribly un-nuanced view of reality and, keeping with my theme, it’s a complete misunderstanding of the forces that constrain, guide, and structure projects.

My previous post on the technical abstraction stack puts some more meat on that statement so I don’t want to repeat all that but I will approach some of what I said in that post in a different way.

Lots of Quality Bits to Own

Now let’s consider another point from the article:

“The question that makes testers tremble is ‘why didn’t we catch that bug before we shipped?’. Rather than having an honest, well-reasoned answer to this question, testers spend most of their time convincing their company and the community that they don’t actually own quality.

It’s unclear that any of this is actually a statement from experience or how wide that experience actually is. But let’s leave that aside and focus on the substantive point, which is a good one, about that notion of “catching the bug before it shipped.”

This, I would argue, is where specialist testers help everyone be “accountable for quality” by reframing the question. To wit: “What’s the earliest possible time we could have caught this particular type of problem?”

This allows specialist testers to show that there are many times and places where quality degraders could have been introduced and thus caught. And some of those quality degraders will be of very particular types that may require very different methods of detection.

This also gets to the point of talking with delivery teams about a cost of mistake curve. From the time we — as a delivery team — introduce a mistake (something that degrades quality), how big is the feedback loop before we find it? That, right there, is how delivery teams can be accountable for quality: by shortening that feedback loop.

But notice this means specialist testers have to work with the delivery team to put testing at various layers of abstraction: requirements, code (creation), code (execution), architecture, infrastructure, and so on. This can involve hardware and software. This can involve third-party services or providers. This can involve aspects around legalities specific to an industry.

Think of all the aspects of quality in those different areas. That is why testers can’t “own quality” but why they do have to help put in a culture where the delivery team as a whole owns quality.

Ironically, I think the article even makes this point but doesn’t follow it to its conclusion:

“Great testers simply assess the risk of different aspects of the system, factoring in the cost to the business, the cost to find and fix bugs, the likelihood of bugs, and deliver a report showing the organization the balance of money and time for mitigating this risk, and the priority of what risk to mitigate.”

First of all, there is often no “simply” to assessing the risks of “different aspects of the system,” given all those different aspects I just mentioned above.

Beyond that, however, consider that the very act of mitigating the risk is part of the quality. But if the testers can’t be responsible for actually mitigating the risk, then they can’t fully own the quality. How can you own something that there is a major aspect of which — the actual fixing or mitigation — you don’t control? It would seem a pretty small definition of “ownership” if all you had to do was find the bugs but not fix them.

Now, you might say, “Well, testers own the finding of problem and the verifying that the fix to the problem was applied. Obviously that doesn’t include writing the code for the fix.”

Okay, but that fix was done by someone else within a certain time frame. Was that time frame too long? That’s a type of quality. Did that fix introduce technical debt? That’s a type of quality — or rather a degrader of it. Did that fix potentially degrade other qualities, internal or external?

Should the tester “own” those aspects of quality as well?

Or would it make a lot more sense to say that the ownership of quality is distributed over the delivery team?

The answer is obvious: of course it makes sense to say that the ownership of quality is distributed. This was a lesson that it has taken the industry a long time to learn (and is arguably still learning in some sectors).

Aspects of Quality

So, as I said and will continue to assert, testers can’t “own quality.” I would argue that you can’t define quality as a thing to own.

In response to the article, and on that very point I just made, I had asked people to think of “justice.” Who owns that? The person who called the police when seeing a crime? The police officer who arrested the criminal? The lawyers who present the case in a court of law? The jury who convicts the criminal? The judge who oversees the case? The jailers who make sure the criminal is away from the public? Justice has subjective (abstract; moral) and objective (tangible; legal) aspects. As such, justice is, in a sense, distributed.

So is quality. Quality is both a tangible and an abstract and part of what test specialists do is make sure that “quality assurance” is distributed.

This argument was dismissed as a “strawman.” But, again, that’s a charge that can only be lobbied if we know the definition of quality we’re dealing with. Something that, again, the article didn’t clarify.

And the reason I harp on that is because it’s important for people to realize that if you’re going to categorically suggest someone, or some group, should “own” something, you should be capable and willing to define what that something is in operationally specific terms.

While I think aspects of quality can be made operationally specific — and thus measurable and quantifiable — this occurs over a vast spectrum of activities that it would simply be impractical, if not impossible, for one role to own.

I would argue that you can’t define quality as a thing to own, at least without considering certain aspects of quality. I would further argue that the quality of those aspects are likely better owned by specialists in that area.

For example, the quality of a marketing campaign to get more engagement from leads and then the quality of sales to turn those leads into customers is very different from the quality of a feature that those customers might use and get value from which is in turn different from the infrastructure we choose to support our architecture that supplies the platform that delivers those features.

See what I mean? Quality, like justice, is kind a big thing to own if you don’t get specific. It’s also a very small thing to own if you only treat quality related to “code that must be fixed.”

Focus or Ownership?

Assuming quality is the “focus” of testers can lead some to assume it’s not the focus of others, since “owning” tends to imply “sole focus.” Ridiculous? Well, the article seems to suggest that as well:

“…if there isn’t anyone in a software team solely focused on quality, then no one really owns quality and it doesn’t get the proper attention.”

This is, to me, demonstrably wrong and I can literally point to every single place I’ve ever worked for experiential data points to back me up on that. And I’m fairly sure most others reading this could as well. (If not, I’d be curious to hear your experiences.)

Developers should be focused on quality. So should business people. So should marketing. So should sales. So should operations. Quality is how we make sure refactoring is done well and how we reduce technical debt and keep design cheap. Quality is how a business defines delivering value and how it determines which experiences do and don’t do so. Quality is ultimately a competitive advantage which is an aspect a business, as a whole, has to own as it engages with its competition in the marketplace.

So are we truly saying only one role or one team “owns” that? Really?

Of course not.

Owning Aspects of Quality

All this being said, I’ll return to something I hinted at above. I do think it’s possible to be a bit more nuanced here. Test specialists can “own” aspects of quality, such as, for example, making sure that those aspects are as visible as possible to the delivery team. Sure, anyone could technically own that aspect but test specialists would potentially be a good choice for spearheading that effort.

Consider the idea of sprint grooming, spec workshops or the domain exploration meetings. Who owns those? The business team? The developers? The test specialists?

The test specialists should ideally be a stabilizing force while the business team would be the initiating force. Developers would potentially be a guiding force. But does anyone “own” that spec workshop process? I would argue not really. Yet the spec workshop is how we often define the shared understanding of what we mean by quality for a given feature.

Consider the “agile process” where you have teams following sprints and there are scrum masters that facilitate teams as they go through those sprints. That’s an aspect of internal quality. Do testers “own” that as well?

It’s tangibles like this that I think need to be discussed more rather than universal absolutes like the article I’m arguing against.

The Conflation Shift

Along with all of this, it’s important not to conflate things. One of the responses to my points in the comments of the article was this:

“They [testers] should own it. They are best suited to own it, it’s their focus. If they can’t influence this they should be in a different role.”

Notice how we shifted from “own it” to “influence it.” Those are two very different things. You can influence without owning. And, in fact, I would argue that’s exactly the point. Test specialists may have an outsized influence on a quality focus and they should, in my view. But part of that influence is making sure everyone knows that different aspects of quality are “owned” by different people, often with specific specialities.

The responsibility for quality is distributed due to the very fact that those specialties will all have insights on how quality is being enhanced or degraded. What a test specialist can do is help delivery teams harness those insights. This would certainly be a form of accountability but without the requirement of ownership.

While testers do not and should not “own quality,” as I hope I’ve made clear, testers should be huge influencers around how quality is perceived at a given moment in time, how it is encoded as a shared understanding, and the experiments that are put in place across abstraction levels (from business to code) to verify, falsify, and implausify.

Industry Precedent

The shift from testers having an outsized influence to instead having ownership is what led to a lot of problems in the industry back in the late 1980s and early 1990s. It’s how we had the whole “Quality Gatekeeper” mentality that led to so many problems.

After we got out of that particular trench, the industry started to then shift too much in the other direction. We came up with SDETs and said the quality is now “owned” by the developers. They just have to be “developers that know how to test.” Then the industry started to shift again, just slightly, to “testers that know how to develop.”

But the key fallacy here was any one group — whomever they actually are and whatever they are called — should be “owning quality.”

Shifting the roles didn’t solve the problem; it just … well, it just shifted the problem. Conflating the roles didn’t solve the problem; it just made the problems more opaque. But the key systemic issue was still in place. Quality wasn’t distributed. In part this is how we got the automation technocracy (“testing turned into a programming problem”). It’s how we are now getting into the algorithm technocracy (“testing turned into an algorithm”).

That last point is important when you consider that the author of the article I’m arguing against has said: “The thing that testers don’t realize is that AI is perfectly suited to replace testing activities.”

So … would the AI “own quality” then?

Or how “The AI testing singularity is the moment software can test itself without human intervention.”

So if there is no “human intervention,” how are the humans owning the quality that testing is presumably informing them about?

All of what I’ve just said here is an important bit of history, both past and present to understand. We have people literally advocating for the very same mistakes we’ve made in the past. Knowing this history is important! As I’ve often argued, testing is very much about thinking like a historian.

The Impossible Via the Impractical?

A final point from the article:

“Testers fear getting in trouble with promotions or even being let go if they accept ownership of quality.”

Again, another absolutist statement without anything to back it up. Speaking to the point itself, however, I would argue, no, that’s not the case. Instead testers — particularly the specialists — simply know to avoid impossible goals with impractical methods.

“Owning quality” fits that description quite well.