I see so many people lately talking about the “death of manual testing.” Opinions obviously polarize on this but what I don’t see is testers engaging at all with why this perception is there. There is a form of indoctrination that happens across the industry. And testers, by and large, do nothing to combat it. Largely because they ignore where it’s coming from. Let’s talk about this a bit.
This indoctrination is by no means some insidious, planned out mechanism for the destruction of all manual testing or those who specialize in testing. Rather, it’s a natural outgrowth of the way information is presented in a rapidly changing technical and social context. It manifests most in a rising technocracy within the testing field, wherein to survive, you must learn automation and you must basically become as much like a developer as you can.
But then many other testers often wonder why their discipline is talked about in terms of automation only or why some may job opportunities are seemingly focused only on automation.
Let’s Look at Some Evidence
Well, consider this line from the book Continuous Delivery with Docker and Jenkins:
“Automated Acceptance Testing: This replaces the manual QA phase and checks if the features implemented by developers meet the client’s requirements.”
Many such books say statements very much like that. So a lot of people reading these books — and these are going to be developers and development managers — are seeing that “QA” (conflated with “testing” usually) is something that is on the decline.
Consider the book Infrastructure as Code, which says:
“Every problem discovered in manual testing — and in production — may be an opportunity to ask whether an automated test would catch a similar error earlier.”
Somewhat benign, you might think. But if you read the book it’s very clear that building the automation is really what you want to do. And if manual testing does have to happen and if it does happen to find something, consider how automation could have caught it.
In the book Production-Ready Microservices, you’ll read this:
“One of the key principles of software engineering in practice is this: if you have to do something manually more than once, automate it so that you never have to do it again.”
That doesn’t sound terribly bad necessarily. Until you realize some could say that this is why you “automate away” any manual testing, treating one round of testing the same as any other. After all, what’s the difference, right?
And if you combine that with the thought from Infrastructure as Code, you might think that this is the way to gradually reduce “manual testing”. And if you read the Continuous Delivery book, you may feel that you should replace that “QA Phase” entirely anyway.
There is a progression of thought there.
Consequences of the Evidence
Imagine you are a development manager or a developer or a new person coming into the field and these are the things you are being presented with because these are the books you are likely to read to stay relevant in your career. So you are constantly being reinforced in your view that anything “not automated” is probably a waste of time. And thus anyone not working towards automation is likely a waste of space.
And developers — who are building the things that add revenue and deliver value to customers — tend to have the ear of management and the business more. After all, it is true: if the business didn’t have the development (and thus ultimately production code), there would be no business.
So you have business who listens to a particular group (development) who are telling them a message they want to hear (automation can get things out continuously and we can fix and deploy immediately). And that group who is telling them this is constantly reinforced in their view by numerous authors, blogs, podcasts, conferences, and so forth. And the materials provided by those authors, bloggers, and casters are entirely reinforced by an industry that does want to move faster and sustain a much more rapid pace to be competitive.
The Tester Complicity In This
Testers often aren’t aware of these sentiments if they don’t read these books and don’t participate in the places where these books are discussed. And I find many testers who never read these books; who never attempt to learn how (or even if) their discipline is being well-represented.
Instead what testers are doing is regurgitating ideas like “testing and checking” (usually to each other, patting themselves on the back for a distinction well made) or decrying the prevalence of automation (to a wider audience that isn’t listening), or quoting folks like Boris Beizer, Cem Kaner, W. Edwards Deming (to an audience that struggles to see the relevance). Testers are often fighting battles on the wrong battlefield, using the wrong weapons, and part of that is because they’re not engaging in the wider discipline around them or understanding how their fellow peers, particularly developers, understand, consume, and articulate that wider discipline.
I’m not saying these ideas, in the broad strokes, would come as a surprise to testers. But what I am saying is that there is a sustained progression of these ideas throughout the industry. And those ideas are, if anything, gaining more and more traction. Participating in the places and times where those ideas are being discussed is necessary. There needs to be more of a point-counterpoint approach to giving people different ways to look at things and put some nuance into what testing is.
External Forces
I talked about project forces before and the above is one that we are dealing with all the time. It’s an external force that is operative on our industry and I hope I’ve presented it in a way that is fair. Because it’s really easy to see. If you look, that is.
Testers: Broaden Your Horizons
I feel comfortable pontificating on this with my normal opinionated stance because I’ve been talking about the need for testers to be cross-discipline associative for quite some time and in that post I talk about development-level thinking as informing test-level thinking.
Taking another data point, how many testers do you know who have read the book Developer Testing: Building Quality into Software? It’s an excellent look at how some developers view testing and I would argue it’s a book that has a lot of value, particularly because it is not at all dismissive of human-focused testing. But has the phrase “Developer Testing” caused many testers who are not developers to simply not check out the book, and thus learning some valuable insights into testing as perceived by developers.
Testers need to broaden their horizons and understand that, in many cases, where they are choosing to wage their battles is not where the war is even being fought. Testers are often planting their victory flags on hills that no one was trying to take anyway.
I agree completely that everyone in a cross functional Agile team need to broaden their horizons and be more cross-disciplined.
But I don’t think that contradicts the belief that there is something at the core of the testing profession and our skillset that makes us valuable in such a team. And I don’t think we should diminish that core, even though it is important to be cross-disciplined.
I believe (at least for now) that exploring complex systems and uncovering information about the cause and effect in such systems can be at the core of what a tester does. Any tools we can use in this pursuit, such as test automation or acquiring data from the system’s users, is a great benefit of course. But I believe that using an exploratory, risk-based testing approach will still be valuable and critical when developing high quality complex products, no matter how many tools we have to aid us in that effort.
But I agree with you that many of the discussions in the testing community seem more aimed at ourselves (maybe to help create a personal professional identity in an ever-changing complex world), than actually influencing others.
Anyway, interesting article.
Best regards,
Johan
When you say “something at the core”, I agree entirely. I wrote about describing my role as a way to show what I think does make me at least somewhat unique. In Testing is Like… I specifically said:
“The very notion of testing undergirds disciplines like physics, geology, historiography, paleobiology, chemistry, social science, archaeology, linguistics, and so on and so forth. Our discipline has an incredibly rich pedigree that can inform us. Let’s start acting like it.”
We have to harness that aspect of the discipline that sits at the core of many practices that are based on investigation, exploration, and experimentation. And most definitely we have to consider the appropriate tools that support all of that.
The trick is that we also have to combat what is a steady stream of contrary information being provided to the vast array of product owners, development managers, and developers out there. Regardless of your “core” or your “unique” aspects, it’s hard to bring all your powers to bear if you are not hired in the first place. Or if people don’t value your skills enough to listen even if you produce something worth listening to.
This post was more meant just to provide a slightly different take than I’ve seen before for those testers who are questioning why “manual testing is dead” keeps coming up or why there is so much more focus on tools rather than humans when it comes to testing.
For sure we need to be cross-functional and broad enough to understand which forums we need to participate in to actually be able to influence where the industry is heading, and how topics important to us are discussed in those forums. And that is probably not at a test conference, and least not with the topics and the participants most test conferences have today.
I might look to be negative and anti automation but this was the worst case I know of. At my previous job I used to manage students who automated regression tests and it was only to run the testscripts, even the developers could do it after creating the code checking in and building a new version!I think automation works for regression tests where previous implementation exist and specifications are not changed. New functionality is better tested manually ime when a human can do exploratory testing out of spec to find unexpected behaviour or errors in the software
Or to give feedback eg,if a helptext is making sense?
A spell-checker can only point out spelling errors but not tell if the text makes any sense you need a human to do that
You’ve perfectly identified the risks inherent in any sort of work-related silo. And given that testing is (sadly) pretty low down in the corporate pecking order, professional testers – especially in companies that aren’t specifically software houses – are not going to get anywhere near Board level. In the last role I had, I and one other tester were their first ever dedicated testing resources. The first project I worked on was declared “best tested ever” by the CEO but then fell over when it was released into the real world because the specification – which had been done six months before I joined the company by people who were no longer with the company – was actually totally deficient. The second project I was engaged with aimed to tackle these issues by involving testers in requirements gathering, but became over-extended and Board-level changes exposed this to scrutiny. First the project was pulled; then the company’s owners decided to buy in proprietary packages instead of following in-house development; then they decided to go one better and buy the company that built the proprietary packages. Nearly all the in-house devs and all the testers were sacked.
It’s entirely that mind-set, where Board representation of any sort of IT talent is lacking, that will see ‘automation’ as the opportunity to reduce the impact on the bottom line. ‘Automate all the things’ then becomes highly attractive because (to a non-IT person) it implies that costs can be slashed by getting rid of human beings from the development cycle.
I recently had cause to write to the CEO of a major High Street supermarket chain because I’d attempted to use a customer terminal in-store to manage my loyalty card account. It became clear to me that the system had been subjected to automated testing but no human being had looked at it after the system specs had been drawn up. There were no on-screen instructions over where to start, there were counter-intuitive navigation instructions once you did get the app to open, and no-one had ever tested the address matching against the database to see how non-standard addresses were handled. All the code was working exactly to spec but as a user I could not get the desired outcome from using the system. The CEO’s response was that “we recognise that the system is obsolete and it will be replaced next year with a smartphone app”. Whether that will be better designed and tested remains to be seen. I’m not holding my breath.
A couple of months ago I spoke to a local testers’ meetup about the corporate aspect to testing; how business decisions affect our work. My background has a lot of small-‘p’ politics in it (I hesitate to say ‘workplace politics’ because a lot of people don’t think that’s a thing, and in any case in an earlier existence, workplace and national politics were actually one and the same). It’s all very well knowing all the technical stuff, and I defer to my colleagues who know far more than I ever will; but there are political and corporate things that influence testing and directly affect the way testing is done and the eventual impact on the released product.
Well said. I have been working with a software which allows womem to gove birth to babies the size of an elephant, within specification but not making sense when testing manually. And the same company had a developer trying to automate testing for 2 years without succes. In the end they abandoned the project and decided to change tool
I am sorry about any misspelling in my earlier post. Not so easy to type on the mobile after having a stroke. There is more to the story… After the failure with automation one of the key davelopers of the software spent 6-8 weeks to figure out why the automation failed and one guy who is an expert working with the tool (tfs/mtm) added another day, finally getting ONE! Teststep to execute. During this period two testers had to write manualtests during a period of 10weeks to feed the first developer with testdata, So in the end a total of 3 developers spent 2years (1dev), 6-8weeks(1dev), one day(1dev). Two testers added 10weeks to get one(1) functional automated teststep. And in the end they are doing the tests manually. Add additional time for meetings where the developers team had to make decision sbout abandoning the automation project and selecting another tool…And the entire scrumteam had to be at meetings every week/daily to hear that automation did not work. I think there is great advantage in automation but please use common sense. I know that the customers/users of this particular software, midwifes, would go bonkers if no human answered their qestions. An automation tool is not the kind of interaction they would accept. Sometimes we have to discuss the requirements and/or explain the development/testing process. Is there any automation tool up to this task? Or try to bugger developers to remember 15 years old decicions about implementatation of requirements
Jeff – very eloquently explained and you raise a valid point which is different than the usual defense of manual testing. I often read the inclusion of Cem Kaner in a defensive post as the evidence that manual testing isn’t dying and was intrigued by the notion that the testers themselves are complicit.
One area that I think isn’t mentioned is the role of the online publishers. Since the push is pay-to-play for most of these vendors, the running narrative ends up becoming based on he/she who can pay the most to get his or her voice out to the masses. I don’t blame the vendors (perhaps a little bias given my role) necessarily but there is certainly some complicity there too.
Thanks for giving me pause for thought.