A way back I talked about testing focusing on its roots as a technical discipline. I’ve also talked about how companies should interview testers as testers and, most recently, a bit about being generalists with some areas of specialization. I often hear, usually from hiring managers, that what I talk about makes it very hard to hire testers in light of modern day realities. So let’s talk about that.

Modern Day Realities — Or Are They?

First, what are the “modern day realities” that are being referred to? Usually it’s a heavy focus on automation (good!) but coupled with the belief that you must be a trained developer to have any chance of doing this effectively (bad and wrong!). Even the focus on automation becomes bad when it’s not realized that automation is just a technique and the emphasis needs to be on expressive but concise tests first, automation of those second. This “modern day reality” often focuses on having a tester that is in effect really just an adjunct to the development staff or, in some cases, just a slightly less trained developer.

What this leads to is often a focus on testers being interviewed as if they were developers. Even when that doesn’t happen, it leads to testers being interviewed only as if they will be “automaters,” without care or concern as to the wider context in which test automation development takes place. The emphasis on “test automation development” is then put more on the development part than on the test part.

A further problem comes in when the assumption is made that someone who has any programming experience is not as good as someone who has specific programming experience in solutions used in the company. The idea there being that the test solutions must be written in the same language as that used by the developers. I talked a little about this when I posed the question of whether you align the test programming with the development programming so I won’t rehash any of that here.

Most of what I had to say about crafting test solutions I already said when I talked about test solution architects. But here I’ll focus on how I believe you can still find an effective and efficient tester that can come in with a more than good enough technical background while still providing the critical skill of being a broad generalist.

Yes, being a generalist is a skill. And it’s one that becomes harder to maintain for an industry that doesn’t see the value in it.

A Tester Who Generalizes

So let me start with an admission that will likely surprise no one: I have focused my career on being a test specialist but a technology generalist.

So what does that mean for me?

I can install various programming languages on all the common operating systems. In the JVM ecosystem, I have dabbled with Java, Groovy, and Scala. I can run Maven to get my dependencies. I can execute tests using JUnit or JBehave or CucumberJVM. I’ve run test solutions against .NET, usually with C# as the language, using NUnit and SpecFlow. I’ve set up dual Python installations so that I could run test solutions like Robot Framework, Nose, and Lettuce. I’ve set up Ruby using chruby, rbenv, and RVM and have used RSpec, Cucumber, Capybara, Watir, and so forth. I have set up NVM for managing Node.js versions and I’m able to use Grunt (or Gulp) to build test solutions using Mocha, Jasmine, and Chai.

I’ve talked to some testers that seem very impressed by all this and assume I’m “really a developer.” But I’m quick to tell them that I’m an expert in none of these things. I am not the best — and probably not even a good — designer of code constructs in any of these languages. There is so much more that I don’t know than I do and, in each language, I’ve routinely forgotten more than I ever came to know. You put any specialist (and probably even generalist) developer against me and they will get tired from running all those circles around me. I am profoundly stupid when it comes to computer science.

And if I was a developer, I would be worried about that. But I’m not a developer — at least not primarily. My focus — and thus my speciality — is quality assurance and testing. And in that focus, I’m a technology generalist. I know enough about each of those languages, and the ecosystems in which they operate, such that I can tell someone why I would pick and choose one over the other. My test automation specialist tendencies have leaned toward Ruby. Why? Largely for anecdotal reasons specific to myself. I like the language, it has a wide test ecosystem, and I’ve simply been able to do more with it given the time I’ve devoted to it.

But … that’s all about me. So what, right? Well, it matters only insofar as that context is what I try to find in others who work in testing and quality assurance. And it’s what I try to get hiring managers to look at. When I’m seeking testers — even those I want to automate — I’m not looking for someone who can optimize code arrays. I’m not looking for someone who can write any bit of code without once consulting the API documentation for a given library. I’m not looking for a person that can quote every design pattern to me and write an example of it in usage. What I’m looking for is someone who has bothered to explore the wider ecosystem in which a large part of their career will probably deal with.

It’s easy to state a problem so let me spend the rest of this post stating what my approach has been to solving this problem.

A Tester Who Is Mainly “Non-Technical”

So let’s say I want a tester who can think around problems and has a good basis for test thinking. This tester will have to work with automation to some extent, either that which they write themselves or that provided by others. How do I approach finding this tester?

One of the first things I look for is a diversity of approach, stated in resume and backed up by discussion. This person has ideally seen and worked with “TDD”, “BDD”, “EDD”, “ATDD” and whatever other letters have been combined together. They know the problems, have seen the limitations, have seen what works, what doesn’t, and under what contexts. This person realizes all those things are just acronyms and tend to be descriptive rather than prescriptive, regardless of what the pundits and purists say.

For me, a tester, particularly with years of experience, must demonstrate well-thought out answers to these questions:

• “What makes testing an awesome job?”
• “What makes testing complicated?”
• “What, in your view, is the appropriate relationship between tests and requirements?”
• “Given TDD, why don’t we train developers to be testers and have them test?”
• “Given BDD, why don’t we train business analysts to be testers and have them test?”
• “What’s the primary value of testing?”
• “What are the necessary skills for a really good tester?”
• “What do you do to stay up to date on your discipline?”
• “What trends have you noticed in testing and development in the last few years?”

You might (or might not!) be surprised how I often I run into testers who don’t have good answers to those questions. I’m not looking for the “right” answers; I’m just looking for good answers — i.e., those that come from having thought about aspects of their discipline based on observation, actual work, or a combination of the two.

All of this, at least to some extent, shows me if I have a thoughtful person who has introspected a bit about themselves and their career. That’s all experiential. Now to back it up, I would say that this person must be able to do a few things and I would ideally word it just this way on an opportunity description:

• Must demonstrate an ability to discuss domain concepts at a business level and a technical level.

• Must be able to demonstrate – not just talk about; demonstrate – the ability to separate test conditions from data conditions.

• Must be able to demonstrate – not just talk about; demonstrate – how they tell a “bad” test from a “good” test.

• Must be able to demonstrate – again, not just talk about – the ability to take a description of a system and, from that description, build up a model of how they would test that system.

On the first point, testers can demonstrate this by showing how they separate intent from implementation. Why? Because this will tie in with working equally with business folks and developers. On the second point, I like to see this because it ties in with writing tests — a critical activity, even when automating — and knowledge regarding different ways to structure tests. Also critical when automating. The third point is important to me because with this focus, testers should then be able to show me how to convert the bad into the good. The fourth point is important because it shows that testers can go from one abstraction to another and describe that process as they are doing it.

Notice how these elements have applicability whether you are focusing on a technical aspect to testing or not. These elements have just as much focus on manual test writing as they do automated test writing.

The questions and calls-for-demonstration, as worded, are meant to be a sort of “fly paper” approach. The goal of my “catch mechanism” in this context is not just to bring in a bundle of skills that happen to be encased in a person; but rather to bring in a specific type of engaged and engaging personality that happens to encompass enough of the relevant skills.

What does “engaged and engaging personality” mean here? It’s often what people loosely and vaguely describe as “passion”; as in “hire for passion, train for skill.” Yeah, I don’t believe in that. But I do believe that “passion” can be operationalized as someone who is articulate, curious, and investigative in nature. In that case, what I want to see is how they used those traits to gather skills. We can train on some; but you have to bring some to the table as well.

I should point out that for each of those points, you can ask your tester operational questions to ferret out more details during the interview process. Here are some examples:

• “How do you evangelize testing? What are some better and worse ways to do this? Have you seen this taken too far? And if so, what indicated that?”

• “Talk to me about your experiences as a test designer. How do you ensure you are doing effective design?”

• “What does it mean to develop test solutions? What have you done? What have you used that others have done?”

• “What do you think are the different skill sets required between an analyst, an engineer, and a developer, particularly as these relate to testing?”

• “If you had to train me as a tester, and based on your experience with test design, how would you teach me to recognize a bad test from a good test?”

If I do want some focus on automation skills which, by necessity, does require some technical programmatic skills, I might throw a few of these into the mix:

• “In terms of engineering automated solutions, what’s the difference you see between a framework, a library, and an engine?”

• “Talk about some of the challenges of open source testing solutions.”

• “Have you been exposed to BDD tools? Which ones? In what ways did you find them effective? How about ineffective?”

• “Since automated tests are just code, why don’t we just have developers write all the tests? (Assume hiring as many as we need is not a problem.)”

A Tester Who Has Some Technical Repertoire

There is reality to consider, and I want to make sure those hiring managers I talk with know I realize that. So now let’s say I want a tester who has some sort of technical edge to them, regardless of language, framework, etc. How do I look for that if I’m willing to accept a generalist stance? The key there is in the qualifier I used: regardless of language, framework, etc.

Here I ideally want some familiarity with open source. Not just tools, but the tester, in this context, needs to be able to download and install things themselves on their OS of choice. Working with and from the command line should not cause them issue, whether on Windows, OS X or any *nix system. They should be used to gathering information on usage patterns and good practices from various resources (StackOverflow, InfoQ, blogs, etc).

Here’s how I look for this — and this is pretty much how I would word it in an opportunity description:

You are right for this opportunity if you:

* Have experience with Ruby, Perl, Python, or Java. (Used some other language? That’s fine too.)

* Have experience testing complex, high risk projects.

* Are passionate about software quality, using open source technologies, and learning new things.

* Can break down problems into deliverable chunks, and are able to pivot when requirements change.

* Can communicate with technical and non-technical team members about progress, challenges, or new ideas.

Kudos to you if you:

* Have been exposed to numerous testing tools, like Cucumber, Selenium, Capybara, RSpec, JBehave, ScalaTest, EasyB. (Used others not mentioned? Cool. Tell me about them.)

* Employ a working knowledge of modern practices around test design and execution.

* Have very specific opinions about quality assurance and testing and how they integrate into business analysis and development.

* Have code on GitHub for personal or professional open-source projects (Send me links!)

I would ask the tester whether they considered themselves “familiar”, “proficient” or having “expertise” with certain things, particularly if it was a focus of theirs according to either their resume or something they said. That said, these terms shouldn’t just be some boxes that get ticked on a form. If you pose this to someone, it’s important to make it clear to them what you mean by those terms. For example, I have a focus on Ruby, as I mentioned. So if someone asked that of me, I would answer based on this understanding:

• To be familiar with Ruby would mean that the I should have at least seen the language, and maybe even have written a few small programs in it.

• To be proficient in Ruby would mean that I should be ready to code with little to no special training, and should be very comfortable programming in that language, based on past experience.

• To have have expertise in Ruby would imply that I should be adept even with many of the advanced features of the language, have a good knowledge of design patterns and code idioms in the language, and would be a good at mentoring others in use of the language and its ecosystem.

I make sure those distinctions — which are admittedly mine — are known to anyone who asks me what my skill level is. But I do make sure to understand other people’s gauge for skill level so that I don’t mistakenly apply mine to theirs, which is something I find happens way too often with hiring managers and what gets printed in opportunity descriptions.

A Tester Who Is Very Technical

Okay, so be all of that as it may: what if I want a tester who ideally has some sort of development background or is at least willing to balance both disciplines? I want to present this as a contrast between a tester who can utilize development skills in the context of, say, test automation or working with developers, and that of someone who I would want to be doing enterprise development work. The latter would not be a tester, in my view.

So first here’s how I would word things if I’m seeking a test candidate with a decent technical focus:

As a tester with some development background, you will ...

... focus on bug prevention and trouble forecasting by partnering with developers to create and code tests concurrently, rather than chasing or fixing bugs after the fact.

... help developers help themselves by locating weak spots in our code base and designing better, creative ways to break software and identify potential problems.

... develop automated tests, execute functional tests and create tools and infrastructure so developers can test their own code.

... leverage your thorough knowledge of quality to drive test strategy planning and analyze complicated software systems while delivering in-depth defect report analysis including coverage.

If there were specific technologies that we were using, clearly it would help if the candidate had used those tools. However, let’s say we’re using a Java solution to drive WebDriver along with, say, Senbot to tie some things together. I would be no means turn away a tester that had experience with a Ruby solution to drive Watir-WebDriver along with, say, Turnip. Nor would I turn away a tester that only used PHP with the PHP-SeleniumClient and, say, Codeception.

But do you see the point there? Being a generalist, I’m aware of those tools and the technologies they support. But I’m not necessarily good at all, or even any, of them. But I’m generally aware of them. In a general way. (See what I did there?)

Okay, I promised I would contrast this a bit. So let’s consider what I would ask a technical developer, as opposed to a technical tester.

• Write up a small bit of code that shows “tell, don’t ask” in action.
• Create a bit of code that shows a fluent interface in action.
• Describe an object mother and give an example of where you would use one.
• Describe (or code) a test data builder that uses chain-able public methods.
• What are self-describing values in code and how do you implement them?
• What’s a tracer object and why would you use one?
• How would you code logic that tests for information, not representation?
• What are promises and futures in code and how do they lead to effective design?
• What is domain driven design?
• What is the benefit of making transaction boundaries explicit in test code?
• What do loose coupling and high cohesion mean?

To be sure, I know testers who do think about exactly these things when they create or utilize test solutions. I’ve been known to myself a time or two. But my point is that this is a different level of technical skill that often is not needed in a technical testing context.

At this point, with this post and a few others I’ve referenced, I’ve probably said all I can reasonably say about my thoughts on this. What I now need to do is to distill all of this into something much shorter so that it make actually be more useful.