The idea of a test solution architect is a role that should officially exist but doesn’t. My thoughts are still percolating on this but let’s talk about architects in general and then talk about test solution architects.

The Architect

The very title of “Architect” can be an odd one and hard for some managers to categorize. The trick in defining a role is that architects are responsible for a lot of things. Let’s just consider this in the context of software engineering for a moment. Architects, in this context, need to ensure there is a set of principles that can guide development activities. Further, the architect has to make sure that these principles match the overall strategy of their wider group and organization. Architects need to make sure that these principles don’t require working practices that make the lives of developers, or others, miserable. Along with all this, architects need to keep up to date with new technology, and know when to make the right trade-offs for a given technology stack.

That sounds like a lot, right? And it is. Now keep in mind that architects also need to carry people along with them on this journey. Specifically, architects have to make sure that their colleagues (and their managers) understand the decisions being made and are motivated and engaged in terms of carrying out the activities those decisions require.

Just to add some final icing to this particular cake, architects can’t afford to sit in the Chair of Theory in an ivory tower somewhere. Instead, architects need to spend some of their time with various team members, carrying out various team functions, to understand the impact of their decisions.

What the better architects know is that being a person responsible for an architectural approach and ensuring that you execute on this approach isn’t just about making technology decisions. After all, machines and robots don’t rule the world. Not yet anyway. People will be the ones doing the work. Much of the role of the architect thus morphs into being a leader, which has a key focus on helping the team grow, not just in terms of carrying out the vision but also getting them to help refine and modify the vision itself.

The Test Solution Architect

What I said above would probably be agreed upon by most in the industry. What I say next will probably be agreed upon to differing extents, based on your experience. What I’ve found when I discuss this is that most people think something along the lines of, “Well, yeah, duh. Isn’t this what you testers sort of do?” The fact is that, no, what I describe here is not what testers “sort of do.” There are aspects that are done by various people at various times and under various conditions.

But what I’m arguing here is that a test solution architect does all of these things, pretty much all of the time.

The High Level

Let’s start with a high level generalization. A test solution architect fosters sustainable innovation by helping teams make the design choices that do two very specific things: (1) Make test solutions easy to create and understand from the start, but that (2) continue to be the right choices to make those solutions easy to test, extend, and maintain as they grow larger.

But what, exactly, are “test solutions?” To my way of thinking, test solutions are any solutions — people or technology focused — that help treat testing as a design activity and allow tests, whether executed or not, to be used primarily as a communication mechanism. What this ends up meaning in practical terms is that a test solution architect manages two very specific processes:

• The test design and implementation process.
• The test execution and deployment process.

The current focus of many SDETs or SETs is on the second of those, usually with a focus on technology solutions such as testing tools and automated test code writing. This is where many managers, at least in my experience, stop thinking about testing.

There is no doubt benefit to test technology stack and the automation of testing, but test solution architects needs to work on getting people to focus on what any sort of tool is doing rather than on the tool itself. For example, creating a continuous build is less about tools than it is about being able to continuously spot problems. Creating a test framework is less about tools and more about making visible how applications and services work under various conditions.

Related to Testing, as Practice

I’m not going to go too heavy into detail here but I’ll just cover a few points that I don’t often see articulated.

A test solution architect knows that there’s a spectrum between “completely untested” and “thoroughly tested.” Two areas between those extremes might be, for example, “respectably tested” and “comprehensively tested”. You would argue that testers should think this and it may even be true sometimes. But test solution architects must think in terms of distinctions like this all the time, and then work to make those distinctions specific.

A test solution architect is used to dealing with testing friction. Of course, dealing with it means you know what it means in the first place. Testing friction is the decrease in validity and ability of testing of complex systems of rules that change as problems permute in the system. The degree of friction determines the robustness of the testing implementation model you need and test solution architects must work to determine this while balancing competing needs.

Test solution architects have a specific focus on resisting the urge to restructure and reorganize tests so that they are easy to maintain but no longer easy to understand. This is a common trap that many testers fall into, particularly when they let tools (like test management tools) do their “thinking” for them.

Test solution architects realize that they must do two critical things:

1. Use the contents of tests to stay focused on making progress.
2. Use feedback from the tests to raise the quality of the system.

Again, you might think, “Well, yes, any tester should do that, right?” Probably yes, but it’s the role of a test solution architect that shows them how to do so in a way that is effective and efficient, while making sure that tests remain expressive, intent revealing, and implementation guiding.

Test solution architects should know a lot about testability. There is a whole lot to say on that topic but one thing I’ll say here that I don’t see articulated much: many, perhaps even most, bugs arise from entanglements with state and time. Test solution architects realize that, know what it means, and are able to think about what kinds of test solutions are best suited to handle those entanglements.

Finally, I’ll just add a few bullet points that I think test solution architects strive for:

• Use a minimum set of processes and tools.
• Minimize translation paths.
• Provide as unified a “source of truth” as possible.
• Minimum tests, maximum bug yield.

There’s a lot to unpack with each of those statements but this isn’t the post for that.

Related to Testing, as Discipline

Test solution architects should certainly realize that testing is not just about keeping bugs from users. In many ways, that’s epiphenomenal. Testing is about helping the team (collectively, not just the “testers”) to understand the features that the users need and to deliver those features reliably and predictably.

Test solution architects take a “synthesis view” of testing. This means they get teams to deal with the behavior of systems at their surfaces. This allows test solutions to be created that infer the machinery beneath from its reflections above, if you’ll forgive the literary flourish. Practically speaking, this means test solution architects work to understands applications and services by understanding the boundaries that contain them. When you consider that those boundaries are not just technical boundaries, the scope of this activity broadens considerably.

Finally, I already talked about testing that is effective, efficient, and elegant. Test solution architects make that distinction and keep that distinction in mind as they create mechanisms by which tests (as artifact) and testing (as activity) serve the needs of multiple audiences.

Related to Team Dynamics

Test solution architects need to build up test leaders. A test leader here is someone whose responsibility is to foster and maintain the project conditions that make good testing possible; and to train, support, and evaluate responsible testers.

A test solution architect on a team should be acting as a lead. This isn’t necessarily as a technical lead, but rather as a thought leader and practice evangelist. From my personal viewpoint, when I work with a team, I say that each and every tester on my team must have the following personal goals:

• Becoming a credible, high-integrity reporter of information that people value.
• Focus on trouble forecasting by partnering with business to spec features.
• Focus on bug prevention by partnering with developers to create code and tests concurrently.
• Focus on democratizing testing (anyone can view it, anyone can run it).

In order to do this, test solution architects need certain abilities. These abilities have been refined by experience and honed by practice.

• The ability to pull themselves out of a specific test task and see the big picture.
• The ability to manage many threads of work at once.
• The ability to coach and train testers, and people who can help testing.
• The ability to speak in front of people and produce useful and concise documentation as necessary.
• The ability to evaluate and explain testing in general, as well as with respect to particular forms of testing.
• The ability to recognize the difference between administrative testers, technical testers, analytical testers, and emphatic testers.

Learning From Other Disciplines

Test solution architects have to be cross-discipline associative. That’s another statement that requires a lot of unpacking. But let’s just consider a few points.

In computing science, abstraction techniques give us the leverage to amplify human reasoning. That same logic can, and should, be applied to testing. Tests are an abstraction and, if used as a communication mechanism (rather than solely as an execution mechanism), tests can amplify the ability of team members to reason about and understand their applications and services.

Software architects and development engineers know to defer decisions to the last responsible moment. Decisions early mean you have less information. You have to learn to defer as long as possible until you get to the point where not deciding has a bigger impact than deciding, even if you decide wrong.

And speaking of deciding wrong, we’re going to make mistakes. What project and product managers, as just two examples, do is work to reduce the impact of those mistakes. The goal is to make mistakes safer rather than eliminate them entirely. You do that by making decisions that are easy to reverse. You don’t lock into a bad decision made early on. Developers turn that into action by keeping the pieces small and self-contained (modular) with strong interfaces between them so that it’s possible to evolve each piece independently without rippling changes beyond the boundaries of those pieces. Architects find the things that are the immovable constraints and put things that can be changed around them.

You can get pretty far by acting as if every problem has very simple solutions. Then for the 98% of the time that you are right, you save an awful lot of energy to deal with the 2% of the time when you’re wrong and the solutions are not so simple. As just one example, test architects know that if you used focus object tests for the vast majority of the tests you write, while you will get some things wrong, you will have saved a lot of energy to deal with those things when they come up. This would be as opposed to putting most of your emphasis on high-level system tests.

Architecting as Communication

The solutions being discussed are often as much, if not more, about people as it is about technology. And people are wired to be social (for the most part) and get information from each other.

This means test solution architects have a heavy focus on sharing ideas more interactively and synthesizing knowledge more quickly. When dealing with teams and individuals who have not flexed these kinds of knowledge management muscles, test solution architects have to make sure that their team members aren’t just learning new skills, but also practicing the building of good habits and gaining a useful store of heuristics that they can apply conditionally.

Test solution architects are wary of heavy wiki usage, as just one example. We don’t always need more knowledge adding to some resource like a wiki; sometimes we just need better use of existing knowledge. Just having information that is passed around faster and in greater volumes gives people either an overwhelmed feeling or a feeling of false confidence. Both are dangerous. Architects in general have to be aware of this.

Architects also need to provide opportunities for learning and experience exchange. But for both of those to occur, you need people with the ability, the motivation and the opportunity to engage in the work. This goes back to the focus on your artifacts being a communication mechanism. Since tests should be seen primarily as such a mechanism, test solution architects should be well primed to foster ability, encourage motivation, and provide the opportunities when they can.

Hopefully this post provided at least a bit of my current thinking into test solution architects. I plan to expand on these ideas as I gain ability to articulate them more meaningfully.