The Joy of Testing

Awhile back I posted on the idea of wanting to be a tester and, even before that, I posted about why you might stay in testing. So here I’ll somewhat draw those two ideas together.

Obviously I’ll be largely speaking for myself in this post but I would like to think that what I say here resonates with those who have chosen to specialize in testing and quality.

Regular readers know that I tend to reinforce discussions of testing with science or the scientific method in particular. I do that since the basis of testing is, in fact, the scientific method and because testing requires many aspects of the scientific mind. Part of the scientific mind is having a very literal joy at exploring, investigating, discovering and, ultimately, understanding. But there’s another joy that is crucial, which somewhat underlies all of those things: and that’s the joy of spotting connections. Awhile back I talked about being cross-discipline associative and that idea is front-and-center here.

The Staircase of Testing

I’m going to double-down on my science focus here by referencing the book How to Find a Higgs Boson, which likens the search for nature’s mysteries as a staircase:

The author, Ivo Van Vulpen, then says:

“For the past hundred years, we have slowly been making our way down that spiral staircase, like archaeologists unearthing ever deeper structures. And each new level we discover is like a new country, with a wealth of knowledge and insights that offer us revolutionary perspectives on nature and answers to age-old questions. But our new discoveries also raise new questions, exposing mysteries and dreams from still deeper levels of reality. It is clear that we have not yet reached the bottom.”

This is exactly what testing is like. And what it should feel like to you if you are truly experiencing the joy of testing. Ivo also makes the following point:

“But we will have to proceed with caution. It is a world invisible to human eyes, dominated by seemingly magical behaviors …”

Dominated by seemingly magical behaviors? Sure sounds like the software world to me! And while what we test is often not “invisible to human eyes” at one level, there is a great deal of opacity and uncertainty in terms of how what we do see is impacted by what we do not see. Much of what we might be testing is, in fact, “invisible,” at least so far as we are concerned. It really depends where on that staircase you happen to be.

As testers of applications running on a variety of platforms, we ultimately delve into a world that is made up of bits and bytes. But along that path we encounter things like libraries, architectures, infrastructure, and so on. And these things wrap and intertwine and work with each other in sometimes very complex ways.

That can either be a journey of frustration or a journey of joy. Which side of that fence you happen to fall on will likely depend on how much joy you find in testing and, thus, how much of a viable career option testing is for you. And if you are trying to judge how much testing is a viable career option for someone else, you could do worse than figuring out their aptitude for what I talk about in this post.

Discovery Awaits

So, yes, put quite simply, the joy of testing is that it’s a journey of discovery.

This means you are often the first to enter some new terrain, looking at something in some way that no one else has yet. This means that you’re almost certain to run into unexpected problems that don’t yet have a solution. So, as I have discussed before, testing is a bit like cartography (where you have to figure out the map) and testing is like studying the past (where you may have to interpret the maps of others).

If you don’t find the idea of that exciting, and if instead you find that frustrating or tiring, then testing is likely not for you.

Discovery Requires Wanting to Discover

Let’s approach this from a different angle. One thing that’s generally a truism is that it’s very easy for all of us to go through life without really understanding anything about science, per se, or even just about how the world works.

For example, you likely wouldn’t lose a bet if you guessed that people use electricity all the time and yet have no idea how electricity is actually produced or perhaps even what electricity actually is. You will find that many people likely have a very firm opinion on “climate change” and yet have no clue that “weather” and “climate” are very different things and thus, almost certainly, have no understanding of what “climate change” science actually says or doesn’t say.

I pick these examples because we are effectively surrounded by electricity and weather. They are part of our day to day that we accept and either utilize (in the case of electricity) or react to (in the case of weather). The same could be said for the applications that we use in our daily lives. They permeate our lives in various ways and yet many people can go through their day, quite happily, not knowing or caring how any of it actually works.

When someone actually studies a bit of science, they usually begin to notice how little they know about things that seemed so commonplace at first. And then they realize how little they effectively know about science in general.

The Case of Electricity

For example, we learn that electricity has something to do with the human body.

Wait, it does? Yep! Our bodies are made up of cells. Those cells are made up of types of proteins, called molecules. Cell membranes function as a barrier to molecules, acting as a way for the cell to generate electrical currents. But how does that work? Well, we have to discover things. Like the fact that those molecules are made of elements and those elements are made up of atoms. Atoms are made up of protons, neutrons, and electrons, all of which carry an electric charge.

Carry this far enough and you end up at a spot where you realize that biology is reduced to applied chemistry and chemistry is essentially applied quantum mechanics.

But it requires wanting to discover all of that in order to actually get to that point.

The Case of Applications

Now think of one our applications that we might test.

We have what appears to be a simple web interface, let’s say. But what sustains that web interface or that app are some files that are marked up with something called HTML. And perhaps that HTML refers to something called CSS to change the appearance of what we see. And that HTML may refer to some JavaScript which impacts how what we see behaves. And that JavaScript may be part of a wider context that is provided by a library, such as React, which provides a way to describe user interfaces in terms of interacting components that pass data to each other.

Not only that, but our component libraries may also work with a runtime for querying data, say GraphQL. And all of that may be talking to various services, each of which do their own thing and are probably written in some specific language.

Here my article on testers and the abstraction stack is perhaps somewhat relevant. The abstraction stack is very much like that staircase shown above. We understand that the interactions of a given interface may be be reduced to a series of API calls to various endpoints, for example. Those endpoints are designed to accept certain types of requests and then return certain types of responses, perhaps a form of atoms in our testing world.

Just as with those molecules and atoms, you have to want to discover that path from web interface to API endpoints provided by backing services. This is part of understanding those connections that I mentioned earlier, which is one of the true joys of testing.

The Process of Discovery

Over the past centuries, lots of smart human beings have worked to discover many of nature’s secrets. The joy in this is often hard-won because it’s not like nature came with some guide book as to how it all worked. And it’s not like you could ask falling things how gravity worked. Nor could you ask things with an electric charge how electricity was generated or how it propagated.

So, instead of that, people had to study the world, and nature, and the universe in a very systematic way. We had to figure out how nature behaves under various conditions, both ordinary or extreme. Well, hmmm. Doesn’t that sound like something we do in our testing careers? The hunt for how things do and don’t work is part and parcel of the joy of testing. But just like nature, our applications don’t necessarily give up their secrets easily.

And while you can argue that there are guide books of a sort to our applications, in terms of, say, documentation on how JavaScript or other languages work, there is still a lot that doesn’t make sense once we integrate a lot of our systems together.

For us to move forward in science, we have had to perform a great deal of study via observation — often a lot of repeated observation — and then provide descriptions of what was observed so that others could try out what we did and see if they observe what we observed. Anything there sound like testing to you?

Earlier I said that we’re entering new territory all the time but notice, by what I just said, that we might actually be re-entering territory that we have already visited; perhaps numerous times. We are often repeating our observations again and again. And again. There has to be just as much joy in that as there is in attempting to find new observations.

Incidentally, along the way, those scientists, as they observe and describe, are trying to identify patterns. They do that because they’ve learned that those patterns will usually lead to deeper levels of how things actually work.

This is very much what we do with our applications when we develop them and, ideally, when we test them. This is particularly the case when we learn about certain patterns that cause odd behaviors in how things work, which we usually end up calling bugs.

The show Mr. Robot had a particularly good episode where a quite relevant thought was thrown out:

“Debugging’s actually all about finding the bug, about understanding why the bug was there to begin with, about knowing that it’s existence was no accident. It came to you to deliver a message, like an unconscious bubble floating to the surface, popping with a revelation you’ve secretly known all along.”

Yet there’s something interesting that happens here and it, once again, deals with drawing connections between things that might seem entirely disparate.

The Joy of Finding Connections

So, going with Mr. Robot’s notion of a bug that was “there to begin with”, consider this: you can localize one of those bugs. It’s sensible to ask: where is that bug? It has a position, much like, say, a particle. But consider something like quality, which some might associate with the idea of the least amount of bugs possible. The trick is you can’t localize quality. Instead of being like a particle, quality is like a wave.

Think of a wave in a pond. Where, exactly, in the pond is the wave? It’s not really a good way to frame it. The wave simply isn’t localized at a single position; it’s basically spread out at various places in the pond all at once.

If waves are not characterized by position, then what characterizes them? In a nutshell, four things: wavelength, frequency, amplitude, and phase. There are other ways to characterize a given wave, such as the speed at which the wave travels through some medium, like water. But those other properties can always be expressed in terms of the four core properties I just listed.

So if quality is a bit like a wave, let’s ask what characterizes quality. Maybe we say: behavior, performance, security, etc. It can depend, right? And while quality can have many aspects, they are often derived from some of what I just mentioned. For example, how quickly a user can navigate through a form can be considered a form of quality. But it’s a function of behavior (the form elements have to work) and performance (how quickly the form elements respond).

See how connections are being made here? A little bit of wave-particle duality from physics helps me think about quality and bugs. Doing this is a large part of why I enjoy testing. (If you want to see how embarrassingly far I can take this, check out my discussion on how testing helps us understand physics.)

There is deep aspect to the joy of testing that I want to make sure I convey here, which is in part based on the idea of testers being generalists with specialist tendencies. Such people are more apt to spot connections and have a joy in finding them.

Going in Circles

So let’s say you have a dog that’s tied to a pole in a yard. The dog can move away from the pole, but only up to the limit of the leash. Once the leash is pulled taut (and the dog is as far as it can get from the pole), then going around in circles is the only option. Obviously if the dog follows that circular path, it will eventually be right back where it started.

Let’s enter a little fantasy here and say that the dog is able to wear a camera harness and operate it. (What? Don’t all dogs do this?)

Every time the dog crosses over a specific area (marked with the X bones), the dog clicks a picture. So as the dog goes around in the circle, it takes another picture every time the X spot is reached.

Let’s say the camera is a digital one that sends the photos to you. What would you see when you look at those photos? You would see a series of photos that show the exact same thing.

Just looking at the pictures, you have no idea whether the dog took all those pictures after one rotation around the circle or simply stood on the X and took pictures one after another while remaining stationary. The point being that when the dog returns to the same X-point on the circle, everything looks the same as it did before the dog ran around the circle.

What the dog could be considered to be doing is a form of regression testing. It keeps doing the same thing over and over and observes what happens. Right now you’re probably thinking “Well, sheesh, that was a long way to go for such a simple observation.” Agreed! But — bear with me!

Riding on Waves

Now imagine someone in a sailboat on the ocean. The person has dropped anchor so that the boat is at rest and then they climb the mast. Looking around, and particularly ahead, they can see a whole lot of very similar looking waves stretching out for quite a distance. Each wave crest, of course, moves towards the boat, passes along it, and then continues motion behind the boat.

Now, if you’ve ever been in a boat, you know it can be risky to be out on a boat in very choppy (heavy wave) water. So our sailor in the boat, who happens to be a tester, is going to radio certain findings back to people who are waiting on shore. Those people want to ascertain risk. However, those people aren’t just going to accept something like “Barely any waves” or “Lots of waves.” They want something more quantitative. So the tester decides to look at the properties of the waves. And they get systematic about this.

  • The tester decides to measure the number of times that the boat bobs up and down as the waves pass. The tester calls this the frequency.
  • The tester also notices that the there seems to be a consistent difference in height between the peaks and dips of the waves. This seems to have some impact on how much the boat bobs up and down between each wave. The tester calls this the amplitude.
  • The tester also notices that the peaks and dips of the waves are separated by what seems to be a well-defined distance. An interesting feature here is that the peak-to-distance is the same whether the waves are far from the boat or right next to it. The tester calls this the wavelength.

The tester just measured some qualities. Yet there’s something interesting to consider here. The tester would likely notice that when they float up to the crest of the next wave, everything looks the same as it did when they were on the crest of the previous wave.

Does that sound familiar?

After bobbing down one crest and up to the top of the next wave crest, even though the tester knows that they have bobbed up and down one wave, there is really nothing they can see that verifies this.

Again: familiar?

If the tester took a picture every time they were on top of a wave crest, someone looking at those photos would have no idea whether the tester snapped all the pictures quickly while on the crest of a single wave or, instead, took pictures at some distinct pace, waiting for one or two waves to pass by between pictures.

Now surely you see what’s familiar, right?

Discovery: Circles and Waves!

Going around in circles is very similar to floating up and down on an endless series of waves. Riding waves is very similar, in a certain way, to going around and around in a circle.

After riding exactly one wave cycle, from the crest of one wave to the crest of the next, there is no change in the way the world appears. Nothing you can see allows you to determine which wave crest you are on, just as, in going around a circle, there is nothing you can see that tells you how many times you’ve gone around the circle.

The (Endless) Cycle of Testing?

There is a point here. Many people view testing as just this: returning time and time again to the same point, perhaps bobbing endlessly up and down on a series of waves, basically just seeing everything that’s been seen before, endlessly going around a circle in the hopes of maybe seeing something different.

If others see testing that way, you should recognize that and understand that what I described here is basically exactly what they see testing as. If you see testing that way, you won’t find joy in testing. But if you see testing that way, you also probably don’t understand testing. (And I asked before, do testers understand testing?)

Testing is, to be sure, endless. You never stop doing it. But that doesn’t mean it’s solely made up of doing the same thing over and over. Being able to hold those two views in mind is crucial to being a test specialist. But it’s the understanding of how that view of endless testing comes about that really matters.

Discovery Leads to Intuitions

Incidentally, this idea of wave motion and circular motion is a pretty big one in physics. In fact, it leads into something called gauge theory and I’ll spare you all the details of that. The main point is that when people learn this as they study physics, there’s usually a joy in coming to this realization and understanding that what appeared entirely disparate is, in fact, subtly connected.

Likewise, what I described here does point to something — or a few somethings — that are interesting about testing as well. Ways of testing (“wave” or “circular”) can lead us to the same kind of outcomes. But it also shows us that testing, and quality in general, do not have the equivalent of a gauge theory.

I hope the fact that I’ve mentioned gauge theory twice now and tried to somehow relate it to testing moved the needle on your curiosity scale. I’ll leave the idea of a gauge theory and why this relates to testing as something you can perhaps investigate and discover on your own. It’s actually pretty fascinating and I found myself experiencing a great deal of joy when I made the connection.

And that discovery of the connection did lead to a lot of intuitions about how I practice testing, specifically around understanding under what conditions things can go wrong, even when they mostly go perfectly fine. Sort of like the dog routinely clicking those pictures and seeing the same thing — except for that one time when it doesn’t!

Discovery Requires Seeing and Thinking Differently

So, yes, sometimes testing is like where you go around and around with your application and you need to see that it’s looking the same as it did before. That’s important. And, yes, we try to use automation to help us with repetitive tasks but, still, your role can sometimes seem like it’s all about doing the same thing, over and over.

But there’s also something a bit more subtle there. You have to be able to spot patterns, particularly in things that can seem very different. And there are reasons why, in our software world, those differences will crop up in ways that they won’t in physics. And those reasons very much help us understand how software works and, more importantly, how it’s likely to stop working in certain situations or under certain conditions.

And that understanding is crucial to understanding how test specialists think and see differently.

And being able to do that is, at least for me, a core part of the joy of testing.

Share

This article was written by Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.

One thought on “The Joy of Testing”

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.