I was going to title this post “Testing is Like Archaeology.” Then I was thinking of “Testing is Like Geology.” But then I realized, as my argument took shape, that I could have said testing was like paleontology, or geomorphology, or even biography. I realized then that what I really wanted to focus on was how testing was, oftentimes, about studying the past. But to drive that argument home, let’s consider why we study the past. And let’s also consider why such study informs the future.

This is my continuing effort at pursuing the testing is like… idea, following on from testing being like cartography.

Look Forward, Not Backward!

Before we get into the presumed meat of my argument, let’s even consider if my title makes sense. Why is testing like studying the past at all? Wouldn’t it make more sense to say it’s about studying the future or at least what the future might bring? I would argue no. We can’t really “study the future.” We can only reason about what is taking place now — although keep in mind the nature of the project singularity! — and what has taken place. With any luck, we can use those to inform the future. But we can only “study the future” when it has passed through the singularity of “now” and become the past.

Okay, Fine. The Past It is, Then

Glad we got that settled. Alright, so, we study the past because it is, in essence, an abstraction. And that is a really important point that is very easy to forget. The past is ultimately an interpretation, providing us with a vicarious enlargement of experience from which we hope we can draw consensus upon how to act in the future.

As Geoffrey Elton said in The Practice of History:

“What we call history is the mess we call life reduced to some order, pattern, and possibly purpose.”

I like that because it implies the order and pattern comes later, not at the time those elements we are studying were created. Consider what that means for test repositories or the storage of information for how an application works. This focus on the past means that we simply have the information available. But the order and pattern can come later when we figure out the purpose we want the information for.

This means, however, that testing, as a discipline, must consider the kind of markers that the past can leave behind to help those who have to reason about things later on.

While the past is a study of time and space, this study offers us a liberation from the limitations of time and space. For example, an archaeologist, geologist, or biographer all have the freedom to give greater attention to some things than to others and thus to depart from strict chronology; they all have the ability to connect thoughts, concepts, or physical structures that are disconnected in space. They can thus, to a very real extent, rearrange geography as well as causality, thus having the “future” inform what was the past.

Think of photographers who capture moments of history for us. Our sources — the structures — are those snapshots that they take. But later we can arrange those snapshots in various sequences. Those arrangements can provide a certain significance depending on how that particular arrangement works given what we are trying to understand. Beyond even that, there is significance in the gaps between those snapshots. What we don’t see recorded can often be as important as what was.

As Edward Carr pointed out in What Is History?

“Millions of people over thousands of years have crossed the Rubicon. We decide which ones to write about.”

Selective, Simultaneous, Scale

What people who study the past have the capacity for are varying levels of selectivity and of simultaneity. Such people also have the ability to shift the scale of consideration. This means they can select from the cacophony of events to provide a focus on what they think is really important given the current reality. This mean that they can be in several times and places at once. This also means that they can zoom in and out between macroscopic and microscopic levels of detail based on need.

What I’m talking about here is the flexible movement between polarities. When you apply varying selection criteria, when you allow simultaneity to be fluid, when you scale between macro and micro, what you are doing is shifting between what are polarities. You are also, however, harnessing an extremely powerful tool: narrative. You are constructing a narrative that may be bound by time and space, but may not be. Or, at least, need not be.

Consider Geomorphology

Geomorphology is basically the study of processes that create and shape topography. These are essentially explorations — more fascinating than you might think — of how literal landscapes evolve. You can see where steam channels started and acted over time; how landslides, small and large, have sculpted small hills and giant mountains. You can learn how and and why rivers carve gorges of surprising complexity through mountain ranges.

Reading The Rocks Don’t Lie by David Montgomery, I read the following experience report:

“I’ve learned to see what the land used to look like, and what might look like in the future. Reading a landscape is an ongoing process of combining curiosity and inquiry. … Deciphering topography makes geologists natural storytellers. We piece together fragmentary clues in rocks and landforms to connect dots across landscapes, mountain ranges, and continents and tell stories with whole chapters lost to erosion and time.”

David also says this:

Deciphering earth history involves establishing the basic relationships between different rock formations and the nature of boundaries, or contacts, between them. Two layers of sedimentary rock deposited one atop the other without any discontinuity are considered conformable — they accumulated with minimal interruption. An eroded surface leaves a discontinuity between two rock units, a gap representing missing time that geologists call an unconformity. An unconformity represents how far down erosion wore into an ancient landscape before additional sediment was deposited on top.”

I think that’s very much like our projects and the role of testing on them. We have these different “formations” — groupings of tests, smatterings of requirements, bug tickets, prototypes, wire diagrams, past Kanban boards, etc. We can often determine the “contacts” between these formations. We can see where our structures in the past seemed to accumulate with little apparent interruption. But we can also see gaps — those “discontinuities” — where lots of stuff kept piling on, but what was there originally (if anything) was either lost or so mangled out of shape that it essentially ceased to exist.

Testing, as a discipline, should help spot these discontinuities, call attention to them, and then see what it can help reconstruct based on those historical markers it left behind.

Leaving Markers and Indicators

I keep throwing in “structure” and “process” above so I want to get to that distinction but, before that, let me quote John Gaddis from his book The Landscape of History:

“Historians employ abstraction to overcome a different constraint, which is their separation in time from their subjects. Artists coexist with the objects they’re representing, which means that it’s always possible for them to shift the view, adjust the light, or move the model. Historians can’t do this: because what they represent is in the past, they can never alter it. But they can, by that means of a particular form of abstraction we know as narrative, portray movement through time, something an artist can only hint at.”

But testers are not operating in an either-or in that context. We can be artists in the moment but what we do gets recorded as a mechanism that, for all anyone knew, was always like what we just made it.

So we can “change the past” in a certain sense. We can smooth over those discontinuities and gaps and provide a cohesive picture. A picture provided by a narrative that we construct as and when it is needed. That latter part is critical: as and when it is needed. That means we don’t have to worry overmuch about the narrative when the structures are created, but we have to have enough insight to know that eventually various narratives will be needed. So we need techniques to provide markers and indicators.

As Stephen Jay Gould said in “Wonderful Life” said: “The appropriate methods focus on narrative, not experiment as usually conceived.” There’s that idea of “narrative” again. The ability to construct and deconstruct narrative is one of our most potent research tools. It’s also one of our most sophisticated of all methods of inquiry.

There’s a balance here. The more time the narrative covers, the less detail it can provide. It’s perhaps a little like the Heisenberg uncertainty principle, where the precise measurement of one variable renders another imprecise. So there is care that must be taken here.

We are, as I said above, engaged in an act of representation and it’s a truism that representation is simply the rearrangement of reality to suit whatever our purposes for the moment are.

Structures and Processes

Let’s consider that time and space will always the field in which history happens. This means structure and process provide the mechanism by which that history happened. Structures that survive into the present allow us to reconstruct the processes that led to those structures.

Think about this in the context of testing. We often end up with a repository of tests, organized in certain ways. A process led to their creation. That process is, to some extent, inaccessible to us — even if we follow a general pattern. There are always micro-decisions that people make about what to include, what not to include, and so forth. So ultimately what we have is the remaining structure. The same applies to requirements and, in fact, the source code that most of us are ultimately dealing with.

This is very much what archaeologists face: they have the surviving structures of humans but they often have to intuit the process by which those structures came down to them. Likewise, when we think of the work of geologists, the distinction between structure and process corresponds to the one between the present, where structures exist, and the past, where processes produced them.

There was a time when people, including the most learned, believed that just about everything was static. That the world as it is now is, for the most part, the world as it was back then. Eventually, however, science started to take hold in various forms. The work of Charles Lyell in geology, Charles Darwin in biology, George Cuvier in paleontology, Alfred Wegener in meteorology, Robert Boyle in chemistry, Albert Einstein in physics, and so on.

All of these sciences were essentially learning to derive structure from process; specifically to derive past processes from present structures. They were applying the notion of history to their work. As such scientists became, by degrees, historians. Keep in mind, too, that the surviving structures can be artifacts and archives — but they can also be memories of participants, including the oral tradition that has been handed down.

One of the challenges that geologists and paleontologists deal with is that most sources from the past don’t survive. Most daily events, the minutiae that make up history, simply don’t generate a survivable record. And, speaking to our projects, the same applies for us in testing and in business. We have artifacts, but we don’t always have the false starts, the bad paths, the heated discussions, and the group think that led to poor decisions.

Taking some other domains of study, one of the challenges of biologists and astrophysicists is that they must deal with ambiguous evidence or even outright contradictory evidence. What they study is often the result of an underlying complexity, the rules of which we don’t always quite understand or can’t model in their entirety. Certainly we can see the correlation in that from a testing career standpoint, right? This is a bit of how we use tests as representation of the past. We sort of see the realm of the business domain. We can speculate as to why certain tests were written a certain way. But sometimes we are just left with a mass of material.

Those Shifting Polarities

In my post on the shifting polarities of testing I did specifically mention the “derivation of past processes from surviving structures” as one of the areas that I believe the testing discipline sets itself apart on, in terms of being able to flexibly shift between these modes. I hope this post provided a bit of context for that idea.