Testing and the Nature of Time

To quote Doctor Tolian Soran, the villain in Star Trek: Generations, “time is the fire in which we burn.” In a little less fictional of a context, the historian Robert Bloch has said of time that “it is the very plasma in which events are immersed, and the field within which they become intelligible.” Beyond those sentiments, time is also what provides us with a keen notion of polarity, by providing two aspects around the project singularity I talked about. So let’s talk about polarities and time and see if we can’t make this relevant to testing as a discipline.

In my post about the human element of history, I mentioned polarities for the first time:

“I do believe that testing is about moving between polarities of thought.”

In the character of projects I said:

“I believe is a very important quality of our projects and, thus, a quality of people who work on them: the ability to move between polarities.”

and

“I have come to the realization that what we need is an understanding of the polarities we need to move between and some of the viable techniques for doing so.”

But I didn’t really talk about the polarities all that much. I tried to bring this home a bit more in the shifting polarities with at least one example:

“One prime example of this is the movement between the expression of thought and the representation of it. A pure example here might be between exploratory testing and executable scripts.”

But let’s back up a bit here and consider some of the wider context.

Paradigms. (Is That Still a Thing?)

The idea of a “paradigm” is one that has become a bit maligned, as a term, due to over-eager process and methodology enthusiasts who kept telling us to “change our paradigm.” An important concept was appropriated and misapplied. It’s time we recover that word, at least a little bit.

Generally speaking, a scientific discipline is committed to a core theory — or set of theories — that is so deeply entrenched that it largely defines the nature of the particular discipline. Thomas Kuhn, in The Structure of Scientific Revolutions, called this core theory, or set thereof, the “paradigm.” By this he meant the overarching concept that rules, and defines, a scientific discipline at any given time.

Keith Parson’s It Started with Copernicus offers a nice way to frame this idea:

“The role of a paradigm is crucial for several reasons. First, it defines the sorts of problems that a scientific field should study and imposes limits on the kinds of experiments or observations that will be relevant to their solution. Second, the paradigm delimits the kinds of answers that scientists are allowed to give when they solve puzzles.”

There are obvious and, hopefully well-known, examples of these shifts in science. The rise of Darwinism in biology, the credence given to catastrophism in paleontology, the emergence of quantum mechanics in physics.

These all required the shifting of polarities. There was a world before, there was a world after. But in all cases, the world before didn’t just go away. Darwin knew nothing of genetics and therefore much of previous thought was still useful in comprehending life. Uniformitarianism still had its aspects in history even under the recognition that catastrophic events occurred. Quantum theory didn’t negate pre-quantum physics, just expanded it. So there was a need to shift between the polarities.

Notice how I’m shifting from the use of “paradigm” here to “polarity.”

From Paradigm to Polarity

So what shifts in paradigm / polarity have we seen in the context in which testing is operative?

Well, if you go back far enough, you see a manufacturing concept applied to software. We’ve seen ideas of sampling and most certainly of automation applied. Just in automation alone, we’ve seen the shift to large cost-solutions to more diverse open source solutions. We’ve gone from mainframe style applications (“green screen”) to console-based applications (“CLI”) to desktop clients (thick and thin) and then onto web clients (thick and thin). Now we’re entering the realm of distributed and cloud computing. We’ve seen the break down of the monolithic applications into sets of coordinating microservices.

We’ve had to shift from testing just applications on desktops to web-based applications (not just sites) to mobile platforms. We’ve had to incorporate the concept of an API as well as numerous protocols to support them. We’ve added on layers of complexity, like machine learning and data science.

Development strategies have had to shift through all of this but so has testing.

Testing has to find its footing among developer-led approaches like TDD and BDD. These approaches were taking place as we moved out of so-called waterfall development approaches and into the complex morphology that is “agile.” That has led to continuous integration environments. This has coincided with the DevOps movement and continuous deployment environments. Through all of this we have seen the conflation of the tester and the business analyst and the tester and developer, with a role focus on the SDET or the SET and the rise of the technocrats.

Those are all the obvious things.

But as I’ve been talking about in recent posts, I believe time is one of the most important polarities that we deal with. As our release cycles have gotten shorter and shorter, and our applications more and more complex, there has been a need to focus not just on the “testing as activity right now” but the “testing as a means to reason about how we got to where we are right now.”

The Nature of (Project) Time

Time is one of the constants that we all deal with. And time has a topography all its own. Time and space provide the “field” in which history happens. Structure and process, however, provide the mechanism. It’s from structures that survive into the present that we reconstruct processes inaccessible to us because they took place in the past.

I can’t know what someone was thinking about when they wrote certain tests for certain functionality. I can see what they left behind as the tests. That person may be around to tell me what they think they were thinking at that time, but for them it’s just as much a reconstruction of the past as it is for me. The difference is that for them it’s not as vicarious as it is for me.

So we deal with current facts and historical facts. A current fact is exactly that: what we can see, right in front of our face, that demonstrably does one thing rather than another. It’s not open for interpretation; it is what it is. A historical fact is an inference from the relics that the past has left behind. So we have the present, where structures exist, and the past, where processes produced them. This is a key polarity that we testers need the flexibility to move between.

And this is tricky because of that project singularity I wrote about. As the future becomes the past through the singularity of the present, the continuities (the structures) are there but the contingencies (all of the means by which that happened) become a little lost to us. Unless we provided markers for ourselves so that the past was more discoverable.

Okay, so what have we ended up with? We know we can derive structures from processes. After all, the past proves that to us. But what about the reverse? Can we derive processes from structures? Keeping in mind that some of those processes are the means by which people internally made micro-decisions, or the decisions that were made in one-off meetings but not recorded in bug tickets or requirements, and so on. So we do have what ended up becoming real. Can we recreate the processes by which it must have become real?

Well, don’t we do that all the time? Our projects are essentially regular cycles of testing and development, right?

Project Landscapes

Okay, let’s approach this from a different angle. If we consider the scope of our project activities as “landscapes”, then we can say that we have both cartographic landscapes and historical landscapes. That is a key polarity that we need the flexibility to move between.

Why is this a polarity and what does it matter? Well, consider that the first of those landscapes — the cartographic — is bound by time and space, the other — historical — is not. When something is not bound by time and space (and here I mean bound in the strict sense), it becomes possible to manipulate time and space. The “dimensions” can be compressed, expanded, compared, measured. Freedom from time and space removes you of the strict need for chronology. This freedom allows you to give greater attention to certain things and less to others. You can rearrange what passes for “geography” in your context.

Consider simultaneity, which is the ability to be in more than one place at one time. We can do that in thought — such as considering tests in two different areas that are related by a workflow) — but we can also consider each aspect alone. Moving between those is a polarity. Scale is another: shifting between the microscopic — say, unit tests or data integrity tests — and the macroscopic — say, full UI with ReactJS connecting to an API that pulls information from third-party servers and shoves it to an HBase which uses Hadoop to perform map reduce operations for storage in Cassandra. Moving between these is a polarity.

It’s a point I’ve been struggling to make in numerous posts but manipulations of time, space, and scale are what happens when you depart from the literal and deal with the representation. A “literal representation” would, of course, simply be the thing itself. This means we can, in the context of our projects, have literalists — dealing with reality as it is — and abstractionists — dealing with representations of that reality. Again, this is a polarity and it’s a critical one that we move between.

I say that because you have to consider that representations become a way to arrange, or rearrange, reality to suit our needs at the moment. This has an interesting aspect on projects where we have the reality — let’s say, the source code implementation — and the abstraction — the description given by the business. I say that’s interesting because some might consider it to be the reverse. Yet which of those is more bound by time and space? I think that’s an interesting thought albeit one I’m having a hard time articulating concisely.

Regarding these manipulations, consider also that we have different means of manipulation that we can perform. Those manipulations are meant to tell us something; to verify something for us. That means we have different means of verification. For example, we have exploration and we have scripts. That’s another polarity, right? We have the discovery of business value and the elaboration of it. Certainly a polarity. We have the so-called statement of requirement, or use cases, or whatever, but then we have the tests that expand upon or illustrate that requirement. Yet another polarity. We have testing (as design) and checks (as execution). You guessed it: a polarity. We have the expression of verifications in natural language and the expression of verifications in code. Say it with me: a polarity.

Verifications

That focus on verifications is interesting to me. I started off earlier saying we have cartographic and historical aspects to our project landscapes. Well, verification breaks down along those exact same lines.

Cartographic verification depends on how well the mapmaker achieves a fit between what is being mapped and the requirements of the person who needs the map. This is an act of fitting representations to reality but customizing the representation for a purpose. You might want to check out my thoughts on how testing is like cartography.

Historical verification is a more interesting thing. What does it mean in the context of testing? Well, let’s consider that verification tends to take place by repeating actual processes. It’s essentially the re-running of experiments. In order to provide verification of findings, we tend to rely upon reproducible experimentation. This is, in effect, the rerunning of time, and the manipulation of variables that this procedure allows. But that’s our checking aspects, right? I can apply different input conditions (manipulation of variables) with the latest build and see what happens. I can also change those input conditions with the same build.

Our checking is done in the laboratory of an environment with a series of checks that we can execute and gather observations from.

But now consider, however, that fields like astronomy, geology, paleontology, evolutionary biology, and others don’t easily fit within the confines of laboratories. They can’t just “rerun time” in the exact same way and apply different conditions. They necessarily concern themselves, as does the practice of history, with interdependent variables interacting in complicated ways over extended periods of time. But when you consider testing, and not just checking, it’s the same situation, isn’t it?

Our projects are made up of people acting with their own values, biases, interests, attention spans, dedication, level of detail, etc. We don’t just deal with the build or the input conditions. We deal with the means by which people reason about those things. We deal with the ways that people make mistakes about how they apply those things. And we do this not just for a current project but about all the projects that have come before. The weight of the past is there, showing us much about where we’ve been, what we did, and hopefully — if we can glean it — why we did it.

In both cases of the verification I’m talking about, we want to reduce the potentially-infinitely complex to a finite, manageable, frame of reference. We move between these verifications. There, again, is a polarity. And I would argue it’s the key polarity. It’s one of the key means by which testing — in the modern sense of the word — continues to have relevance beyond just its checking aspects and beyond being a skill set that can be employed by any developer or business analyst.

Experiments and History

Any discipline that has experimentation as its basis also has a history that it can rely upon. That history, however, can be more or less difficult to decipher. So you need historians. In our case, project historians. And I believe testing, as an activity and as a discipline, can — and must — become this to remain relevant the project landscapes I described above: the increase in fast-paced deliveries of ever-more complex solutions into ever more fragmented and diverse environments.

I’m fully aware that some of the most learned practitioners in our discipline would likely entirely disagree with me on this. These are still ideas I’m exploring but I’m coming to believe them more and more as I put them into practice in my own career.

I’ll end this post with what I think is an excellent, and relevant, quote from the excellent book The Art of Time Travel: Historians and Their Craft:

“So here is our double historical quest: to be astonished as well as to understand. This tension goes to the heart of the historical enterprise – a tension between the past as familiar (and continuous with our own experience) and the past as strange (and therefore able to widen our understanding of what it means to be human). The essence of good history is this balance between empathy and perspective, intimacy and distance.

Historians immerse themselves in context; they give themselves wholly and sensually to the mysterious, alchemical power of archives. As well as gathering and weighing evidence, piece by piece with forensic intensity, they sensitize themselves to nuance and meaning, to the whole tenor of an era, the full character of a person.

Historians move constantly between reading and thinking their way into the lives and minds of the people of the past – giving them back their present with all its future possibilities – and seeing them with perspective, from afar, with a bracing sense of their strangeness.”

Properly abstracted, this sentiment has been highly relevant to my testing career.

About Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.
This entry was posted in Indefinito. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *