What do maps do? Well, let’s think about how they great created. They get created by humans. So maps essentially distill the experience of others for the purpose of helping you understand a particular terrain. Maps help you get from where you are to where you want to go. Maps thus serve as a way to package up a vicarious experience by providing a representation.

Cartographers are people who create maps and it is very much a discipline. Further, it’s a discipline that shares a lot with testing. So let’s talk about that.

In my last post, I mentioned the coastline paradox, but I didn’t really drive home why it mattered. This paradox is often stated in the context of Lewis Richardson’s “how long is the coastline of Britain?” question.

The answer is that there is no answer or, at least, not just one answer. The answer is contextual.

The idea of the “paradox” being that the further down you go in your scale of measurement, the more irregularities of coastline you will encounter. This means that the length of coastline you measure can expand (or contract) in relation to the way that you’re measuring it. Yet, of course, the actual reality of the coastline is not changing, right? After all, the coastline is there.

So what does this tell us?

It means that we can choose to precisely delineate but we can also choose to generalize and abstract. This is how we create a representation of truth. But a key point that often gets left off is that our particular mode of representation determines whatever it is that we’re representing. How we choose to measure or deal with something is important and impacts what we are measuring. This is very much like the famous uncertainty principle in physics: the act of observation alters what’s being observed.

Grid Lines

Let’s consider what cartographers do. In a very real sense they reduce the (potentially) infinitely complex to a finite, manageable, frame of reference.

This involves the imposition of artificial grids — longitude and latitude — on spatial landscapes. This is similar to how historians place artificial grids — years, centuries — on temporal landscapes.

I would argue that testers do something very similar in their search for truth. Testers have their own form of project and product cartography that their discipline and artifacts help to make sense of.

But wait a minute! In my post on the character of projects, I talked about the “grid lines of a culture” and how laying down a grid “can stifle particularity and privilege legibility”. But isn’t that exactly what we want from the standpoint of making maps? We don’t want each person’s nuance to come out when they make a map, right? And we certainly want legibility, don’t we?

Yes. Consider that another way to look at map-making, and thus cartography, is an attempt to reclaim a sense of the entirety of something. Yet what we reclaim can never be that thing.

Put another way, to try to represent everything that’s in a particular landscape would be just about as ridiculous as attempting to recount everything that actually happened during a given period of time. Such a map, like such a recounting of events, would have to become what it represented.

So cartographers have to avoid a full, literal representation in making maps. Obviously if you had a fully literal representation you would have the thing itself. This is a similar challenge we face in testing particularly when we decide on different modeling techniques, whether programmatic or not. We have to be careful that the model we are creating is not as complex as the real thing or it may as well be the real thing.

But then how we do we capture truth about our terrain or our projects if we are saying that we can’t, and in fact should, attempt to represent the entirety? Well, looking at this from the cartographic viewpoint, we can see that having a model with literal fidelity to what it represents means that we lose the distillation; we lose the element that transmits a vicarious experience.

So let’s switch gears here and talk about verification.

Verification and Verisimilitude

Most people reading this blog will know that “verification” means something specific in testing. One of the challenges the industry has faced is testers who continue to harp on about “validation vs verification.” This is a discussion that needs to stop happening as it adds no value.

Testing, as a discipline, is ultimately about fitting representations to reality. Those representations usually come in the form of requirements of some sort. Tests, as artifacts, are a way to verify whether our representations fit reality in a way that is sensible.

Well, similarly the notion of “verification” in cartography takes place by fitting representations to reality. So you have the physical landscape, but, as we just talked about, you wouldn’t want to try to replicate that actual landscape. You have, in your mind, reasons for representing the landscape in a certain way. You want to give people the information they need for the purposes that the map is used for. And you want them to find their way through the map without having to rely on their own immediate senses. That’s why they are reading your map to begin with: to draw on the generalized experience of others. And keep in mind: they have the map itself, which results from fitting together what is actually there with what the user of the map needs to know about what is there.

The fit becomes more precise the more the actual landscape is investigated. Historically, the first maps of any newly discovered are are usually going to be fairly crude representations. But as exploration proceeds, the features of the map will become more specific. But people will have need of the map for different reasons. Some might want the quickest route. Others might want the scenic route. Others might want a route that showcases “interesting” things.

The point is that, as time goes on, there will be multiple maps of the same area. Those maps will be prepared for different purposes.

Cartographic verification is, therefore, entirely relative: it depends upon how well the cartographer achieves a fit between the landscape that’s being mapped and the requirements of those for whom the map is being made.

And that is exactly true of testing as well. The artifacts we leave behind on our projects serve as information that can be incorporated into “maps” of our project terrain. They provide a means for us to reason about where we’ve been, where we are, and where we would like to get to.

Representations and Choices

Maps represent realities they can’t entirely replicate. Maps vary the scale, the scope, and the content according to user need, which means maps are created based upon a set of choices. The same applies to tests. We choose to write particular requirements and tests — and choose to write them in particular ways — because we believe those options are better than other options. A key point that testers need to understand is that your tests live on: they serve as the representations of reality and are used as maps later on.

Maps are never free from preconceptions. There’s always some prior reason for what is and is not shown. The same applies to requirements and tests. We choose to limit what we represent. This very act of limiting, however, is predicated upon the preconceptions we bring as to what we think is important or what will be necessary to know in the future.

And, ultimately, we evaluate maps according to their usefulness. We ask things like: Is the layout of the map legible? Is the representation credible? Does the map extend our perceptions beyond what we ourselves can manage? If the answers to these questions are positive, that means the map performs the practical task of getting us from here to there. The same applies to our requirements and our tests.

There is a lot of material here I would like to unpack further, particularly my claim that discussions about “verification and validation” are particularly useless in testing. However, for right now, I’ll close with saying that a key lesson people who specialize in testing have to understand is that, on our projects, there is the reality to be represented, there is the representation itself, and then — most crucially — there is reception of that representation by those who need and/or want to use it.