Select Mode

Product Management and the Quality Focus

In a series of posts I talked about product development. But now let’s dig in a little to the idea of product management. Although not often framed as such, product management is very much an area of quality assurance. The intersection is crucial in the modern technology industry.

Many businesses need to move fast in order to sustain a style of development that allows for rapid delivery and thus rapid feedback. This allows the cost-of-mistake curve to stay tight.

Yet there’s a challenge here. Going fast in the long term is really not possible, at least in a sustainable way, unless the quality of the design and of the understanding of the business domain is under control at all times. This is part of what product management provides and is key to understanding how it intersects with quality assurance. For too long the technology industry has treated the two as if they are entirely distinct disciplines.

My own experiences in showing how the disciplines interrelate was coming to the realization that you sometimes have to remove the “product” from product management and replace it with “solution” and thus provide solution management. A semantic nit-pick? Well, let’s consider the distinction a bit.

Distinction: Solutions and Products

Here’s a way of framing things that I think is critical for product managers to promote: treat every release as an experiment and be mindful of what you want to learn.

I’ll be the first to admit that this comes about from my background in scientific experimentation but, as I’ve discussed, the scientific method is the heartbeat of testing and I believe that a version of it is very much operative in product management. It’s just more obvious, I think, when product management is framed around solutions rather than products.

Thus, taken to a logical conclusion, what I’ve said is a way of saying we should focus on minimum viable solutions, not minimum viable products. But what does that actually mean?

The minimum viable solution is the smallest solution that successfully achieves its desired outcomes. Those outcomes will be captured in acceptance criteria. The creation of that acceptance criteria is how we perform quality assurance, and thus a form of testing, at one of the earliest times we are likely to make mistakes. (And note that this is well before any coding even begins!)

I’ve found some of the above can cause cognitive friction, particularly around reframing “minimal viable product.” A minimal viable product is, or at least should be, the smallest thing you could create or do to prove or disprove an assumption. Sounds like an experiment to me. But doing this usually means testing assumptions about what users do or do not want. But how do you know that? You can only know that when you have provided the product to the users so they can give feedback on it!

In whatever form of Agile/Lean you practice, the idea of “done” should only mean that the work is in the hands of users and feedback is possible. When that’s not the case yet, the appropriate term is “Ready for Release” or “Ready for Production.” This gets a little bit into how tickets in our system of choice, such as Jira, are setup and the workflow they follow. This is something I’ll talk about in a follow-up to this post.

So, for now, rather than focusing on that workflow let’s instead focus on another distinction.

Distinction: Iterate and Increment

As product managers, the goal is generally to get people to start consciously using two modes of thinking:

  • Use iterative thinking to evaluate and make changes to what we’ve already made.
  • Use incremental thinking to make entirely new things or additions around existing things.

A good way to view this is by considering feature iteration. So let’s get a visualized context for some of the details around being iterative (usually good) and being incremental (usually bad) in the context of feature development.

Incrementing vs Iterating

In his article Don’t Know What I Want, But I Know How to Get It, Jeff Patton used a sketch of Mona Lisa to illustrate the difference between iterative and incremental development. Here I’ll build on that a bit.

This also ties in with quality assurance and leads to a lot of thoughts on testing, some of which you’ll find in my post on the shape of testing. Product management is one of the forces that act on testing in order to allow the shapes I describe there.

So let’s provisionally agree that the goal is to avoid a purely incremental approach. For example:

Here the problem is that the outcome is being envisioned as some fully complete thing. This tends to lead to an incremental strategy:

What this means is you never actually deliver a viable outcome until you deliver the full thing that you’ve thought up. You build it up in increments. And those increments tend to focus on outputs rather than outcomes. And those outputs tend to be focused on dates of delivery rather than value of what’s actually being delivered.

The goal is thus to adopt an iterative approach.

Here the outcome is envisioned as just that: an outcome. That outcome is not stated as some completed thing but rather as a benefit that is desired. This tends to lead to an iterative strategy:

Here you can potentially be delivering a viable outcome at all points, while also giving yourself the freedom to gather feedback and determine how the vision may have to change.

As the painting evolves notice how the hands change position, for example, and how the background changes a bit. But, overall, we end up with the outcome. And nothing says we had to end up at evolution 5 of the design. Perhaps ending up at 4 would have been good enough.

While having these outcomes in mind and as part of our discussions is good, we also have to make that we capture the salient details. That brings us to stories.

Iterating with Stories

Any story will, ideally, be focused on a persona that is operating whatever your proposed solution is, specifically in the context of a desired outcome.

For something to actually be a story, it must be possible to provide feature injection statements:

Note that a user story, ultimately, will be composed of tasks. While the user story talks about the “what” (and thus the intent), the tasks talk about the “how” (the implementation).

A user story will also have a feature specification attached to it, which means scenarios. These scenarios should illustrate successful interactions with the feature such that it can determined if the outcomes are met. If the scenarios are written as acceptance criteria then the outcomes are tied to the acceptance criteria. Which, not coincidentally, means the tests are tied to the requirements in that the tests really are the requirements. Or, if you prefer, the requirements are written in the form of tests.

Those scenarios can use a test injection format.

I should note that while I show the common “Given-When-Then” format there, I do that just for illustrative purposes. The actual structure of the scenarios is much less important than having scenarios in the first place.

It probably won’t hurt to also note that I previously talked about this idea of injections and I do think it’s a powerful way to consider what goes into your stories.

What you end up with is a series of stories, all with feature injection to communicate relative value:

And, of course, there may be stories that are thematically related, which is what an epic provides:

A key thing to keep in mind here is that each user story is effectively providing a minimum viable solution. It’s a solution to a need that a user has. This is as opposed to a product to provide a user need. That can seem like a very subtle distinction but it is an important one.

And that takes into other area of product management that closely aligns with quality assurance by making sure that we understand quality is a shifting perception of value over time. Thus any notion of quality has to be mapped to the experience of someone who can make a judgment about quality. This means considering perspectives and that takes us into user mapping.

User Mapping

User mapping is a defined technique within the context of product delivery. The idea is that by building the story from the user’s perspective, the delivery team identifies how users interact with the product and what requirements need to be met to facilitate those interactions. This is all about mapping the user to the features.

This is distinct from, but related to, the idea of a user experience map. An experience map is a method of visualizing the entire end-to-end user experience that an average user will go through in order to accomplish some goal.

The Story Map

A key part of having a story map is telling the high-level story in the first place. This is where narrative intuition comes in. I talked about narrative intuition a bit when discussing having an intuition for abstraction. And just like how testing can be like writing fiction so, in a way, is product management.

Let’s break this down into some particulars. User tasks are the basic building blocks of a story map. A common visualization of this is like the following, from the book User Story Mapping:

The visual is fine but, like most such visuals, it’s relying on a more descriptive mode of thought. So let’s break that down a bit and make it a bit more prescriptive.

  • Activities and high-level tasks form the backbone of a story map.

A common technique here is read the backbone of the map (above the blue line), with the conjunction phrase “… and then they …” between each activity. Organizing a narrative flow with conjunctions allows you to more easily see the parts of the story that are missing.

  • Activities aggregate (organize) tasks directed at a common goal.

This is crucial. Tasks are organized by activities across the top of the map. That’s your breadth view of things. But there’s also the depth view. The depth of such a map contains variations, alternative tasks that a user might embark on, and exception situations.

  • The map tells the story of a feature, step-by-step from a user’s perspective.

Now here’s what I think are two really important points that all of this leads to and this will please those who like to have some metrics.

Metrics From the Map

The first point is that the width and depth of a typical story map gave a sense of the project size. Yeah, it’s not a specific quantity but you can often compare such maps for different features and, by using that breadth and depth view, get a pretty accurate feel for what the size of the work is like.

The second point is that the number of paths through the map are a good (not perfect, but good) indicator of complexity.

What often isn’t reflected in all of our work is uncertainty and risk. To handle that in the context I’m describing, you can increase the frequency and fidelity of your story mapping activities.

  1. By increasing the frequency of when you map, you tend to identify more risks.
  2. By increasing the fidelity of the map, you tend to better manage the risks.

I personally think this is really nice because it aligns the process (user mapping) with the artifact produced by the process (user map).

Changing the fidelity of map is a way to modulate your iteration pace which is another way of saying you can figure out what a sustainable development pace means. Sustainable, in this case, meaning not just that the feature gets developed but that the feature gets developed in a way that can be tested (thus testability is a primary quality attribute) and can be developed in a way that allows for determining the value delivered given that quality is situational and subjective.

It’s a product truism that the biggest driver for missing dates — or having work carry over even if the date was reached — was unplanned work. And that unplanned work is mostly due to uncertainties and realized risks.

Story Slices

Product management will usually promote the idea using story slices to identify all the tasks and details relevant to a specific outcome.

This is the same idea as was reflected above in the previous image but here you form slices that are minimum viable solutions; minimum tasks that will allow the immediate outcome to be possible. A slightly more conceptual view might help:

Many product teams advocate for three slices along with a “walking skeleton” (also called a “functional skeleton”).

Notice how the “Mona Lisa” concept ties in nicely with feature iteration. In essence the three slice model is some variant of the mantra: “First make it work. Then make it work better.” Product management, however, will expand on that idea.

  • [SEE IT WORK] First slice (functional walking skeleton) cuts all the way through the functionality.
    • Once you build all those pieces, you can see the functionality working from end to end.
    • It wouldn’t work in all the situations it needs to, and if you shipped to users this way, they would likely be less than thrilled.
    • But the end-to-end part lets you put real data in it to see how well it performs and perhaps how well it scales.
    • The goal here is: essential features or user steps that cross through the entire product with the use of data.
  • [MAKE IT BETTER] Second slice is to build up the functionality — to get it closer to releasable.
    • This will reveal characteristics that the feature should have. Here is where you start to layer on various types of qualities.
    • The goal here is: fill in and round out features by focusing on optional things users might do and this helps provide scale.
  • [MAKE IT RELEASABLE] Third slice to refine the feature, to make it as polished as it can be.
    • The goal here is: refine and make it more efficient to use and to use data at scale.

Here’s a key thing to understand about the above list. It may look like it can be broken down into something like “Version 1”, “Version 2”, “Version 3” and so on. But that’s not the case. That would be like the incremental Mona Lisa approach shown earlier rather than the iterative.

Notice the walking skeleton image above talks only about “releases”, not “versions.”

And I call that out because product management wants to get the team to a point where they are working through the above three elements vary rapidly for any given set of feature work. It’s not so much about “Release version 2.1.13 out to our users” as it is about “Release these features that provide these outcomes to our users.”

Story Framing

All of this lets you frame the work being done by injecting user types (personae), the frequency of use of some aspect of the functionality, and opportunities in the form of rewards and pain points that the user may experience on the journey.

This is a crucial part of making sure the story and the users are mapped around the central value being delivered by features. This also help us make sure we are considering different types of users since what one user considers a rewarding experience may be painful to another.

Thinking Around Solutions

Even with my emphasis on “solution” above rather than “product” there’s a danger here too that product management must be aware of and deal with.

It’s easy to talk about a solution; it’s less easy to do the critical thinking required to uncover the issues the solution is supposed to resolve and the multiple types of solutions that might resolve those issues. It’s less easy to find evidence that proves what specific problem is the one that should be solved and then exploring systemic implications of the solution. It’s less easy to identify and overcome constraints around solutions.

My point here is that you have to have the ability to “move off the solution.” Thus product managers, much like testers, have to shift between polarities!

This means you start freeing yourself from promoting something specific, at least initially, and instead focus on understanding what problems you, collectively as a delivery team, want to address and the results that will be valued. The delivery team thus explores whether what you all work on is a good fit for what is most being sought, which is another way of saying you focus on outcomes rather than just outputs.

The challenge here for product managers is that, ultimately, we’re dealing with perceptions of quality. And those perceptions come about in a context of multiple characteristic usage patterns, differing levels of relevant abstractions, varying levels of user expertise, and impactful technological changes over time.

Core Values

This allows a product management team to promote some core values.

One core value is a focus on collaboration by crafting solutions that allow the business and developers to jointly explore business needs and the possible ways of answering those needs in a technical context.

Another core value is a focus on transparency by crafting solutions so that what developers do is reflected immediately in what the business sees. Likewise, what the business needs is reflected immediately in what the developers build.

Yet another core value is a focus on allowing solutions to evolve as both developers and the business become more experienced with expressing the problems they are trying to solve and building focused solutions for those problems.

Since this work is taking place in the context of a delivery team, this means internal and external qualities can be accommodated in terms of the decision making around what is desired to be released and the feasibility of actually releasing it, with agreed upon levels of quality and within acceptable timeframes.

Thus the Quality Focus…

And that last bit right there is the key take away, I think: product management is focused on the idea of getting feedback on solutions by delivering outcomes in acceptable timeframes and with levels of quality that stand the best chance of engaging users.

Yet there’s one more bit to this quality focus, which is how we take all those stories we create, and move them through our delivery team workflow. That’s what I’ll talk about in the next post.


This article was written by Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.