Awhile back I posted my call to action posts: Testers: Act Like a Developer. Since that time, I’ve done a few posts on what that looks like specifically and why it matters. This post is another one of those, showing the very early stages of a project.

Previously I talked about the integration pact, which was essentially the idea that problems live at the seams. Something pretty much any tester becomes aware of very quickly. This post is another way of describing the seam.

I spent part of my morning doing something that should have been boring: setting up the scaffolding for a new project called quendor-rs, which is an offshoot of my quendor-py, which is an offshoot of my quendor. I was not writing the software itself yet. Just the plumbing around it. Commit conventions, enforcement hooks, CI checks, merge policy. The kind of thing every project needs and every tutorial makes look like a fifteen-minute task.

It took the better part my morning, and almost none of that time went to the work itself. It went to caveats. “One more thing about this.” “Oh, wait, that reveals this other thing.” By the end I was asking a question I know every developer has asked: why is everything like this? These ecosystems have existed for years. Shouldn’t they be sorted out by now? It’s one thing to know developers ask that question. It’s another to experience why they ask it. And I think testers are well served by knowing exactly that.

I ended my investigations with an actual answer, and it was one I came to long ago since I’ve been developing and testing for a long time. When I first ran into this, way back in the primeval era when dinosaurs still wore polyester, it changed how I look at this kind of friction. But the answer only makes sense against the specifics, so let me show you the specifics first.

The Setup

The project I’m working on, in all of its incarnations, is a Z-Machine interpreter written in Rust. The Z-Machine is the virtual machine that runs Infocom-era interactive fiction. That detail mostly doesn’t matter here. What matters is what I wanted from the project’s process, before the first real line of code:

1. Commit messages following the Conventional Commits specification.
2. Local enforcement, so a malformed message is rejected at commit time.
3. CI enforcement, so nothing malformed can land on main regardless of where it was committed.
4. A pull-request-only main branch with squash merging, where the PR title becomes the commit message.
5. Automated dependency updates, so the tooling doesn’t silently rot.

Nothing exotic. Every piece is mainstream: git, GitHub rulesets, GitHub Actions, the gh CLI, a commit-lint tool called cocogitto, its companion GitHub action, Dependabot. Each piece is mature, documented, and widely used.

Here’s what happened at every joint where two of those pieces met.

Five Failures, No Bugs

The tool that changed its meaning. The documented way to install cocogitto’s git hook used to be cog install-hook commit-msg. By version 7, that syntax no longer exists; the tool now installs hooks you declare in a config file. Fair enough. Except that running the new form, cog install-hook --all, with no config file present, exits successfully while installing all zero of the hooks you’ve defined. Green check mark, nothing enforced. I only discovered the hook wasn’t installed because I went looking for what the command had actually done. Which was nothing.

The action that changed its contract. The GitHub action for the same tool went from check: true in version 3 to a required command: check input in version 4. A workflow written from v3-era examples and naively bumped to v4 fails on first run. The CLI and the action are siblings from the same project, and even they don’t drift in sync.

The flag that works, except when it doesn’t. The gh pr merge command has a --delete-branch flag whose help text reads “Delete the local and remote branch after merge.” Combine it with --auto, meaning merge when checks pass … well, it turns out the deletion silently doesn’t happen, because gh merely enables auto-merge server-side and exits. So, when GitHub performs the merge minutes later, there’s no process left to delete anything. No warning tells you this. The truth is in the source code, in an early-return guard the documentation has never heard of. Better still: whether the flag works depends on timing: if the checks happen to already be green when you run the command, it degrades to an immediate merge and the deletion works exactly as documented.

The commit nobody wrote. With everything finally wired up, the very first pull request failed its own commit-message check. Every commit on the branch was valid. The failure was a commit that I didn’t write: on pull-request events, GitHub checks out a synthetic merge commit (effectively: “Merge abc123 into def456”) representing your branch as if merged. The lint tool, doing exactly its job, validated that machine-generated message and rejected it. Two correct behaviors, one broken result. The fix is a config setting that exists precisely because this collision is common enough to deserve
one.

The version nobody chose. The CI action, it turns out, bundles version 6.4 of the lint tool. My machine runs 7.0. Nothing broke. This time. But it means “passes locally” and “passes in CI” are answered by different
software, and someday that gap will likely be a bug report.

Notice what this list does not contain: a single defective tool. Every component did what its maintainers intended. All five failures live in the same place: the joint between two things.

Nobody Owns the Pairs

Here’s the explanation I originally landed on many, many moons ago and it’s the one that holds true today.

Each of these tools is owned. Somebody maintains cocogitto, somebody maintains gh, somebody maintains GitHub’s merge machinery. Within each tool, there’s a test suite, a release process, a place to file bugs.

The pairs of tools are owned by nobody. GitHub doesn’t test its synthetic merge commits against third-party commit linters. The action’s maintainer doesn’t coordinate releases with the CLI it wraps. The gh documentation team doesn’t re-derive every flag interaction when auto-merge semantics change upstream. Each maintainer’s responsibility ends at their own boundary (reasonably so!) which means the interactions between boundaries are maintained by no one.

And the interactions are where I spent my morning. Compose eight tools and you get dozens of pairwise seams, most of which have never been exercised by enough people to have their edges filed down. There is no mechanism by which a bazaar of independent maintainers converges on “sorted,” because sorted-ness is a property of compositions, and a composition belongs to the person doing the composing.

Which means, as a tester who dabbles in development, I recognized my role immediately: I was the integration test. First execution, production environment, no prior runs. Of course it found defects. That’s what integration tests are for.

That “I was the integration test” part is something that I believe is front and center to the relationship between developers and testers and, thus, between development and testing.

The Cathedral Next Door

Here’s the observation that keeps this from curdling into cynicism. While all of this was happening, one part of the stack produced zero caveats: the Rust toolchain. The compiler, the build tool, the formatter, the linter, the toolchain manager: flawless, all morning. Not one surprise. (That also applied to my Python and Node incarnations of this project.)

That’s not luck, and it’s not because Rust’s tools are simpler. It’s structure. The Rust toolchain is a cathedral: one project owns the compiler, the build system, the formatter, and the linter, so the seams between them are internal: somebody’s job to test before every release. The friction in my morning lived almost entirely in the policy-and-automation periphery around GitHub, which is maximally bazaar-shaped: independent tools, no release coordination, composed ad hoc by each user.

So the feeling that ecosystems should be sorted out by now isn’t naive. The sorted ones exist. They’re the ones with owners. What can’t ever be fully sorted is the open composition space around them. And that’s not a failure of engineering discipline; it’s the price of an ecosystem being open. Anyone can build a tool; no one is required to test it against every other tool. You get the bazaar’s variety and the bazaar’s seams together, or neither.

What This Buys You

Three consolations, all practical.

The cost is front-loaded. Setup day is expensive because setup is when you cross every seam for the first time. The loop I’ll actually live in for my project — edit, commit, open a PR, watch checks, squash-merge — now runs entirely on paths that have been walked and pinned down. Seams don’t need re-crossing daily.

Automation converts ambushes into signals. The two robots I installed exist precisely to keep it that way. Dependency updates turn version drift into visible pull requests instead of silent breakage. CI turns “the composition broke” into a red check instead of a corrupted main. The caveats don’t disappear. They stop being ambushes.

Empiricism is the only documentation that never lies. The recurring method that got me through the morning was: check the source, not the docs, and not my memory. The --delete-branch behavior wasn’t in any manual; it was in an if statement. The hook no-op wasn’t announced; it was discovered by looking at what the command actually wrote to disk. In a cathedral you can trust the manual. At the seams of a bazaar, the only reliable spec is the behavior itself. Which is to say: at the seams, everyone is a tester, whether they signed up for it or not.

That last sentence is probably the whole post. The tools were fine. The pairs were untested. (Or, at the very least, circumstantially and situationally tested.) And untested, circumstantially tested, and situationally tested things, when finally executed, behave exactly the way our profession has always said they would.