I was going to call this post “Interrogating My Own Code” but that was a bit opaque, even though it’s entirely accurate. I was then going to title it “When the Developer Is the Tester” because that’s part of what I’m talking about. But I ended up on what I believe is the more salient point.
This idea of the role, not a person is something I referenced a bit when I talked about mass extinctions in relation to testing. Trust me: it made sense at the time!
Here I want to dig into what it looks like when a developer tests. Note: not when a developer is a tester or when a tester is a developer. What I’m going to show you is code I wrote. There were times I was acting as a developer and times I was acting as a tester. I was never doing both simultaneously. Granted, the interleaving may have been quick in some cases, shifting back and forth rapidly, but the roles were always distinct.
It’s just like how you can’t truly multi-task. We aren’t wired for that. But we can context switch. Sometimes very rapidly.
This post is about code but you don’t have to be a coder to get value from this. At least I don’t think so. But I do think testers should have the ability to look at code and reason about it. At a bare minimum, that helps with automation. But beyond that, the much more important part is you start to learn the language of developers and the integration seams they deal with. You also start to build up a theory of error, understanding why we make mistakes when we build complex things. And you learn tests can put pressure on design.
Setting the Context
Some context before the code. The project this comes from is a Z-Machine interpreter written in Python. (It’s my Voxam project, if you’re curious.) The Z-Machine is the virtual machine Infocom built in 1979 to run its text adventures (Zork and its kin), and a “story file” is a compiled game for that machine. The first 64 bytes of every story file are a header: fixed offsets holding the version, release number, serial code, a checksum, and the addresses of structures the interpreter needs. The § citations you’ll see in the code refer to the Z-Machine Standard, the community specification this project treats as its requirements document. In what follows, Story is my class for a loaded file and Header is a typed view over those first 64 bytes.
Examining the Code
First, here’s some production code called story.py. To keep things simple, in all the listings that follow I’ve removed imports just to keep this relatively concise. Anything that looks undefined — HEADER_SIZE, the offset constants, the two error types — comes from the header module or a small shared errors module. I have kept all the comments because they can help situate the code.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
# The byte at address 0 holds the version number, 1 to 8 (§11.1). VERSION_RANGE = range(1, 9) @dataclass(frozen=True) class Story: """A story file held in memory, validated enough to identify it. Attributes: data: The raw bytes of the story file. """ data: bytes def __post_init__(self) -> None: """Reject byte content that cannot be a story file. Raises: StoryError: If the content is too short to hold a header or its version byte is out of range. """ if len(self.data) < HEADER_SIZE: msg = ( f"story file is {len(self.data)} bytes, but the header " f"alone requires {HEADER_SIZE} (§1.1.1.1)" ) raise ZMachineStoryError(msg) if self.data[0] not in VERSION_RANGE: msg = ( f"story file declares version {self.data[0]}, but only " f"versions 1 to 8 exist (§11.1)" ) raise ZMachineStoryError(msg) @classmethod def load(cls, path: Path) -> Self: """Read and validate a story file from disk. Args: path: Location of the story file. Returns: The loaded story. Raises: StoryError: If the file cannot be a story file. """ return cls(path.read_bytes()) @property def header(self) -> Header: """Typed access to this story's header fields (§11.1).""" return Header(self.data) @property def version(self) -> int: """The Z-Machine version this story targets (§11.1).""" return self.header.version |
Here’s another bit of production code called header.py.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 |
# Dynamic memory must contain at least 64 bytes (§1.1.1), and the first 64 # bytes are the header (§1.1.1.1), so no story file can be shorter. HEADER_SIZE = 64 # Field locations from the table in §11.1. RELEASE = 0x02 HIGH_MEMORY_BASE = 0x04 INITIAL_PC = 0x06 DICTIONARY = 0x08 OBJECT_TABLE = 0x0A GLOBAL_VARIABLES = 0x0C STATIC_MEMORY_BASE = 0x0E SERIAL_START = 0x12 SERIAL_END = 0x18 ABBREVIATIONS_TABLE = 0x18 FILE_LENGTH = 0x1A CHECKSUM = 0x1C # The file length is stored divided by a version-dependent constant # (§11.1.6). FILE_LENGTH_SCALE = {1: 2, 2: 2, 3: 2, 4: 4, 5: 4, 6: 8, 7: 8, 8: 8} # Verification sums the bytes from $0040 up to the stored file length, # modulo $10000; padding beyond that length must be excluded (§15, verify). CHECKSUM_START = 0x40 CHECKSUM_MODULO = 0x10000 # In Version 6 the word at $06 is the packed address of a "main" routine # rather than the byte address of a first instruction (§11.1). PACKED_PC_VERSION = 6 @dataclass(frozen=True) class Header: """A read-only view of the header fields within story file memory. Attributes: data: The full story file bytes, of which the first 64 form the header (§1.1.1.1). """ data: bytes def __post_init__(self) -> None: """Reject byte content too short to contain a header. Raises: ZMachineHeaderError: If fewer than 64 bytes are present. """ if len(self.data) < HEADER_SIZE: msg = ( f"a header requires {HEADER_SIZE} bytes, but only " f"{len(self.data)} are present (§1.1.1.1)" ) raise ZMachineHeaderError(msg) def _word(self, offset: int) -> int: """Read the big-endian word at a byte offset (§2.1).""" return int.from_bytes(self.data[offset : offset + 2], "big") @property def version(self) -> int: """The Z-Machine version this story targets (§11.1).""" return self.data[0] @property def release(self) -> int: """The release number of this story (§11.1).""" return self._word(RELEASE) @property def serial_number(self) -> str: """Six ASCII characters, conventionally the compile date (§11.1).""" return self.data[SERIAL_START:SERIAL_END].decode("ascii") @property def declared_file_length(self) -> int: """The story length in bytes, unscaled from the header word (§11.1.6). The file on disk may be longer than this: interpreters must allow for padding beyond the declared length (§15, verify remarks). """ return self._word(FILE_LENGTH) * FILE_LENGTH_SCALE[self.version] @property def stored_checksum(self) -> int: """The checksum the compiler recorded at $1c (§11.1).""" return self._word(CHECKSUM) @property def computed_checksum(self) -> int: """The checksum of the story bytes actually present (§15, verify).""" story = self.data[CHECKSUM_START : self.declared_file_length] return sum(story) % CHECKSUM_MODULO @property def high_memory_base(self) -> int: """The byte address at which high memory begins (§11.1).""" return self._word(HIGH_MEMORY_BASE) @property def dictionary_address(self) -> int: """The byte address of the dictionary (§11.1).""" return self._word(DICTIONARY) @property def object_table_address(self) -> int: """The byte address of the object table (§11.1).""" return self._word(OBJECT_TABLE) @property def global_variables_address(self) -> int: """The byte address of the global variables table (§11.1).""" return self._word(GLOBAL_VARIABLES) @property def static_memory_base(self) -> int: """The byte address at which static memory begins (§11.1).""" return self._word(STATIC_MEMORY_BASE) @property def abbreviations_table_address(self) -> int: """The byte address of the abbreviations table (§11.1).""" return self._word(ABBREVIATIONS_TABLE) @property def initial_program_counter(self) -> int: """The byte address of the first instruction to execute (§11.1). Raises: ZMachineHeaderError: In Version 6, where the word at $06 is a packed routine address instead. """ if self.version == PACKED_PC_VERSION: msg = ( "version 6 stores a packed routine address at $06, not an " "initial program counter; use main_routine_packed_address " "(§11.1)" ) raise ZMachineHeaderError(msg) return self._word(INITIAL_PC) @property def main_routine_packed_address(self) -> int: """The packed address of the initial routine in Version 6 (§11.1). Unpacking this address requires the rules of §1.2.3, which will arrive alongside routine calls. Raises: ZMachineHeaderError: In any version other than 6, where the word at $06 is a byte address instead. """ if self.version != PACKED_PC_VERSION: msg = ( f"version {self.version} stores an initial program counter " f"at $06, not a packed routine address; use " f"initial_program_counter (§11.1)" ) raise ZMachineHeaderError(msg) return self._word(INITIAL_PC) def verify(self) -> bool: """Report whether the computed and stored checksums agree (§15). Some early Version 3 files store no length or checksum at all (§11.1), so a mismatch against a stored zero may mean "absent" rather than "corrupt". """ return self.computed_checksum == self.stored_checksum |
Observing The Code
It may look like a lot but it’s relatively simple code when all is said and done. Now, I would ask you, as a curious tester, to take a look at the code and just reason about it a bit. I want you to see if you can spot one of the potential enemies of quality.
Go ahead. Take a look before reading on. I’ll wait.
I hasten to add: this requires nothing beyond the Mark 1 eyeball. Even with no coding skills whatsoever or never having seen Python in your life, that does not stop the ability to discern what I’m talking about here.
If you’re not sure what I’m talking about, the primary enemy of quality is opacity. And this shows up in various ways: ambiguity, inconsistency, and contradiction. One element that happens in specs or code is an aspect of those: duplication. With that hint provided, take a look again.
Finding anything?
There are two bits of duplication here worth spotting. That’s pretty much the final hint. Before you move on, make sure you see them.
Ready?
First Issue
Okay, so, before we even get to the test for this code, let’s talk about those duplications. As a tester, you should be able to note that the __post_init__ in header.py has a check that matches one of the conditions in the __post_init of story.py. Is that deliberate?
Yes. Deliberate. But the instinct to interrogate it is correct. Now let me give you the actual reasoning, as I see it, and then where I would (and did) draw the line.
The two checks share a predicate (len(data) >= 64) but they’re guarding different things, and that difference shows up in everything around them.
- Story’s check answers “can this content be a story file at all?” That’s an admissibility judgment about a file, raising ZMachineStoryError, cited to the file-level rule (§1.1.1 in the spec).
- Header’s check answers “can this view safely read its fields?”. That’s a precondition of a class, raising ZMachineHeaderError. Without it, code like
Header(b"\x03")doesn’t crash. Python slicing is forgiving, so calling the_word()method on truncated bytes quietly returns garbage (an empty slice becomes 0). A silently-wrong header is far worse than a loud one.
What matters here is the design principle underneath: each frozen dataclass defends its own invariant at construction, without assuming who constructed it. Header is public and independently constructible; the tests I’m going to show you soon build it directly from tampered bytes, bypassing Story entirely. If Header relies on “someone upstream already checked,” the check is only as good as every current and future call site. That’s the make-invalid-states-unrepresentable principle, applied per class. It’s also redundancy in the defense-in-depth sense, not the copy-paste (duplication) sense.
That distinction, shown with one of the simplest examples possible, is what it means to be a developer thinking like a tester at a particular moment in time, whether that’s before writing the code, while writing the code, or after writing the code..
I call that out because one part of the role of a tester is to show, not just tell. And I see a lot of sentiment out there that “developers should test better.” Well, if you think that, you should be able to show what that means in their context.
I should also note that I’m dealing with internal qualities here. Those may bubble up to external qualities.
I talk about these quality distinction when I talk about my role as a quality and test specialist and I provide an example (in a gaming context) when I talk about navigating qualities.
That said, if you did find the above code (and test!) smells, I do want to affirm that what you detected is real, and here’s the honest part: this duplication is a symptom of both classes reading raw bytes directly.
As a developer, I’m thinking that ends naturally in the next branch of work which would be implementing a memory model. Once a Memory class owns the bytes and provides bounds-checked reads, both Story and Header read through it, and “are there enough bytes?” gets exactly one home. At that point Header’s check likely dissolves into Memory’s bounds checking rather than being deleted outright.
So, as an engineer acting in the developer role, I wrote the above code. As that same engineer, acting in the tester role looking at the code I wrote, my recommendation to myself was: keep both for this branch and let the memory branch absorb the duplication properly, rather than pre-solving it now by coupling Story to Header (which would mean Story raising header-flavored errors for what is really a truncated-file problem, or constructing throwaway objects in __post_init__).
Now, if I just couldn’t abide the thought of duplication even for a short duration, and if I were going to trim one of them, the one that must survive is Story’s. Without it, self.data[0] on an empty file raises a bare IndexError before any friendly error can fire. Dropping Header’s would also mean dropping a crucial test (which I’ll be showing you called test_rejects_content_too_short_for_a_header) and accepting that a directly-constructed short Header misreads silently. I wouldn’t make that trade, but it is a coherent position if you document “construct via Story.header” as the contract.
Second Issue
Okay, so that’s one duplication. I mentioned there were two. Hopefully you spotted the second and now, given the above narrative, let’s ask: Does that same logic apply to the version property which, for now, exists duplicated on both story and header?
Answer: No! And the distinction between the two cases is worth making crisp, because it’s the difference between duplication that’s doing work and duplication that isn’t.
The length checks we dealt with above are two different assertions that happen to share a predicate: different questions (“admissible file?” vs. “safe view?”), different exceptions, different failure consequences. Deleting either one loses something real. By contrast, version is one fact with two copies. Both read data[0], both mean the same thing, both cite §11.1, and neither adds any protection the other lacks. There’s no defense-in-depth story here. If the two ever disagreed it could only be a bug in one of them, which is exactly the situation duplication creates and delegation makes impossible.
So, acting as the tester to my developer, my recommendation: dedupe this one. The authoritative home is Header and that’s because the version number is a header field; that’s its spec-given address. Story.version is still worth keeping as ergonomic surface, but as a delegation:
|
1 2 3 |
@property def version(self) -> int: return self.header.version |
One definition of the read, one place a future fix would go. The Header construction per access is a frozen dataclass wrapping the same bytes reference: effectively free.
And that is, in fact, the change I made. The story.py listing earlier shows the pre-change state deliberately, so that you could spot the duplication yourself before I resolved it.
One boundary note so the pattern doesn’t over-generalize: Story.__post_init__ should keep reading self.data[0] raw rather than going through self.header.version. Validation code runs before the object has earned its invariants, and routing it through Header would re-entangle the two classes at exactly the seam we just discussed keeping separate. Plus it works only while Story happens to check length first. Raw access during validation, delegated access after: that’s the general rule, and, as a developler, I konw thatwill come up again when Memory arrives.
A useful heuristic to carry forward: when you spot duplication, ask “if these two copies diverged, would that ever be correct?” If yes (the length checks; one could legitimately become stricter), they’re separate things. If no (version), they’re one thing written twice.
Testing the Code
Now let’s look at the tests. Here is test_story.py:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
from pathlib import Path import pytest from assertpy import assert_that from voxam.errors import ZMachineStoryError from voxam.zmachine.header import HEADER_SIZE from voxam.zmachine.story import Story def story_bytes(version: int = 3, size: int = HEADER_SIZE) -> bytes: return bytes([version]) + bytes(size - 1) def test_rejects_content_too_short_for_header() -> None: with pytest.raises(ZMachineStoryError, match="63 bytes"): Story(story_bytes(size=HEADER_SIZE - 1)) def test_rejects_empty_content() -> None: with pytest.raises(ZMachineStoryError, match="0 bytes"): Story(b"") @pytest.mark.parametrize("version", range(1, 9)) def test_accepts_every_valid_version(version: int) -> None: story = Story(story_bytes(version=version)) assert_that(story.version).is_equal_to(version) @pytest.mark.parametrize("version", [0, 9, 255]) def test_rejects_versions_that_do_not_exist(version: int) -> None: with pytest.raises(ZMachineStoryError, match=f"version {version}"): Story(story_bytes(version=version)) def test_loads_from_disk(tmp_path: Path) -> None: path = tmp_path / "test.z3" path.write_bytes(story_bytes(version=3)) story = Story.load(path) assert_that(story.version).is_equal_to(3) |
And here is test_header.py:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 |
FIXTURES = Path(__file__).parent.parent / "fixtures" ALL_VERSIONS = range(1, 9) def fixture_story(version: int) -> Story: (path,) = FIXTURES.glob(f"simple-test-r*-s260727.z{version}") return Story.load(path) def synthetic_header(version: int = 3, words: dict[int, int] | None = None) -> bytes: data = bytearray(HEADER_SIZE) data[0] = version for offset, value in (words or {}).items(): data[offset : offset + 2] = value.to_bytes(2, "big") return bytes(data) # The expected releases record what Inform actually emits: 0 for its # Version 1 and 2 targets, 1 for Version 3 and later. The fixture file # names carry the same values. @pytest.mark.parametrize( ("version", "release"), [(1, 0), (2, 0), (3, 1), (4, 1), (5, 1), (6, 1), (7, 1), (8, 1)], ) def test_reads_identity_of_every_fixture(version: int, release: int) -> None: header = fixture_story(version).header assert_that(header.version).is_equal_to(version) assert_that(header.release).is_equal_to(release) assert_that(header.serial_number).is_equal_to("260727") def test_rejects_content_too_short_for_a_header() -> None: with pytest.raises(ZMachineHeaderError, match="63"): Header(bytes(HEADER_SIZE - 1)) @pytest.mark.parametrize("version", ALL_VERSIONS) def test_declared_length_never_exceeds_actual_size(version: int) -> None: story = fixture_story(version) assert_that(story.header.declared_file_length).is_less_than_or_equal_to( len(story.data) ) @pytest.mark.parametrize( ("version", "scaled_length"), [ (1, 512), (2, 512), (3, 512), (4, 1024), (5, 1024), (6, 2048), (7, 2048), (8, 2048), ], ) def test_scales_declared_length_by_version(version: int, scaled_length: int) -> None: header = Header(synthetic_header(version=version, words={0x1A: 0x0100})) assert_that(header.declared_file_length).is_equal_to(scaled_length) def test_computes_checksum_by_the_spec_rule() -> None: story = synthetic_header(words={0x1A: 288, 0x1C: 0xFE00}) story += bytes([0xFF]) * 512 story += bytes([0xAA]) * 10 header = Header(story) assert_that(header.computed_checksum).is_equal_to(0xFE00) assert_that(header.verify()).is_true() @pytest.mark.parametrize("version", ALL_VERSIONS) def test_every_fixture_passes_verification(version: int) -> None: header = fixture_story(version).header assert_that(header.verify()).is_true() def test_detects_corruption_through_verification() -> None: data = bytearray(fixture_story(3).data) data[CHECKSUM_START] ^= 0xFF header = Header(bytes(data)) assert_that(header.verify()).is_false() @pytest.mark.parametrize("version", [v for v in ALL_VERSIONS if v != PACKED_PC_VERSION]) def test_initial_program_counter_lies_within_the_file(version: int) -> None: story = fixture_story(version) pc = story.header.initial_program_counter assert_that(pc).is_greater_than_or_equal_to(HEADER_SIZE) assert_that(pc).is_less_than(len(story.data)) def test_version_6_stores_a_packed_routine_address() -> None: header = fixture_story(6).header assert_that(header.main_routine_packed_address).is_greater_than(0) def test_version_6_refuses_an_initial_program_counter() -> None: header = fixture_story(6).header with pytest.raises(ZMachineHeaderError, match="packed routine address"): _ = header.initial_program_counter def test_other_versions_refuse_a_packed_routine_address() -> None: header = fixture_story(5).header with pytest.raises(ZMachineHeaderError, match="initial program counter"): _ = header.main_routine_packed_address def test_reads_field_values_from_spec_offsets() -> None: header = Header( synthetic_header( words={ 0x02: 0x0102, 0x04: 0x2030, 0x06: 0x2233, 0x08: 0x0400, 0x0A: 0x0500, 0x0C: 0x0600, 0x0E: 0x0700, 0x18: 0x0800, 0x1C: 0xBEEF, } ) ) assert_that(header.release).is_equal_to(0x0102) assert_that(header.high_memory_base).is_equal_to(0x2030) assert_that(header.initial_program_counter).is_equal_to(0x2233) assert_that(header.dictionary_address).is_equal_to(0x0400) assert_that(header.object_table_address).is_equal_to(0x0500) assert_that(header.global_variables_address).is_equal_to(0x0600) assert_that(header.static_memory_base).is_equal_to(0x0700) assert_that(header.abbreviations_table_address).is_equal_to(0x0800) assert_that(header.stored_checksum).is_equal_to(0xBEEF) |
There’s lots here a tester might glom onto, but one thing that should stick out is:
|
1 |
[(1, 0), (2, 0), (3, 1), (4, 1), (5, 1), (6, 1), (7, 1), (8, 1)], |
As a tester that would immediately trigger my Spidey Sense. Why isn’t the release number uniform across eight files compiled from the same source? And why should anyone trust that table? Answering that requires a confession, because the story behind those two columns is the most tester-flavored part of this whole exercise.
The FIXTURES being referred to are eight tiny story files, one per Z-Machine version, all compiled from the same source. When I first added them, every one was named with r0 — release 0 — because that’s what I believed the release to be. Then, while writing these tests, I misremembered the value twice in a row (“they’re all 1” … “no, wait, all 0”) and finally did what I should have done first: looked at the actual bytes. The bytes said neither. They said 0, 0, 1, 1, 1, 1, 1, 1.
Before trusting that reading — because at this point my own header-reading code was as much a suspect as my memory — I cross-checked with a small, completely independent script I’ve had for a while that parses Z-code headers using none of this project’s code. It agreed: 0, 0, then 1 for the rest. It turns out the Inform compiler emits release 0 for its Version 1 and 2 targets and release 1 for Version 3 and later. My code was right. My test data’s names were wrong.
So the fix went into the fixtures, not the code: I renamed the files to tell the truth.
- simple-test-r0-s260727.z1
- simple-test-r0-s260727.z2
- simple-test-r1-s260727.z3
- simple-test-r1-s260727.z4
- simple-test-r1-s260727.z5
- simple-test-r1-s260727.z6
- simple-test-r1-s260727.z7
- simple-test-r1-s260727.z8
That rename is what makes the release assertion legitimate at all. The generalizable rule, which will recur every time I look at fixture tests: assert fixture values only when the assertion is anchored to something outside the file itself; in this case, the filename’s serial code, its extension’s version. Otherwise, test the behavior with synthetic bytes where you chose the expected value. Fixtures prove the interpreter handles reality; synthetic bytes prove it handles each field correctly.
But notice the trap I nearly walked into. Renaming files to match the bytes and then calling the names an “external anchor” would be circular reasoning! The anchor would just be the bytes wearing a costume. What breaks the circle is the independent cross-check: a second implementation, sharing no code with mine, agreeing on what the bytes say. Only then did the filenames become trustworthy anchors rather than echoes.
With truthful names, the release assertion becomes exactly what the serial assertion is: a fixture-identity check anchored outside the bytes. The explicit table, rather than parsing r0 and r1 back out of the filenames,- keeps the expected values visible in the test, and the comment above it records why they differ. The lesson I would want a tester to carry out of this: sometimes the bug is in the test data. Fixture metadata can lie just as easily as code can, and it deserves the same scrutiny.
Let’s also consider the checksum tests. Look back at the test listing and ask a tester’s question: which of those checksum tests is doing a job none of the others can?
At an earlier point in this work, every checksum test I had was relational: computed == stored for the fixtures, or != after tampering. Nothing pinned computed_checksum to an exact value that could be verified by hand. The fixture tests do have a real oracle hiding in them (Inform computed those stored checksums independently, so eight agreements is strong evidence) but no relational test would catch, say, a start-boundary error that happened to cancel out. A strict test fixes the expected value by arithmetic, not by the code’s own output.
That gap is why the test test_computes_checksum_by_the_spec_rule exists. It’s one test, but it pins four properties of the algorithm at once, each of which would independently break the exact value.
- Start boundary: if summing began at $00 instead of $40, the version byte and the two planted header words would leak in (+290).
- End boundary: the sum must stop at the declared length (576), not the actual file size.
- Padding exclusion: the ten 0xAA bytes exist purely to be wrongly includable (+1700 if the code used
len(data)). - Modulo wrap: 0x1FE00 exceeds a word, so a missing % 0x10000 produces 130560, not 65024.
That’s the anatomy of a strict test for any spec algorithm: choose inputs so that every plausible mistake perturbs the hand-computed answer. The relational tests I already have then do the complementary job, which is proving the algorithm agrees with reality (Inform’s stored values) rather than just with my arithmetic.
With this, the checksum behavior is fully specified by tests: exact value (this), real-world agreement (eight fixtures), corruption detection (the tamper test), and field read (the 0xBEEF synthetic).
Careful? Lucky? Or disciplined?
Notice what actually happened with those release numbers, though, before I move on. I didn’t catch my own mistake by being careful. I caught it by not trusting myself. By treating my own memory of “what the bytes probably say” as a claim that needed a second, independent witness before it could stand. That’s not a virtue I always have as a person. In what will come as a shock to no one, I misremember things and take shortcuts like anyone else. What I count on is the discipline that the tester role imposes on me from outside.
That’s the whole essay, really, compressed into one small embarrassing anecdote. The developer wrote code he believed was correct. The person who wrote it also believed the fixture names were correct, twice, in two different wrong directions. Neither of those beliefs was worth anything on its own. What broke the tie wasn’t more confidence; it was a second implementation that owed the first one nothing, and a role willing to ask “but how do you know?” even of its own author.
The checksum test does the same job at a smaller scale. It doesn’t exist because I’m thorough. It exists because, standing in the tester’s role, I asked the developer’s role a question he hadn’t thought to ask himself: “if you were wrong about the boundary, or the padding, or the modulo, would this test even notice?” The honest answer was no, not with what I had. So the role went and built a test that would notice: four ways a plausible mistake could hide, each one now with nowhere left to go.
None of this required two people. It required one person willing to keep switching hats, quickly enough and honestly enough that neither hat got to relax into assuming the other one already checked. That is the discipline. Not “developers should test better,” as an instruction handed down from outside, but a role, occupied deliberately, asked to interrogate a person who is, like many of us, a pretty unreliable narrator of his own work.
