In this post, I’ll give you an idea of how the Grue project is set up, which involves discussing the programming language and ecosystem to be used. This is the kind of information that, as a tester, I like to know. Indeed, these details matter for anyone looking to develop a Z-Machine implementation.
As brought up in the first post in this series, the goal here will be to write a Z-Machine implementation. What does that actually mean, though?
Z-Machine Implementation
What the above means is that Grue has to act both as an emulator (software that operates like the hardware of a machine) and an interpreter (software that takes code as input and executes it). Let’s break this down.
To write an emulator, you need a specification for the machine in question. I’ll be using the Z-Machine Standards Document, version 1.0 as a starting point. There is also a version 1.1, but I’m uncertain how much I’ll worry about that for Grue.
To write an interpreter and test your emulator, you need the very thing that makes a Z-Machine worthwhile to emulate in the first place: zcode programs. Andrew Plotkin has made available The Obsessively Complete Infocom Catalog, which is a great resource that provides the story files that were produced by Infocom. I also have my own zcode catalog that serves as a reference location for many zcode programs.
One of my goals for these posts is to show someone how to write a Z-Machine emulator and interpreter, thus serving as a pedagogical device. I’ll intersperse this here and there with some details focused on testing, which will have some relevance to this blog’s primary content. But what language am I going to write this thing in?
Python
As with any such project, you must choose a programming language and ecosystem you will work within. I’m going to go with Python for this project. I grew to really enjoy working in Python when I worked on my Pacumen project.
To support testability, I’m going to keep the ecosystem that I work within really simple. I’m not too worried about Python versions, but I plan for Grue to support Python 3.8.2 up to the current versions. I will be using Poetry for the virtual environment manager, but realistically, it doesn’t matter.
If you know Python at all, you probably know that it’s packaging ecosystem has always been a bit of a mess. Two good articles that showcase this are a call to improve Python packaging and then a one year later follow up to that. Spoiler: not much was improved.
I plan to use only a few extra libraries or build dependencies here to keep things as simple as possible. There will be some development dependencies, like code quality tools (linters) and a test runner.
Getting Reference Information
Any time you’re designing something like a virtual machine, beyond having a specification and example programs that run on that machine, it’s good to have some means of disassembling those programs to see how they are constructed. In this context, there are some tools known as the ztools.
I’ve provided the two most important of those as Mac/Linux versions (txd, infodump) or Windows versions (txd.exe, infodump.exe). You would run these tools against existing zcode programs like the Infocom text adventures. I’ve provided a series of the primary programs I’ll use to test Grue.
I want to bring to your attention the extensions of those files, which have to do with the fact that there are multiple “versions” or generations of the Z-Machine. Each such generation was an update to the previous one. For any such emulator, you need to be able to test those various versions. To my knowledge, there isn’t a version 4 (z4) of the Zork game, and there weren’t many version 4 games made anyway. That’s where Trinity comes in above. That happens to be my favorite Infocom game. I put Mini-Zork in there because that’s a simplified version of the Zork game that a lot of people use when building a Z-Machine.
You can use the ztools to generate data dumps for the above files. Here’s an example of how I use the tools and the particular switches to generate the information I need:
./txd -andw0 zork1.z3 > zork1.z3.txd.txt
./infodump -fw0 zork1.z3 > zork1.z3.infodump.txt
I have those examples ready to go as zork1.z3.infodump.txt and zork1.z3.txd.txt.
Beyond having those zcode programs and having a disassembly of them, what’s even nicer is if you can get your hands on the source code of the programs themselves. A challenge with disassembly is that it’s an abstract view of what the logic is doing. This can make it hard to reason how the program behaves in a specific context. Thus, I gathered up the source code:
Many developers have to go through this level of effort when they build complex things. Considering the case of Grue, I’ve had to see if there was a specification (there is) and if it’s good (it’s … okay). I had to see if there were example programs I could use to test the machine I would build based on the specification (there are). I had to know if I had a way to understand the binary data of those zcode programs (I do) and if I could get the actual source contents of that binary data (I could).
Oracles
One thing to note is that those zcode programs, disassembly data, and source code are all oracles. In the testing world, an oracle guides your understanding of what you see. In this case, beyond helping to understand if the implementation matches the specification, those oracles also help to recognize bugs.
The zcode programs are authoritative oracles. They were released to the public at large, are executable, and are demonstrably how the programs are supposed to function when they run on a Z-Machine. Interestingly, even while being authoritative oracles, those programs may still have bugs in their implementation. The specification is likewise an authoritative oracle as it was the agreed-upon standard that a community wrote to specify how a Z-Machine works.
The disassembly and the source code are suggestive oracles. In one sense, I can relate them back to the zcode programs they were derived from. Still, in terms of helping to recognize a bug, their “demonstrable standard” is a bit lower because they are not directly executable nor do they, just by existing as a working implementation, necessarily follow the standard specification.
An interesting point to consider here is that if you do, in fact, build a specification-compliant Z-Machine emulator and interpreter, you have then effectively created an executable specification.
In the next post, I’ll show the basis of the Python implementation and, from that, I can actually start doing something.
