In this post, I’ll give you some idea of how the Grue project was set up, which involves talking about the programming language and ecosystem being used. This is the kind of information that, as a tester, I like to know. Certainly for anyone looking to develop a Z-Machine implementation, these details matter.
As brought up in the first post in this series, the goal here will be to write a Z-Machine implementation.
What does that actually mean, though?
Z-Machine Implementation
What the above means is that Grue has to act both as an emulator (software that operates like the hardware of a machine) and an interpreter (software that takes code as input and executes it).
So let’s break this down a little bit.
In order to write an emulator, you need a specification for the machine in question. I’ll be using the Z-Machine Standards Document, version 1.0 as a starting point. There is also a version 1.1 but I’m uncertain how much I’ll worry about that for Grue.
In order to write an interpreter, and also to test the emulator that you’re writing, you need the very thing that makes a Z-Machine useful to emulate in the first place: zcode programs. Andrew Plotkin has made available The Obsessively Complete Infocom Catalog which is a great resource that provides the story files that were produced by Infocom. I also have my own zcode catalog that serves as a reference location for many zcode programs.
A very large goal of mine for these posts is to show someone how to write a Z-Machine emulator and interpreter, thus serving as a bit of a pedagogical device. I’ll intersperse this here and there with some details relevant to this blog, which is, of course, primarily about testing.
Python
As with any such project, you have to choose a programming language and ecosystem you are going to work within. I’m going to go with Python for this project. I grew to really enjoy working in Python when I worked on my Pacumen project.
To support testability, I’m going to keep the ecosystem that I work within really simple. I’m not going to be too worried about Python versions but I plan for Grue to support from Python 3.8.2 up to the current versions. I will be using Poetry for the virtual environment manager but, realistically, it doesn’t matter. I don’t plan on using a whole lot of extra libraries or build dependencies here because my goal is to keep things as simple as possible. That said, there will be some development dependencies, like code quality tools (linters) and a test runner.
Gather the Oracles
Any time you’re designing something like a virtual machine, beyond having a specification and beyond having example programs that run on that machine, it’s good to have some means of disassembling those programs to see how they are constructed. In this context, there are some tools known as the ztools.
I’ve provided the two most important of those as Mac/Linux versions (txd, infodump) or Windows versions (txd.exe, infodump.exe). You would run these tools against existing zcode programs, such as the Infocom text adventures. I’ve provided a series of those programs which are the primary programs I’ll use to test Grue.
You’ll notice the extensions there and this has to do with the fact that there are multiple “versions” or generations of the Z-Machine. Each such generation was an update to the previous one and for any such emulator, you need to be able to test those various versions. To my knowledge, there isn’t a version 4 (z4) of the Zork game and there weren’t many version 4 games made anyway. That’s where Trinity comes in above. I put Mini-Zork in there because that’s a simplified version of the Zork game that a lot of people use when building a Z-Machine.
For each of the above files, you can use the ztools to generate data dumps of information. Here’s an example of how I use the tools and the particular switches to generate the information I need:
./txd -andw0 zork1.z3 > zork1.z3.txd.txt ./infodump -fw0 zork1.z3 > zork1.z3.infodump.txt
I have those examples ready to go as zork1.z3.infodump.txt and zork1.z3.txd.txt.
Beyond having those zcode programs and having a disassembly of them, what’s even nicer is if you can get your hands on the source code of the programs themselves. A challenge with disassembly is that it’s an abstracted view of what the logic is doing. This can make it hard to reason about how the program is behaving in a specific context. Thus I gathered up the source code:
This level of effort is what a lot of developers have to go through when they build complex things. Here I’ve had to see if there was a specification (there is) and if it’s good (it’s … okay). I had to see if there were example programs I could use to test the machine I build based on the specification (there are). I had to see if I had a way to understand the binary data of those zcode programs (I do) and if I could get the actual source contents of that binary data (I could).
One thing to note here: those zcode programs, that disassembly data, and that source code are all oracles. You could argue that in the world of testing an oracle is something that guides your understanding of what you are seeing. In this case, beyond helping understand if the implementation matches the specification, those oracles also help to recognize bugs in the implementation.
The zcode programs are what we would call authoritative oracles. They were released to the public at large, they are executable, and they are demonstrably how the programs are supposed to function when they run on a Z-Machine. Which, interestingly enough, doesn’t mean they are entirely without bugs in implementation.
The specification is likewise an authoritative oracle as it was the agreed upon standard that a community wrote to specify how a Z-Machine works. The disassembly and the source code could be said to be suggestive oracles. In one sense, I can relate them back to the zcode program but in terms of helping to recognize a bug, their “demonstrable standard” is a bit lower because they are not directly executable.
What’s somewhat interesting is that if you do in fact build a specification-compliant Z-Machine emulator and interpreter, you have then effectively created an executable specification.
Getting Set Up
If your goal through this series of posts is simply learn how to build a Z-Machine interpreter, there is no reason to directly follow along with the exact implementation of my code. But you might undertake to build your own Z-Machine implementation based on mine. You might even choose to follow along with this series. As such, knowing how to get the code on your local machine and execute it is pretty crucial.
Likewise, I think it’s very crucial for testers to be able to get at and see the same artifacts that developers use as they develop their applications, particularly if those artifacts are oracles.
I also think it’s pretty crucial that testers are able to build a local copy of the code base on their own machines and are able to run it. It’s dismaying how many times I see this is not the case in my professional career.
The repo is available at Grue.
I personally like to see how any projects are setup. In the case of Grue, I mentioned I’m using Poetry and thus I’m also using a pyproject.toml file. In that file, you can look under the section called “tool.poetry.group.dev.dependencies”.
As a tester, what the list would show me is that a test runner is being used (pytest) along with an assertions or expectations library (expects). I see that a formatting tool is being used (black) and that a linting tool is used (flake8). I also see that a lot of flake8 related libraries are being used. I also see a static type checker is in place (mypy).
If I see stuff like that, I know the developers are possibly taking internal qualities seriously. That, of course, presumes they have an easy way of running all those tools and looking at the results. Which I should likewise be able to do as a tester. In the case of this project, I’m using invoke to make it easy to run everything I need.
If you choose to use Poetry, you can run:
poetry install
That will set up the virtual environment and install the necessary dependencies. You can run the program by:
poetry run grue
Just call one of the zcode programs that you got from one of the resources I provided above. I set up an alias called gcheck that runs the following:
poetry run invoke check
That causes all the tasks in the tasks.py module to execute.
Again, none of this is necessary if you are just reading along to learn more about the Z-Machine and how to implement one. But if you did want to start with your own project, my own repo is probably not a bad start to that. You can check out my tag “0.1.0” from the repository if you just want the bare minimum starting state. The repository will be flagged with various tags to make jumping in at various points easier.
With this set up out of the way, the next post in this series will begin discussing exactly how to implement a Z-Machine.
