Select Mode

Memory Constraints in Grue

In this post I want to talk about some of the memory constraints that the Z-Machine specification indicates exist for any zcode program. These are constraints that any interpreter, like Grue, would presumably have to respect.

I was originally going to have some code in this post regarding those constraints but I realized the specification was unclear to me on a series of points. So this post will be more descriptive in nature as I try to determine if I understand what I’m reading and thus at least have some inkling of what I will have to do.

A Memory Ceiling

One thing I found in the Z-Machine specification was this statement:

Note that the total of dynamic plus static memory must not exceed 64K. (In fact, 64K minus 2 bytes.) This is the most serious limitation on the Z-machine.

That’s actually in a “remark” in the part about the memory map. What struck me here is that this statement seems to imply that the high memory is technically above the 64 kilobyte memory that could be addressed by 16-bit pointers, which would be inconsistent with what I said in previous posts.

I say this because if the dynamic + static memory can be up to 64 kilobytes but there is also a high memory, then, by definition, the high memory would have to be above 64 kilobytes. However, before I got into that, I wanted to figure out what the “minus 2 bytes” thing was about.

The result of my figuring it out is: I have no idea. It doesn’t seem to make sense.

What Does 64K – 2 Mean?

The issue here is the vagueness in how the Z-Machine specifies the boundaries between memory regions. Consider that the specification also says this of static memory:

Its extent is not defined in the header (or anywhere else), though it must end by the last byte of the story file or by byte address $0ffff (whichever is lower).

The only plausible interpretation of the 2-byte reduction I could come up with is the following.

  • Static memory must end by the last byte of the story file or by byte address $0FFFF, whichever is lower. The phrase “must end by” might mean “at” rather than “before.” If we interpret this as meaning static memory can end exactly at $0FFFF, that means static memory and high memory would meet at the upper limit of addressable space, and no overlap is allowed.
  • If high memory starts at $0FFFF (the last byte of the 64K space), then static memory could, at best, extend back to $0FFFE. That means the last usable byte of dynamic memory could only be $0FFFD. This gives us the “64K minus 2 bytes” outcome, where dynamic and static memory combined must end before the very last byte of addressable space.

The key here would be ensuring that dynamic, static, and high memory don’t overlap. The idea that the lowest high memory can start is $0FFFF and the last usable address for static memory is $0FFFE fits well with the need to prevent overlap.

The problem is that this is not an interpreter concern. It’s the compiler’s responsibility, not the interpreter’s, to enforce these constraints. The interpreter only runs the bytecode produced by the compiler, so any interpreter assumes that the memory layout it receives from any compiled zcode program is valid.

So, my suggestion would be that the “minus 2 bytes” may refer to ensuring no overlap between static and high memory, with the last two bytes being allocated for high memory starting at $0FFFF. This would mean that the total applies to just dynamic memory and, if that’s true, that makes sense under my assumption because it ensures static memory is kept distinct from the executable (high memory).

Yet the Z-Machine specification also says this:

The bottom of high memory may overlap with the top of static memory (but not with dynamic memory).

Doesn’t this contradict the “no overlap” thing I just said? Well, this statement means that high memory can start at an address that is shared with static memory — essentially, static and high memory can coexist in the same space, but only when static memory has fully filled its space and no more writing will happen to it, since static memory is read-only.

If I’m reading all this correctly, what these constraints ensure is that:

  • Dynamic memory (the mutable part) can never overlap with high memory, because that would cause the interpreter to accidentally overwrite code with game data.
  • Static memory (the read-only part) can coexist with high memory, as the static memory is not modified during runtime, so there is no risk of corruption.

The “64K minus 2 bytes” constraint is likely still there as a technical limitation to avoid memory conflicts. The compiler must ensure that static and dynamic memory fit neatly under the high memory mark, respecting the 64K limit. The fact that static memory can overlap with high memory doesn’t negate this; it simply allows the bytecode (high memory) to be packed more tightly.

In looking at many Infocom games, it’s interesting to note that dynamic memory rarely exceeds 32K. I don’t think I’ve found an instance of any game going over 48K.

High Memory and 64K

All this finally takes me back to the other point I needed to clarify for myself, which was what I said earlier: the statement from the specification seemed to imply that the high memory is above the 64 kilobyte memory that could be addressed by 16-bit pointers.

I think the key here is understanding how high memory fits into the 64K limit in the Z-Machine.

The phrase from the Z-Machine specification — “the total of dynamic plus static memory must not exceed 64K (64K minus 2 bytes)” — is referring to dynamic memory (mutable game state) and static memory (read-only data like text strings). The total size of these two regions combined must stay within the 64K addressable space because they’re both part of the same 16-bit memory space.

High memory is not “above” the 64K memory limit; it’s simply another part of the same 64K addressable space, just like dynamic and static memory. So here’s the basic layout as I understand it.

  • Dynamic memory: Starts at address 0 and continues up to a certain point.
  • Static memory: Follows immediately after dynamic memory, continuing up to the start of high memory.
  • High memory: Begins after static memory and extends to the end of the 64K addressable memory space.

In other words, high memory is still within the 64K address space, and it holds the machine code (bytecode) of the game itself. All of this — dynamic memory, static memory, and high memory — must fit within the 64K memory limit imposed by the 16-bit addressing scheme. The “64K minus 2 bytes,” at least as I now understand it, refers to a small overhead in memory allocation, but it doesn’t mean high memory is outside this limit.

The specification’s focus on dynamic and static memory highlights the fact that these are the areas where game designers had to be careful about memory usage, as both regions are essential for game logic and data. High memory, which contains the game’s code, doesn’t need to be mutable or heavily managed by the game itself. It’s essentially “read-only” machine code.

This aligns with this statement from the Z-Machine specification:

The original idea behind the high memory mark was that everything below it should be stored in the interpreter’s RAM, while what was above could reasonably be kept in “virtual memory”, i.e., loaded off disc as needed.

The high memory mark is essentially a boundary within the 64K addressable space that separates the static/dynamic memory (which needs to be kept in RAM) from the high memory, which contains the game code (bytecode) that could be loaded into RAM or kept in virtual memory (disk) as needed.

Everything below the high memory mark (i.e., dynamic and static memory) must be stored in the interpreter’s RAM because this is the part of memory that the Z-Machine accesses frequently during gameplay. This includes mutable data (dynamic memory) and read-only data (static memory), both of which must be immediately accessible in RAM.

Everything above the high memory mark contains the game’s machine code (bytecode). The interpreter can manage this more flexibly. For example, the interpreter might choose to load parts of high memory from disk as needed, rather than keeping all of it in RAM at once. This would allow interpreters running on memory-constrained systems to treat high memory more like virtual memory—loading bytecode instructions from disk (or another source) into RAM as the game progresses.

Filling out what I talked about regarding the basics of memory, the idea behind “virtual memory” here is that the Z-Machine doesn’t need constant access to all of the high memory at once. Since high memory contains the game’s executable code (which is often larger than the game’s data), interpreters can manage this code more efficiently by loading only the necessary parts as required, saving RAM space.

This design allows the Z-Machine to fit larger games within the 64K memory limit while also keeping the runtime memory footprint as small as possible. By offloading high memory to disk or ROM, the interpreter can run complex games on systems with very limited RAM, which was crucial for computers of the time.

This Got Me Where?

Well, I know the basic conceptual scheme of the memory map, which is that dynamic and static memory must reside in RAM, while high memory can be swapped in and out from disk, giving interpreters the flexibility to work within hardware constraints.

However, Grue will not be operating such memory constrained environments where these constraints would actually matter. But those constraints did matter to the compilers that produced the zcode programs that Grue will read. How much of of this will actually matter remains to be seen but in the next post I’ll look at having Grue check some of these constraints as part its operation.

Share

This article was written by Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.