This post continues on directly from the second post in this series. Assuming you’ve been following along, you’ve broken your input into a stream of tokens. Now you need some way to recognize higher-level patterns. This is where Racc comes in: Racc lets you describe what you want to do with those tokens. That’s what I’ll be covering here: how the parser works with the lexer.

In my previous post on this subject I started off on the learning process for building a lexer with the Rex (Rexical) tool. Here I want to update the logic I provided in that post to show how to make it more testable. I then want to expand on the example of using Rex with something a bit more substantive.

As a tester, I like to know how things work. So when I started investigating the basis for the tool Cucumber, I came across the concept of Gherkin. That was written with the help of a system call Ragel. What I was curious about was whether I could build one of these languages on my own, perhaps for a customized testing tool. This caused me to stumble upon Rex and Racc, which are two Ruby-based tools that help you build your own languages. The documentation on these tools, however, is on the bad side of awful. This post is purely to document and share what I learned.