This post continues on directly from the second post in this series. Assuming you’ve been following along, you’ve broken your input into a stream of tokens. Now you need some way to recognize higher-level patterns. This is where Racc comes in: Racc lets you describe what you want to do with those tokens. That’s what I’ll be covering here: how the parser works with the lexer.
A Tester Learns Rex and Racc, Part 2
In my previous post on this subject I started off on the learning process for building a lexer with the Rex (Rexical) tool. Here I want to update the logic I provided in that post to show how to make it more testable. I then want to expand on the example of using Rex with something a bit more substantive.
A Tester Learns Rex and Racc, Part 1
As a tester, I like to know how things work. So when I started investigating the basis for the tool Cucumber, I came across the concept of Gherkin. That was written with the help of a system call Ragel. What I was curious about was whether I could build one of these languages on my own, perhaps for a customized testing tool. This caused me to stumble upon Rex and Racc, which are two Ruby-based tools that help you build your own languages. The documentation on these tools, however, is on the bad side of awful. This post is purely to document and share what I learned.
Seeing and Thinking Differently
In talking about test teams as inventors, I mentioned that Albert Szent-Gyorgyi said “Discovery consists of looking at the same thing as everyone else does and thinking something different.” I wanted to go back to that thought because it’s not the act of “thinking something different” but rather the act of “thinking differently” that really matters to me. This is even more so the case in an industry where testing and development continue to move closer together and, in fact, often merge.
Effective Tests, Not Positive and Negative Tests
My opinion is that the “positive” and “negative” distinctions for tests are a faulty conceptual distinction. To me, only “old school” testers talk about “positive testing” and “negative testing.” Pretty bluntly stated, huh? I feel strongly about this because the way the terms are promoted, they take focus away from how a tester should be thinking about testing. That’s my belief anyway. I’ll try to defend that.
Continue reading Effective Tests, Not Positive and Negative Tests
Testers Write Tools — Like Test Libraries!
I keep looking for ways to get junior testers up to speed on writing test frameworks and test libraries. It’s a very good skill to have and it’s one that’s in demand. Beyond that, I think it paves the way for helping testing to evolve into what it must become: a group of practitioners that are test solution developers. As such, I’ve been using my Symbiont library for this purpose.
There Is No “Test Phase”
When people talk about the “testing phase” of a project, you already have a problem because that implies testing is an activity that is relegated to a certain phase as opposed to a structural element of the entire project itself.
Specifying Application Workflow Activities
Lately I’ve been writing a lot of specifications and by that I mean test specifications. And by that I mean the specifications you tend to write in tools like Cucumber, SpecFlow, Lettuce and so on. What’s been interesting is deciding at what level of intent to write at.
Symbiont Generators and Web Objects
Symbiont is a little further along, enough so that it makes sense to look at the basic idea of what it means to provide a façade over two tools. The focus here is really just to show you some tool construction from the ground up. This may help you understand tools like page-object or watir-page-helper a little bit or they may spur on your own creative efforts as you build your own tools.
Evolving the Symbiont
In my previous post about the start of my Symbiont project, I gave a few examples of what I can do now with libraries like Selenium and Watir. It probably wasn’t entirely clear what direction Symbiont was going to go, given what I described there. Here I’ll to provide a bit more of a roadmap for what Symbiont will need to accomplish.
Rise of the Symbiont
Lately I have been developing lots of test solutions, most of them having to do with automated testing. I originally developed a library that I called Lucid. This was in turn based on a library I was originally calling Terminus. Now I’m trying to distill a lot of what have learned into a library that I’m calling Symbiont.
I’ll be perfectly honest: what I’m writing here is largely for my own benefit as I get my thoughts in order, but I do have hopes for Symbiont to be something useful. So I will document its development path here and if someone besides me finds anything of interest in it, that’s great.
Tests as Specifications
In Testing’s Brave New World, I ended up talking about BDD and the concepts that testing, acting as a design activity, has to work within. That was a post fairly heavy on nomenclature concerns. I sort of skirted around the issue that it’s the tools we use in our BDD type activities that are often forcing us to consider this terminology. Even if you are not using a specific tool, but rather a technique like spec workshops — which is a “BDD activity” — you are still wrestling with these terms.
So let’s put this in some context.
Test Writers Step in Early
In my last post about what tests are for I talked a bit about the ability of test writers to be nimble in terms of getting quality-focused information — usually as a result collaborative discussions — encoded into readable tests that focus on intent rather than implementation. All of that’s easy to say but test writers sometimes have a hard time figuring out how to exactly do it in the context of the collaboration possibilities in their environment. Here I want to talk a little about the core ideas, backing them up with an example.
What Are Your Tests For?
A key question to ask is what you want to get out of your tests. I’ve already talked about how I think the purpose of testing is to find information and then communicate that information. That’s great and all … but … what do you actually want to get out of the tests themselves? The way you answer that question will shape your testing function to a large extent, particularly when it comes to picking supporting tools, such as for test management or test automation.
Test Teams Need Inventors: Balancing Creativity
In a previous post I talked about inventors as people who see and think differently. I also brought the idea of inventors having to institute cultural change in some cases. What I didn’t do is how an inventor utilizes their skill-set and mind-set to get that change going. So let’s talk about that. Hopefully I won’t make too much of a fool of myself.
Continue reading Test Teams Need Inventors: Balancing Creativity
Test Teams Need Inventors: Seeing and Thinking Differently
In various posts I’ve tried to show how I believe testers can invent solutions to problems they are encountering. These solutions do not always have to be tool-based in nature. Sometimes you are presenting a new way of thinking about processes, sometimes you are reframing problems, sometimes you a providing a hopeful vision, sometimes you are in fact coming up with ad-hoc tools, and sometimes you are coming up with new techniques. At the core of this, however, is thinking about problems and thinking about solutions. It’s about being an inventor.
Continue reading Test Teams Need Inventors: Seeing and Thinking Differently
Cars, Shapes, and Tests
I think the future of testing relies less on a sole focus on test execution and more on the ability to write tests in the language of the business domain, effectively tying together the role of business analyst and tester. I think that future also includes the ability of testers to instrument certain artifacts, like requirements documents, so that the requirements, acting as tests, can be converted to different formats, most notably automated tests. There’s a lot of contention out there around these ideas from “purist” testers: those who feel that these attempts provide a single-source test format is getting away from testing and more into development.
While I agree in part that this is putting some focus on development activities, I disagree that this gets away from testing. Effective testing, to me, is about putting testing where it will do the most good in the most responsible fashion. Sometimes that testing is finding new bugs, sometimes it’s providing confidence that bugs haven’t come back. Either way, testing is all about communication.
Epistemological Angst (or When Does “Before” Lose Meaning?)
I find myself in a philosophical mood today and it’s based on some experiences with testers that simply don’t ask questions. As testers we have to ask questions. Lots of questions. We also have to recognize when we are getting answers. Sometimes, however, we have to realize that sometimes answers aren’t that easy to come by. And here I’m not just talking about the idea that no one has the answers. What about when answers just aren’t possible at all? Can that happen? Of course it can. It depends on the types of questions being asked and the context in which answers are expected. (See my time travel post for one example of this.)
So — just bear with me here — I want every tester out there to consider a question: what existed before the big bang?
Continue reading Epistemological Angst (or When Does “Before” Lose Meaning?)
Testing’s Brave New World
As a tester you will often encounter environments that desire to practice some form of Behavior-Driven Development (BDD) and testing practices must fit within that. The term “BDD” may be used more or less vaguely. One thing that’s often somewhat common is that people will start to promote so-called “BDD tools.” Examples here would be tools like Cucumber, Spinach, Turnip, SpecFlow, Behat, Lettuce, and so forth. These tools all ask you to adhere to some sort of organization based on terms like “feature”, “story”, “scenario” and so on. What this often ends up doing is totally missing the point.
But what is the point?
Finding Those Hidden Bugs
One area of testing that often gets neglected are the various ways that bugs can be ferreted out by considering how they “hide.” Testing an application is sometimes like walking a mine field, where you have to trigger the mines in order to see where they are hidden. Another analogy might be a virus hunter, who has to find signs of an outbreak by testing the population, some of whom are hiding to cover up the fact that they have a virus. Here I want to present some information about why testers (and developers!) should consider a technique known as hidden-fault analysis and, more importantly, show how the notion of hidden faults can skew test results.