Many people feel that tools like Cucumber are a waste of time. But are they? Let’s talk about that.

The common refrain is usually a variation on these statements:

1. Cucumber is just one more tool and I want to minimize my tools.
2. There’s no reason to provide some English statements. What matters is the code.

There is truth to both of these, if you take time to look at the motivations behind these reasons. However, if you look at the first objection, the context is usually due to a category error. People feel that Cucumber is just doing something that they already do with an existing tool.

Regarding the second objection, the idea focuses on the dislike of an abstraction layer. In this case, the English (or other natural language) feature files serve as an abstraction over the code that allows those statements to execute. So the idea here is that the natural language is not needed, and the code can simply be referenced instead.

Regarding the first objection, I think part of this may be why some members of the Cucumber community go out of their way to say that Cucumber is not a testing tool and is thus misunderstood. The motivation here may be to get people to stop seeing Cucumber as simply a slightly different form of a tool like RSpec. I’m being Ruby-centric but these same debates play out in languages like Python (between Lettuce and Nose), .NET (between SpecFlow and NUnit), Java (between JBehave or Cucumber JVM and JUnit), JavaScript (between Yadda and Mocha).

There is also some conflation around the concepts of the execution tools (like Cucumber, SpecFlow, etc) and the expression language (Gherkin). Gherkin simply provides a structuring element that helps guide people in terms of expression of ideas. Tools like Cucumber simply utilize that structure to make execution of those ideas a bit easier. But you can use the ideas behind Gherkin entirely separate from any particular tool.

The Value Add for Testing

I’ve been asked many times why I continue to support tools like Cucumber and structuring languages like Gherkin. So here are my thoughts on this.

• If I’m not going to automate, I have to write tests in some way.
• A Gherkin-style approach is not any worse than many others and has many benefits.


• If I am going to automate, I want the intent of tests to be expressed outside of code.
• Again, a Gherkin-style approach is not any worse than many solutions.

What tools like Cucumber do is allow you to have the manual tests directly automatable. Thus if the written test (the source of truth) changes, the automated test must as well.

Tools like Cucumber are a means to encode a shared notion of truth that has built in traceability and is always executable. This means the shared notion of truth is always demonstrable.

Note again here that Gherkin is only one small part. It’s simply the structuring element. Gherkin places no restrictions on how you actually describe your tests or the domain they are working in.

The Value Add for Collaborating

Now let’s take that up the ladder a bit and realize that tools like Cucumber are often correctly stated to be collaboration tools. (They are testing tools and collaboration tools; let’s not fall into the Tyranny of the Or.)

But collaboration with whom? Well, generally “the business” or some representative of the business. Really, however, it doesn’t matter whether you have such a person or not. You always have requirements. The only question is whether they are explicitly stated somewhere or are implicit bits of shared knowledge that float around in the mind space of your team. Some people argue against tools like Cucumber by saying “Well, the business doesn’t participate so why bother?” Or: “We don’t even have business analysts.” Okay, but none of that changes the fact that you do have requirements.

So what you start to get into here is the notion that you can write your requirements as your tests. Maybe not all requirements, but certainly the elaborated ones.

The idea of “requirements as tests” does sound potentially useful, right? You have statements of requirements. And, as most of us know, part of the way a requirement is “good” is for it to be testable. And for a requirement to be testable, it must largely be written as a test. So here you have a situation where the requirement is the manual test and is the automated test. Again, single source of truth with built-in traceability.

Again, sounds useful, right? Except, according to some, when you put it into practice. After all, there are generally more tests than requirements because you use tests to break down a requirement, don’t you? Yet — isn’t the sum total of what those tests prove a form of specification, and thus requirements? The answer is: yes, they are.

Once you look at tests as a specification, you see this. Once you open up tools like Cucumber as testing tools and requirements elicitation tools, then you realize you have a lot of freedom in how you express yourself. You also have many more opportunities to shoot yourself in the foot, of course. But a tool like Cucumber can only enforce so much discipline. Gherkin should be about it. It enforces a structuring element. But it doesn’t, and shouldn’t, enforce how you express your requirements and or your tests. That is where you get into a Business Description Language (BDL) and a Test Description Language (TDL).

But Is It Worth It?

So with that bit said, there is the question of whether all of this is worth it. Because that’s really what people are asking when they consider the use of tools like Cucumber. We’re basically asking: is the abstraction layer worth the time and effort? Even if business is not directly involved?

This goes back to what I said earlier: even if I’m automating, I don’t want all of my tests to be solely expressed in code. There needs to be a means by which tests can live as human phrases.

Does that mean you need tools like Cucumber? What about the RSpec type solutions, where the phrases can be embedded in the code?

Those are viable solutions, no doubt. Taking that into account, for me it goes back to this question: who are the consumers (both actual and potential) of my tests? Tests can serve a broad audience, not just in terms of their execution output but in the very nature of their use as a communication mechanism. Should people have to read code to get at your tests?

In my view, no. Tests should be capable of living as an independent artifact. Yet you can, as just one idea, create a “publish” mechanism that would take just the natural statements of RSpec out and display those in a friendly format. Then you get into the question of whether people write RSpec natural language statements the same way they write Cucumber natural language statements. That’s not a topic I’ll tackle here but the answer is generally no, at least in my experience. So if you value the communication aspect, in terms of written communication of a shared notion of quality, you do have to factor in how the tool you are using encourages or limits that expression, either explicitly or implicitly.

I think the main thing I try to get across to people is that there is benefit to the following:

• There is benefit to statements of requirement stated as tests.
• There is benefit to those statements having a structuring element around them that guides expression.
• There is benefit to having a direct and actionable connection between the artifacts of requirements, manual tests, and automated tests.

What I really try to promote is the idea of a single source of truth with built-in traceability that provides a shared notion of quality. If you believe in that idea, the above list provides the actual mechanism by which you can achieve that. Tools like Cucumber, SpecFlow and the others all are one mechanism by which to begin achieving that vision.