An Introduction to Using Cucumber-JVM

A lot of testers I run into learn Cucumber on Ruby. This tends to be a very easy path since the overhead on a dynamic language like Ruby is generally pretty low. However, some testers need or want to run Cucumber on the Java Virtual Machine. Yes, you can use JRuby to run the Ruby version of Cucumber on the JVM. But you can also use Cucumber-JVM, which is a port of the Ruby version into Java. However, I find many testers have a hard time getting started with it, particularly if they are coming from Ruby. What follows is the tutorial I wanted and either didn’t find or didn’t look hard enough to find.

To follow along, make sure you have a Java development environment set up on your machine. This means the JDK as well as the JRE. I should note that I have done all of the steps below on Java 1.5 up to 1.8.

You are also going to need Maven for what I show you. Note that you can use Ant if such is your preference but many people seem to be using Maven. The setup for Maven is generally simple: unzip the distribution, create an M2_HOME environment variable, and add M2_HOME/bin to your path. To test that Maven is working:

$ mvn -v

If you do want to use Ant, setting it up is quite similar. Create an environment variable called ANT_HOME, add ANT_HOME/bin to your path. To test it:

$ ant -v

There are ways to do everything I do in this post via Eclipse, NetBeans, IntelliJ or whatever editor you prefer for Java. For this post, I’m going to focus on the command line mainly because I believe any test framework should be understandable and easily manipulable via the command line before you bring a graphical user interface into the mix. However, feel free to use whatever method you like.

Creating Your Project, Maven Style

To get started with the Maven approach, first create a project using the following command:

mvn archetype:generate -DgroupId=nostalgic.textadv.voxam -DartifactId=Voxam -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

I don’t want this to turn into a Maven tutorial but basically what this is doing is creating a simple project to get started. The project directory will be called Voxam (which matches the Artifact ID above) and the Java package structure will use what you specify in the Group ID (nostalgic.textadv.voxam, in my case). When the above command runs, you should have this structure:

Voxam
|   pom.xml
|
----src
    ----main
    |   ----java
    |       ----nostalgic
    |           ----textadv
    |               ----voxam
    |                       App.java
    |
    ----test
        ----java
            ----nostalgic
                ----textadv
                    ----voxam
                            AppTest.java

After creating a new Maven project, you’ll generally need to either add some dependencies to your project’s pom.xml file or update the existing ones. We’re going to do both. First change your existing pom.xml file so that it uses JUnit 4 instead of JUnit 3:

Now, still in the pom, add the dependencies you need to get Cucumber-JVM as part of your project:

It doesn’t matter what order you put these dependencies in. I put them above the junit one but it’s up to you.

Incidentally, this is where I start to lose some Ruby purists. Some testers coming from the Ruby world feel that all of the above is much more complicated. On the one hand, I sympathize but on the other hand I try to caution testers about this. Yes, Java is famous for setting up lots of directories that correspond to packages, which is often why you need a high-end IDE to even work with Java. And, yes, XML is a bloated beast and is a bit old-school, compared to upstarts like JSON or YAML. However, once you get past that, the pom.xml is conceptually no different at all than using a Gemfile and Bundler in Ruby. The nice thing is that once you have your pom file structure set, you rarely have to worry about it.

So my advice to Rubyists: don’t judge everything you do with Java against the standard of how you do it in Ruby. If it helps, there’s often just as much culture shock for Java developers going to Ruby.

One thing I should note is that with the specific version numbers referenced in the pom.xml file above, I’m using the most stable versions that exist at the time of writing this post. These may be different by the time you read this. In any event, to make sure everything works, let’s run the following command:

mvn test

If this is your first time using Maven or your first time using these particular dependencies, you will see a long string of files that are being downloaded into the global maven repository. You ultimately should see something like this as output:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running example.atm.AppTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] --------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] --------------------------------------------------------------
[INFO] Total time: 3.407 s

The actual test that’s being run there is that of the AppTest.java file, which references the App.java file. These pre-generated files are basically in place to make the initial Maven structure work but they also give you a clue as to how your eventual Cucumber project is going to need to be set up.

At this point, get rid of the App.java and AppTest.java files. You certainly aren’t going to use them since they are nothing more than stub files to get you a working Maven Java project. Now let’s make sure that we’ll start out with a clean build. Run the following:

mvn clean

The above path structure, generated by Maven, will provide a location for what Cucumber-JVM calls step definitions as well as for any application logic or test helper code. We’ll get to that in a bit. But first, in order to have something to test, I need an application.

Application Logic

You can use Cucumber-JVM to directly test against the code of a working application or you can use it to test a remote application, without touching the actual application code at all. Here I’ll focus on the former; in another post, I’ll tackle the latter.

So to do this, let’s put a simple application in place and, further, let’s put some unit tests in place for it as well. Then, when we’ve got that, we can put some feature files in front of that application and put Cucumber-JVM through its paces. For this next bit, I’m going to use a simple text parser application I wrote for the purposes of this post. Rather than have you type a whole bunch of stuff in, grab the following two files and put them in src/main/java/nostalgic/textadv/voxam.

You can run the following to make sure those compile:

mvn compile

If you want to make sure the application actually works, you can do this from Maven:

mvn exec:java -Dexec.mainClass="nostalgic.textadv.voxam.Parser"

You can also just run it directly from within the Voxam directory:

java -cp target/classes nostalgic.textadv.voxam.Parser

Somewhere in the output, you’ll see this:

Parse Command: cut tree with chainsaw
Verb: cut
Direct Object: tree
Indirect Object: chainsaw

So with that we know we have a working application. Now let’s put in some unit tests for the app. Again, rather than have you type it out, put the following file in src/test/java/nostalgic/textadv/voxam.

Now let’s see if the tests run against the application:

mvn test

You should see that 11 tests are run and you should get no errors.

Now let’s start using Cucumber-JVM to test the application at an an acceptance level.

Feature Files

You have to create a directory structure where Cucumber will expect your features files to be. At minimum, you can create the following:

mkdir src/test/resources

Now here’s an interesting thing. You could just put your feature files right in that “resources” directory. However, you can also create a structure within resources that matches your application code package structure. But it doesn’t seem that you have to. This is an area of Java that I’m still learning, to be honest, so here’s what I did. I created the following directory:

mkdir src/test/resources/nostalgic/textadv/voxam

Just to be clear, that gives me the following directory structure:

----src
    ----main
    |   ----java
    |       ----nostalgic
    |           ----textadv
    |               ----voxam
    ----test
        ----java
        |   ----nostalgic
        |       ----textadv
        |           ----voxam
        ----resources
            ----nostalgic
                ----textadv
                    ----voxam

Now create a file called parser.feature in src/test/resources/nostalgic/textadv/voxam.

Let’s write the start of a simple scenario.

Ability: Parsing User Input

  Background:
    Given the game parser
    
  Scenario: Parse a valid single word command
    When the command "inventory" is parsed
    Then the verb is "inventory"
    And  there is no direct object
    And  the parser response is "Success"

Now we have to execute the feature file, which essentially means parsing it and determining which, if any, of the phrases are matched by step definitions. The feature file is just a text file with English statements, as opposed to source code. Unlike Ruby Cucumber, which serves as its own runner for executing feature files, Cucumber-JVM requires you to create a runner.

Test Runner

With Cucumber-JVM, you run the feature files using a JUnit Runner, which you will already have available to you from the Cucumber-Junit Library that you referenced in your pom file. (You can use other test runners, like TestNG, if you want.)

What this means is that you need to create a test class, and annotate it with JUnit’s @RunWith annotation. So first create a file called TestRunner.java in src/test/java/nostalgic/textadv/voxam. Note I’m keeping everything I do in the same package here. Does that matter? I’m going on the assumption that it does for the moment but I’ll revisit this later. Do note that the actual file name here does not matter; call it whatever you like. Here’s what should go in it:

Now let’s execute what we have and see if anything happens. Run this command:

mvn test -q

This is just running the Maven test goal with the quiet option just to keep the output down to a dull roar. What should happen is that your feature file is read and Cucumber-JVM reports by saying that you have undefined steps and some step definitions are provided for you to use. Also important to note is that your unit tests are still running as part of this. Specifically, you’ll see output like this:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running nostalgic.textadv.voxam.ParserTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec
Running nostalgic.textadv.voxam.TestRunner
UUUUU

1 Scenarios (1 undefined)
5 Steps (5 undefined)
0m0.000s

The ParserTest are the unit tests that I had you put in place. The TestRunner is, of course, the runner we just created to parse the feature files.

Now let’s get into making the feature file executable.

Step Definitions

Okay, so to make this work, create a step definition file. Create ParserSteps.java in the src/test/java/nostalgic/textadv/voxam directory. The name of the file does not matter. The file should look like this:

Incidentally, for those of you who like to compare, you might note that with Java you cannot just execute the phrase methods (Given, When, Then) directly. Instead you have to decorate a method with an annotation that matches the phrase from the feature file. Your method that actually executes the logic can be called anything you want.

All that being said, you can use other JVM-based languages (like Groovy, Scala or Clojure), all of which do a lot to remove the boilerplate, allowing your code to be more expressive. I usually tell testers that what they are getting out of Cucumber-JVM is not the ability to use Java with Cucumber, but the ability to run on the JVM with any JVM-based language. I’ll show examples of other JVM languages, likely in another post.

Okay, so, if you now run the Maven test goal again, you’ll see that you have a pending exception appear, which is what you should expect. Let’s replace the PendingException for the Given step with some code:

If you run mvn test -q again, you should see that the Given step now passes and now the pending exception has moved on to the When step. That should give you confidence that Cucumber-JVM is working very much like the Cucumber Ruby version (if you are coming from Ruby) or just working at all if you are coming to this fresh. Let’s now get the entire set of steps functioning. Change your remaining step definitions to look like this:

Do note that in the source above I changed some of the parameter names for the methods to be a little more readable than just “arg1”. Also note that in order to run the assertions you will need one more import at the top of the file:

If you run your tests now, the entire scenario should pass.

Some Formatting, Please

The output you are getting probably looks a lot like this:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running nostalgic.textadv.voxam.ParserTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec
Running nostalgic.textadv.voxam.TestRunner
.....

1 Scenarios (1 passed)
5 Steps (5 passed)
0m0.152s

Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.734 sec

Results :

Tests run: 17, Failures: 0, Errors: 0, Skipped: 0

You may want to have a little better reporting generated, such as being able to see the details of the scenario that has executed. You can do this by changing your TestRunner.java and using Cucumber options. To do this, you have to import those options and then add an annotated method:

Here I’ve added three formatters to be generated. The “pretty” refers to Cucumber’s formatter that provides nice looking output that shows scenario details in the console. The HTML formatter generates a series of files in the test-report directory. (So note that you don’t specify an HTML file for the formatter; you specify the directory for where the HTML results will be stored.) The JSON formatter outputs a report in, not surprisingly, JSON-compliant format. (Note that here you do specify a filename.)

Different Feature Directory

Let’s say you want to store your feature files in something other than src/test/resources. You can do that but then you have to let Cucumber-JVM know where the files are. If you want to try it, change your resources directory to specs.

Important! If you do this and run your tests right now, everything will still pass. Why? Because Maven is running the tests against what was compiled and built in the target directory. So if you want to see what happens, after changing src/test/resources to src/test/specs, run mvn clean and then run mvn test -q.

Your test runner will now still run but the output is:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running nostalgic.textadv.voxam.ParserTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec
Running nostalgic.textadv.voxam.TestRunner
No features found at [classpath:nostalgic/textadv/voxam]

0 Scenarios
0 Steps
0m0.000s

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.516 sec

Results :

Tests run: 11, Failures: 0, Errors: 0, Skipped: 0

The unit tests run but the TestRunner could find no specs. What you can do, however, is tell the TestRunner where the specs are. Change TestRunner.java so that it looks like this:

Now you should find that your feature files are being executed again when you run the Maven test command.

Note as well that the feature files do not have to live in the test directory. For example, I moved my specs directory so that my structured looked like this:

Voxam
|   pom.xml
|
----src
    ----main
    |   ----java
    |       ----nostalgic
    |           ----textadv
    |               ----voxam
    |                       Command.java
    |                       Parser.java
    |
    ----specs
    |   ----nostalgic
    |       ----textadv
    |           ----voxam
    |                   parser.feature
    |
    ----test
        ----java
            ----nostalgic
                ----textadv
                    ----voxam
                            ParserSteps.java
                            ParserTest.java
                            TestRunner.java

I then just had to update my CucumberOptions in TestRunner.java accordingly.

A Few Last Points

Does parser.feature have to live in the package structure?

No. To prove that, just take parser.feature above and move it directly into the specs directory. You’ll find it works just fine.

Does ParserSteps have to live in the package?

Not necessarily. As long as some step definitions that match your feature are somewhere in the src/test/java directory, you should be fine.

Does TestRunner.java have to live in the package?

No. And, in fact, if you are testing application logic that live in different packages, most likely you are going to want the test runner to be sitting outside of the packages so that you can run tests for any application logic.

As an example, here is one structure, that works just fine for me:

Voxam
|   pom.xml
|
----src
    ----main
    |   ----java
    |       ----nostalgic
    |           ----textadv
    |               ----voxam
    |                       Command.java
    |                       Parser.java
    |
    ----specs
    |       parser.feature
    |
    ----test
        ----java
            |   TestRunner.java
            |
            ----nostalgic
            |   ----textadv
            |       ----voxam
            |               ParserTest.java
            |
            ----steps
                    ParserSteps.java

A caveat, however, is that all works for me now with my simple little test app. I don’t know if that flexbility of moving files around can be sustained when you have a more comprehensive application with different packages. I’m still learning that part.

Essentially the you have to keep in mind that you are using Maven and the JVM, both of which impose restrictions on where they will look for source code. (The same applies if you are using something like Ant or Gradle or Ivy instead of Maven.) So, regarding the step definitions, for example, you could put them wherever you want as long as the following conditions are met:

  1. Make sure the step definition code is compiled as part of the build/test process.
  2. Make sure the compiled step definition code is put on the classpath.
  3. Tell Cucumber-JVM the packages on the classpath it should search for step definitions.

The first two of those are not directly related to Cucumber-JVM but rather to the nature of running on the JVM and the build tool you use. If you want to put your step definitions somewhere else, you need to tell Maven to compile the classes there and then you have to put the resulting compiled classes on the classpath when you are running tests. The third item in the list, if you don’t tell Cucumber-JVM anything to the contrary, it will search the package of the test runner class (and its sub-packages) for step definitions.

So … Did We Learn Anything?

You’ve seen here that Cucumber-JVM is a pure Java implementation of Cucumber. You’ve also seen that Cucumber-JVM hooks directly into JUnit and uses it as a test runner. You’ve seen you have some flexibility in terms of where you put some of the supporting elements. If you are coming from a Ruby background, you should have some confidence that Cucumber-JVM works similar to the Ruby version of Cucumber, although I’ll be the first to admit it’s more cumbersome in Java and nowhere near as much fun.

I’ll likely post a few more forays into using Cucumber-JVM as I continue to work with it.

About Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.
This entry was posted in Automation, Cucumber-JVM, Java. Bookmark the permalink.

7 Responses to An Introduction to Using Cucumber-JVM

  1. Ven says:

    Great Post and a great introduction to Cucumber-JVM.  I have fair deal of Java development experience but very new to BDD and Cucumber.  This article was something that I was exactly looking for.  Looking forward to your future articles in this space.

     

    Thanks, Ven

  2. Pierre Gagne says:

    Great tutorial!  It helps understand how the different parts of Cucumber-JVM fit together.

    Thanks!

  3. Henrique Luz says:

    Hi, nice post. I’ve been read anothers and this was the best.

    It helped me a lot. Thanks!

    I used JBehave in another projects and I noticed the configuration was too expensive. I liked Cucumber setup because is too easy. But in another hands, the JBehave reports has more informations and that is more rich. You can correct me if I’m wrong, but Cucumber only generates report that has Scenarios that passed or failed in only html? Is possible to view a table with a short summary like how many scenarios passed/failed, some graphics or something like that?

     

    Best regards from Brazil.

     

     

  4. jitendra kumar says:

    Great work! Keep it up. found this very useful.

  5. Brownchild says:

    Fantastic work.   This indeed is the tutorial that all newcomers wished they found much sooner.   Your effort here is very much appreciated.

    Thank you.

  6. Rick Hough says:

    Wonderful intro article, thank you very much. I’ve done a bit of Ruby with Selenium and was not sure about implementing Cucumber. Your tutorial is excellent and has exactly the right amount of detail for me.

  7. h says:

    Very useful , can’t thank enough, the hours i spent just trying to find out why the step definition file was not getting picked up. Thank you for the hard work and detailed explanation along with examples , it benefited me a lot.. Wishes

Leave a Reply

Your email address will not be published. Required fields are marked *