Thinking and Testing with Executable Specs – Part 2

This post is the second in my series regarding approaching testing when you are using an executable specification type tool. Assuming you want to follow along, the first post is fairly critical to understanding how the project is currently set up.

Let’s get right back to business. With that last step I created in my prior post, I finally had a step I could actually use at this point. Meaning, I had a granular enough step that didn’t require any further assumptions and so I could actually do something with it. Since I’m using Capybara and since I defined the app_host (in borg.rb in the engine directory), I can just add one line to the step:

Here I’m using Capybara’s visit method to call up the browser and go to the specific URL. If you run cucumber (and assuming you have my Sinatra application running, as explained in the first post) you should see the browser start up and the login page appear due to the navigation via the visit method.

Capybara has the annoying habit of closing the browser immediately after execution. Believe it or not, unlike every other library out there, there’s not even a built-in configuration option to keep the browser open should you want to do so. So you either have to work out a way to do this on your own — and there a few options — or just put some sleep statements as the last line in your steps. This will keep the browser open for the duration of the sleep.

What’s really nice is that I can now reuse that step I just created. I’m going to take the second iteration of my test step and reuse this step within that step. Specifically, here’s what I end up with:

Notice what I did there. I used the “I am on the login page” step from within the “I am logged in as an administrator” step. I could have just put a visit method call within the second step but this way I can avoid duplicating logic all over the place.

So now I’ll take the @wip tag off my second scenario and put one on my first scenario. I say that because, if I did this correctly, the second scenario now reuses the step of the first scenario. To be clear, my create_user.feature looks like this now:

Scenario: Go to login page
  Given I am on the login page

Scenario: Login as a role
  Given I am logged in as an administrator

@wip @starting
Scenario: Create a user
  When I create a "Clinical Administrator" user

See what I’m doing there? I’m going to move down the chain until I get the step I actually want — the @starting one — working. Running cucumber again should now do the same thing: take me to the login page of my application.

Now I need my test to actually log in otherwise I’m going nowhere fast. This means I need the text to fill in the information on the login form. I need a step for that, however, since none of my current steps are really worded to allow that. Granted, my “Login as a role” scenario kind of does that, but I really need to be able to login as specific user. Rather than change my existing scenario and steps, I’m going to create a new one.

So now every scenario in my create_user.feature file gets a @wip tag. I then create another scenario:

Scenario: Login as specific user
  Given I login as "administrator" with "admin"

For my own personal organizational purposes (which I talked a bit about in the first post), I put this new scenario between “Login as a role” and “Create a user”. I do this because it’s a more generic step.

Wait … more generic? Isn’t that a little odd. I’m saying this:

Given I login as "administrator" with "admin"

is more generic than this:

Given I am logged in as an administrator

Does that strike you as weird? After all the first one is much more specific, is it not? It lists an actual user name and password. The latter just indicates a role. The key thing here are the quotes. These make the steps more generic because you can fill in different information while using the same step. These are the little mental shifts you go through as you do these executable specifications. Mind, I’m not saying my approach is correct or even necessarily makes the most sense. I’m just giving you a snapshot in time of my thinking.

Now here’s the details filled in for that new step:

The problem is that this step, by itself, doesn’t work because it doesn’t go to the login page. This really isn’t the step I want to use anyway because it’s so specific. I don’t necessarily want someone having to litter the tests with a bunch of literal references to exactly who should log in. I don’t know that I want my spec writers to write at that level of detail. I’d rather them say the role because then if the exact login details change — say password “admin” is made “P@sswo3d” — I can just change some logic behind the scenes rather than all my tests. So that means my “I am logged in as an administrator” step is really what I want. That one already goes to the login page, so now I can combine my steps again. I’ll add one more line to my second step:

So with that in place, my “Login as a role” scenario is the one I execute. I remove the @wip from it while everything else keeps its @wip. Here’s what my test spec looks like now:

Scenario: Go to login page
  Given I am on the login page

Scenario: Login as a role
  Given I am logged in as an administrator

Scenario: Login as specific user
  Given I login as "administrator" with "admin"

@wip @starting
Scenario: Create a user
  When I create a "Clinical Administrator" user

When I run this with cucumber, I should see that my site is navigated to and I get logged in.

I should see that, but I don’t! One problem is I now realize these steps won’t work because the elements of the login page are in an iframe. (Yeah, some sites actually still do that, sadly enough!) I could just use the within_frame method of Capybara in my steps. But this is a chance for me to use a helper method.

So create a file called helpers.rb and put it in the specs/engine directory. Here’s what goes in that file:

Here I’ve just defined a simple method to check if a given element is in a frame or not and, if it is, I’ll call Capybara’s built-in method to handle this. I also include my module in the World, which is an approach Cucumber uses to make sure logic you define is accessible to all step logic. This now lets change my previous step as such:

Now it should all work. And it does! If you’ve been following along and everything is set up properly, you should see the browser navigate to http://localhost:4567/login, type in some information into the login form, and then submit the information taking you to a landing page. Again, feel free to use sleep statements as a poor-man’s way of getting Capybara to not close the browser before you can see what happened.

Okay, so let’s take stock. Where are we here?

  • I know I can go to a specific page. (Tested with my “Go to login page” scenario.)
  • I know I can login as a specific user. (Tested with my “Login as specific user” scenario.)
  • I know I can login as a generic role, reusing the above two scenarios. (Tested with my “Login as a role” scenario.)

So now I want to go back to my @starting step. I’ll add the @wip tags to every scenario except that one. This step — the one I really want to work — still does nothing. At this point I know I need to reuse a step that takes me to the login page and logs in as an administrator. Why? Because only administrators can create users.

Try it. There’s not much logic to this little application but one bit is that if you login with a name other than administrator, you will not see a users link on the landing page. Only someone logged in as administrator will see that link.

So now let’s reuse my generic role scenario in my step:

Running cucumber right now should do the same thing you saw before: go to the login page and login. This works because my steps are now chained. Here’s how it all reads conceptually (with supporting comments in parentheses):

When I create a "Clinical Administrator" role
  (which requires that)
  I am logged in as an administrator
    (so to do that)
    I am on the login page
    (and while there)
    I login as "administrator" with "admin"

This is pretty good progress. I’ve managed to chain together a few steps, learned how to incorporate a helper method, and started to realize the process that I have to follow if I want tests that specify intent rather than implementation. That last point is important so let me focus on that for a second before stopping. Here are the basic steps I have so far:

  • Given I am on the login page
  • Given I am logged in as an administrator
  • Given I login as “administrator” with “admin”
  • When I create a “Clinical Administrator” user

Only the last of those is really saying what somebody wants to do. The rest are just context. The fact that I go to the login page and login with a specific administrator’s credentials are less important here. It’s a given — hence the clause used — that I need to login as an administrator to create a user. My ability to login as such is not the test here. What I’m really focusing on is a test that creates a specific kind of user. The other elements were what I created on my path to doing this. In all cases, however, notice I did not do something like this:

Given I am on the login page
When I fill in loginname with "administrator"
and I fill in password with "admin"
and I click the Log In button

That is considered a very imperative scenario. What I opted for instead were declarative scenarios. These tend to be shorter, as you can see, but they also hide much of the implementation detail behind the scenes. Granted, everything I’ve shown you so far is so small in structure that the differences might not seem to amount to much. And, in fact, you’ll probably notice that in reality I got to my declarative scenario by building up and reusing imperative scenarios.

In the next post, we’ll start interacting with that users page.


This article was written by Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.

2 thoughts on “Thinking and Testing with Executable Specs – Part 2”

  1. Good stuff. Shows how you can take executable specs and make them work to your advantage without worrying about behind the scenes methods after you have created them.

    Question: You want to create a user, which means that you have to be logged in to the application as an admin. So that means you have to first login. Is there an advantage to having the “navigate to login page” as a separate scenario rather than integrated behind the scenes for the login as role “x” scenario? I understand that navigating to pages will be essential for future tests, but does it merit it’s own scenario (I am really not sure at this point, just wondering what your thoughts on it are)?

    1. In fact, you have anticipated what’s coming in Part 3! I agree with your sentiment. The “go to login page” type stuff would not be a standalone scenario for me. I think part of the evolution of a tester who uses a tool like Cucumber is learning what does and what does not merit a scenario versus a set of steps that you know you can call on.

      What I’ve found helpful is to make those “mistakes” and come to that conclusion as you learn how to craft more and more declarative scenarios.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.