In this post I’ll continue putting Lucid through its paces. It’s definitely presumed you have gone through part 1 and part 2, since I’ll build off of the example and directory structure that we worked on. Here I want to show a few more variations of Lucid execution.
If you went through the last post you may have ended up with your directory structure in a variety of ways. Here’s what mine currently looks like:
.
+---common
¦ triangle_calc.rb
¦
+---specs
¦ triangle.spec
¦
+---steps
triangle_steps.rb
Let’s flesh out the test specification just so you can make sure that a full test spec works as well as showing a few other ways you can use Lucid.
Add the following scenarios to your test specification:
Scenario: Isosceles
Given the length values 2, 2, 1
When the triangle is calculated
Then the triangle type is "isosceles"
Scenario: Equilateral
Given the length values 2, 2, 2
When the triangle is calculated
Then the triangle type is "equilateral"
Scenario: Right
Given the length values 3, 4, 5
When the triangle is calculated
Then the triangle type is "right"
Scenario: Impossible
Given the length values 3, 2, 1
When the triangle is calculated
Then the triangle type is "impossible"
Scenario: Invalid
Given the length values 2, 0, 2
When the triangle is calculated
Then the triangle type is "invalid"
That’s a lot of scenarios to add but they all certainly seem pretty close to each other. So the matchers we wrote in the previous post probably should have us covered. Yet, you may want to determine what steps are and are not recognized, or matched, without having Lucid actually execute the scenarios. This becomes much more important when you have lengthy scenarios or scenarios that call up browsers. To see what test steps will be matched, you can run the following command:
$ lucid --dry-run
If any steps were found not to have matchers, Lucid would inform you of that. If matchers are found, Lucid will indicate that by saying the step has been skipped. In the context of a dry run, skipped means “I found a matcher so I could run this step; but you said this was just a dry run, so I’ll skip it.”
If you run the lucid command without the dry-run option you should find that all of these scenarios pass.
Running Selected Scenarios
Since we now have a few scenarios, what if I just want to run one of them? You can do that in one of two ways. You can choose the line number to run or you can choose the name of the scenario. So, for example, you can run a test spec starting from an individual line. Both of these are equivalent:
$ lucid specs/triangle.spec --line 29 $ lucid specs/triangle.spec:29
In my test specification, this would run the “Equilateral” scenario. This may or may not be the same for you depending upon how you organized your test specifications or whether you have extra blank lines. Note that if you pick a line number that equates to a step in a scenario, the full scenario will be run. So if I had used “30”, “31”, or “32” — each of which would be a line of one of the steps in the “Equilateral” scenario — then the full scenario would be run.
You can run multiple line numbers, which would likely equate to multiple scenarios. For example, in my test specification, if I wanted to run just the “Scalene”, “Right” and “Invalid” scenarios, I could do either one of these commands:
$ lucid specs/triangle.spec --lines 19:34:44 $ lucid specs/triangle.spec:19:34:44
As you can imagine, having to know the line number can be a little annoying. Further, the line numbers are certainly going to change if you add or remove scenarios and steps from the test specification. So another approach is that you can execute scenarios by name. For example, if I want to run just the “Impossible” scenario, I could do this:
$ lucid --name Impossible
Note that you can also include the name in quotes and, in fact, you would have to if the scenario name had spaces. So for example if you had a scenario named “Scalene Triangles” then to run that scenario by name, you would have to do this:
$ lucid --name "Scalene Triangles"
You can run multiple individual scenarios by specifying each of their names:
$lucid --name "Scalene" --name "Right"
One thing you might note here is that when I ran the command specifying the line numbers, I called out the spec file whereas I did not do that when specifying the name. I could have called out the specific spec file to run with the name option as well. If you don’t specify the spec name when running Lucid with the name option, then Lucid will look for all spec files and find any that have that name and execute them. The same does not apply to line numbers. If you don’t specify a file, Lucid would have no way to know which line in which file you meant.
One other thing to note about using the name option is that if you had multiple scenarios that matched the name you provided, Lucid would run all of them.
You can also specify patterns. For example, with the test specification we’re working with, were I to do this:
$ lucid --name "ss"
Then only the scenario titled “Impossible” would be executed because it’s the only one that matches the pattern “ss”. Incidentally, because using the name option is based on pattern it does mean that it is case sensitive. So if you want to run the Scalene scenario, make sure your pattern is “Scalene” (with capital) and not “scalene”.
Running Tagged Scenarios
You’ve already seen that you can selectively run scenarios via providing a name or name pattern as well as by specific line numbers. However, eventually you will end up with many test specifications, spread out over numerous directories. You may want to run subsets of those specifications, or related groups of scenarios, regardless of what they are named or where they are located. That’s a perfect case of where you would use tagging.
As an example, say you want to flag the scenarios that lead to “bad triangles” so that you can run just those, separate from the other scenarios. Change your test specification to add the @bad tag above each scenario title:
@bad
Scenario: Impossible
...
@bad
Scenario: Invalid
...
Now tell Lucid that you want to run just the scenarios tagged with @bad as such:
$ lucid --tags @bad
Now let’s add the tag @named to the three specifically named triangles by adding the tag to those scenarios:
@named
Scenario: Scalene
...
@named
Scenario: Isosceles
...
@named
Scenario: Equilateral
...
Then let’s add two tags — @angle and @named — to the only remaining untagged scenario:
@angle @named
Scenario: Right
...
Now if you want Lucid to run just the scenarios tagged with @angle or @named, you can do this:
$ lucid --tags @named,@angle
Here the comma is interpreted as a logical OR, meaning that Lucid will run any scenarios that have the tag @named OR that have the tag @angle.
Now let’s say you want to run a scenario if it has two specific tags. Specifically: if any scenario has the tag @angle as well as @named you want to run it. You can do this:
$ lucid --tags @angle --tags @named
Here the use of the –tags option twice treats them as a logical AND. So only a scenario that has both @angle AND @named as a tag will be executed.
You can also exclude scenarios based on their tag. Let’s say you do not want to run any scenarios that return bad triangles. You could do this:
$ lucid --tags ~@bad
You could also combine tag operations. Perhaps you don’t want to run any bad triangle scenarios but but only want to run scenarios that provide an angle, as opposed to a named triangle. You could do this:
$ lucid --tags ~@bad --tags @angle
Finally, maybe you want Lucid to stop execution if there are more scenarios than expected of a certain type. This is a good use for tags as well. Consider this:
$ lucid --tags @bad:1
This will cause Lucid to fail if there are more scenarios than expected of a certain type. In this case, if there is more than one scenario tagged as @bad, Lucid will fail.
Do note that right now the output for this is particularly ugly. It’s an area I plan to change in a future implementation of Lucid.
Outlining your Scenarios
To close out this post, let’s step back from how we’re executing the scenarios to show a different way to format the scenario. If you have a set of related scenarios that are effectively performing the same test condition while changing the data condition each time, you might want to consider a scenario outline. To see this in action, either remove all of the scenarios you have created and replace them with the code below or simply create a new spec file:
Scenario Outline: Triangles
Given the length values <side1>, <side2>, <side3>
When the triangle is calculated
Then the triangle type is "<type>"
Examples:
| side1 | side2 | side3 | type |
| 1 | 1 | 1 | equilateral |
| 2 | 2 | 2 | equilateral |
| 4 | 5 | 6 | scalene |
| 10 | 8 | 3 | scalene |
| 3 | 3 | 4 | isosceles |
| 2 | 2 | 1 | isosceles |
| 0 | 4 | 5 | invalid |
| 2 | 0 | 2 | invalid |
| 1 | 4 | 6 | impossible |
| 3 | 2 | 1 | impossible |
| 3 | 4 | 5 | right |
If you run lucid with that scenario outline in place you will find each element of the table is executed and passed, just as they were when broken out as individual scenarios. In fact, you can show how each of the scenarios is being executed by telling Lucid to expand the generalized outline so that the test and data conditions are reported individually. To do that, just run this:
$ lucid --expand
Is this outline approach a better way or a worse way? Well, one thing is certain: when we had more scenarios we could more selectively run them. We could tag certain ones, we could execute them solely by name or by line number, and so on. We have lost a lot of that with going to a scenario outline. On the other hand, we’ve certainly made everything much more concise and, it could be argued, easier to add to later. In fact, as you can see, I’ve added more data variations above than we had in the original scenarios.
This post will conclude the use of the sample triangle application. In the next post, I want to create a web based version of the triangle application so that Lucid can be used in the context of a web application. That will allow me to show the project generation abilities of Lucid.
