Writing Test Solutions with JavaScript: Testing Browser Apps

If you look at the JavaScript test tool ecosystem, you will find it’s generally quite large. But you will see the same tool names pop up fairly regularly. The decisions actually fall along some fairly simple and standard lines. In this post I want to show you at least some of your choices and then apply those choices to testing against a web application.

Okay, So let’s first cover some test tool distinctions and choices and then we’ll get into some coding and practice. The goal here will be to pick some tools without necessarily worrying if they are the best or the most popular. No matter what you choose, you’ll always wonder if you should have chosen something else. It’s the good ol’ paradox of choice raising its ugly head.

Choosing Your Tools

Test Runner

One of the first questions to answer is this: Which test runner am I going to use to execute my tests?

Actually, there is another question that you should probably think about and at least provisionally come to grips with prior to this, which is whether or not you want the abstraction layer of a BDD-style solution. This would be where tools like CucumberJS, Yadda, ChimpJS, and so on come into play. You don’t, however, necessarily need to answer this yet. For test solutions, keeping your abstraction layers to a minimum can be handy, particularly at the start.

As far as the test runner, your general candidates are going to be Jasmine and Mocha. There are plenty of others, but these are the ones you are going to hear about the most, at least initially. Developers will look at these tools as a means of testing JavaScript code, usually at the unit level. However, these tools do provide a way to wrap tests at any level. What you gain from these tools is a means of providing a test runner with test reporting capabilities.

Browser Driver

A second question to answer is this: What am I going to use to control the browser?

Your main choice here is pretty simple: Selenium. Yes, you’ll hear about lots and lots of tools like Protractor, Karma, Nightwatch and … lots of others. However, the fact is that just about any test framework you use that controls or drives a browser will be using Selenium at the bottom of its stack.

Most testers at the application level use Selenium or, at the very least, some wrapper around Selenium. A slight problem is that Selenium — depending on details of how it’s being called — can be the slowest of the browser drivers. The reason for this is because it is, in fact, driving an actual browser, with all the processing overhead that this entails.

If you want to run your tests a bit faster, without the overhead of the browser GUI, then you’ll want a headless browser driver. You have to be careful with these, however. The term “headless” here means that the driver doesn’t drive an actual browser but rather relies on the request/response traffic that a an actual browser would. Thus there is a level of emulation that may give false positives or false negatives. A big distinction among headless browser drivers is whether or not they execute JavaScript. Not executing JavaScript is a bad thing on most browser apps you’ll test because JavaScript is pretty much everywhere. The most common tool you’ll hear about, and likely want to use, is PhantomJS.

As far as a testing strategy when utilizing browser drivers, many experienced testers would suggest that ideally the tests you write should be applicable without change to a headless browser first. This way you can quickly validate functionality because these tests run much faster than those that are driven via actual browsers. Once that functionality is validated, then you can test against actual browsers automated via Selenium to check cross-browser compatibility.

As one more component to this, Selenium implements a Domain Specific Language called Selenese. This is basically meant to be a high-level API for representing Selenium commands. The idea being that if you group a number of Selenese commands together, you end up with a test script that can control a browser. However, at least for many testers that I know, the API is considered fairly cumbersome to use which is why you’ll often find yourself wanting to use a wrapper around Selenium’s API.

Drive the Automation

A third question to answer is: How am I going to talk to my browser driver (likely Selenium) from JavaScript?

As with all of these questions there are many answers but the most likely candidates you’ll hear about are WD.js, WebDriverJS, WebDriverIO, and Nightwatch.js.

As with Selenium being the most common browser driver, the most most common way to communicate with Selenium is via WebDriver. WebDriver is a library that essentially takes commands from clients (test scripts, in this case) and sends those commands to browsers via a protocol called the JSON Wire Protocol. The results of those commands being executed in the browser context are then returned for inspection and validation.

WebDriver makes direct calls to the browser using the native, built-in, support of the browser for automation. There are WebDriver binaries that talk to various browsers, such as ChromeDriver and Internet Explorer Driver. Selenium itself comes with some WebDrivers included: Firefox, Opera, Safari.

You also need language bindings, which allow you to talk with WebDriver from a given programming language. The aforementioned WebdriverJS, for example, is another name for selenium-webdriver, the official Node.js implementation of the JSONWire (WebDriver Wire) Protocol.

Gets a little confusing, right? At the very least, it’s a lot to take in. So rather than more discussion, let’s put some of this technology to use.

Creating a Project

Before we can start to drive the browser, we need Selenium’s implementation of WebDriver. We’re going to be running Selenium tests using JavaScript using Node.js. So make sure you have Node.js and NPM installed. (See my Setting up WebDev page if you need an assist with that.)

Let’s first create a project folder:

$ mkdir test-project
$ cd test-project
$ npm init

Answer all the questions that the npm init command throws your way. You can accept the defaults if you want.

Install the Tools

Now let’s grab and install the core selenium-webdriver module and save it to our project package.json file:

$ npm install --save selenium-webdriver

I’m using the –save option because this is a test solution. As a tester, you may deploy this test solution to various machines and this –save option makes sure that selenium-webdriver is considered a runtime dependency. Many build tools or continuous integration environments will run an npm install command for JavaScript solutions and doing so it will pull down any runtime dependencies into the environment that the scripts will run on.

We’re also going to need a testing framework to make the process of writing our tests simpler. I mentioned earlier both Jasmine and Mocha. There are a lot more choices beyond these and, quite honestly, this is what can make the JavaScript test ecosystem hard to get into. This is just like what happened with Ruby earlier on, when you had way too many test frameworks, libraries, and engines competing for attention and defocusing attention spans from the need (better testing) to the tool used to fulfill the need.

I happen to prefer Mocha but I’m going to show you Jasmine here. (I covered the use of Mocha already, if you are curious.) So let’s install Jasmine as a runtime dependency as well.

$ npm install --save jasmine

Some people using Jasmine like to run an initialization script and you’ll see this often in tutorials. If you want to do this, you’ll need to install Jasmine globally as well:

$ npm install -g jasmine

Now you can run the following to initialize Jasmine:

$ jasmine init

This will create a spec directory, which is the default location that Jasmine wants test logic placed in.

Next, we need to install a browser driver, which will act as a layer between Selenium WebDriver and the browser to run the tests in. It will capture our commands, send them to the browser and retrieve the results. The driver is browser-specific. There are drivers that talk to all the major browsers, including mobile browsers. Firefox support is usually built directly in to Selenium WebDriver. But I want to show you a case where you use a browser driver that is not built in. So I’ll use ChromeDriver to connect to Google’s Chrome browser.

There’s a couple of ways you can do this. You can download the latest ChromeDriver version and unpack the zip file. However, my preferred method is to use the NPM package of ChromeDriver. If you want to do that:

$ npm install --save chromedriver

This will install the ChromeDriver binary to:


/test-project/node_modules/chromedriver/lib/chromedriver/chromedriver

If you go the route of downloading the driver, then you have to make sure that the driver is on your environment PATH so that it can be found.

Create the Tests

Now that your environment is ready, we’ll create a test file with the test cases that you would like to run. In the example below, I am going to use Selenium and Node to navigate to the Dialogic Weight page using, enter a search value, and check that the calculation of that weight on Mercury is accurate.

Create a file in the spec directory called test-spec.js. Technically you can call these files whatever you want but you should always end your filenames with spec.js because jasmine-node, in particular, will only run these files by default. You can specify other files for Jasmine to run, but if you follow the naming convention you can have Jasmine automatically run any files that follow the convention.

Put the following in the file:

The test suite begins with a call to the Jasmine function describe. This is going to wrap all of the specs (tests). The exact same structure would occur with Mocha as well.

It’s now necessary to make sure that you start up a Selenium session. This session will, of course, be necessary for any tests to actually run against a web browser. You don’t want to do this for each and every test you write, of course. This is where setup comes in handy. Jasmine, like many such tools, provides a beforeEach function. As long as we’re thinking about this, we’re also going to want to make sure to reset the Selenium session for each test. This is where Jasmine’s afterEach function comes in handy.

These before and after functions correspond to setup and teardown activities. The beforeEach function is called once before each spec (test) is run in the describe block. The afterEach function is called once after each spec (test) is run. We’ll get to the actual tests in a bit, but, for now put the following in place:

Everything here is being run under the same context, which means objects (like this.driver) can be passed around. For our browser driver, we’re creating an instance of the Builder class and pass it a set of default settings for, in this case, the Firefox browser. You can see a similar example in the WebDriverJS User’s Guide. The build method is what essentially starts everything off and loads up the browser.

Promises and Callbacks

Now we get into a slightly complicated area of promises. Both Selenium and Jasmine have ways to handle asynchronous testing. But let’s briefly about what that means. Consider this line from above:

Selenium uses promises through a ControlFlow class. This gets into a lot of detail but for now just understand that most of the API methods return a promise that you can then take advantage of. You can read up more on the use of control flows if you want details. In this case, the get method returns a promise, which we can attach a then method to. The then method is part of the WebDriver Promise class.

In the above case, a done function is passed to the then function. Where does done come from? That’s from Jasmine. This function is always passed to the beforeEach, afterEach methods as an argument, whether you need it or not. (It’s also passed to any it methods, which I’ll get to in a bit.) To use the done function, you can include the done argument to the method and then call it after all of the processing is complete.

Kind of involved, right? Here’s what you need to know: if the done function is supplied, the test runner will wait until the calling function — get in our example — is executed before moving on. More examples are available in Jasmine’s documentation on asynchronous support.

Just to make sure it’s clear, considering that final line in beforeEach, Selenium will use its ControlFlow promise class to wait for the browser to load before navigating to the Dialogic website. The get() method returns a promise that has a then() function call attached to it, calling Jasmine’s done() method. All of this song and dance is done to make sure that your tests are not executed until the page is ready to be interacted with.

With this understood, let’s consider the only line in the afterEach method:

Here I’m calling the quit() method on our driver. This will close all processes and browsers that are associated with that particular driver instance. In this case, that would be any Firefox browser that was opened. The quit() method, like the get() method from earlier, returns a promise that can be used to indicate when the afterEach function has completed execution.

Drive a Different Browser

I’m using Firefox above, which is built-in to Selenium but I had us grab the ChromeDriver, so let’s use that. First add the following line to the top of the script:

Then you just change one small part of the beforeEach call:

Let’s talk about executing this test spec before we go too much further.

Executing the Tests

If you installed Jasmine globally, you can run Jasmine at the command line. Further, since you followed the *spec.js naming convention, you could just do this:

$ jasmine

At this point you’ll be told that no specs were found. We’ll fix that in a bit, but do note that you can execute specific test files by simply telling Jasmine what that file is. This is very helpful if you didn’t want to name the file with the convention. For example, I could do this:

$ jasmine spec/my-test.js

Finally, you may not even want to refer to the specific test runner and instead use NPM to run your tests. The main reason for this would be if you want to allow yourself the freedom to use different test runners, but providing one consistent command. To do this, change your package.json file as such:

With that change in place, you can now run the following command:

$ npm test

This will cause NPM to execute whatever command is in the “test” section of the “scripts”. All this being said, we still don’t have any actual tests yet, or what Jasmine calls specs. We have a test specification file, but no actual tests within it. We’ll fix that now. Let’s add a test after the beforeEach and afterEach functions:

Here I’ve added a Jasmine it method, which is an executable test. I’m using Selenium’s findElement() method, which it will use to send a request to the browser via WebDriver. (See how it’s important to keep all these components in mind?) What happens here is that WebDriver communicates with the page markup and returns the first matched element. This is done by using the By namespace, which provides various means by which elements can be located. In this case, I’m using the tagName() method.

Then I’m using the getAttribute() method on the returned element. In this case I’m looking for the ‘id’ attribute of the element. As before, the method returns a promise that will be resolved, which will pass the value of attribute and storing it in the “id” argument passed to the function. From there, I’m using an expect() method. I should note that this is a key difference between Jasmine and Mocha: Jasmine provides its own assertion/expectation library, whereas you have to choose one with Mocha.

The upshot of this test is that I’m checking if the “article” element on the page I have navigated to has an “id” value of “weight”. If it does, that’s a good thing because it’s supposed to! Also, as discussed before, I’m calling Jasmine’s done() method to make sure that the entire function has finished processing.

Dealing with Timings

Now, even with all the stuff I’ve been talking about regarding “waiting for functions to finish”, you might find if you run the test now against my Dialogic site, you will get a failure like this:

Error: Timeout – Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.

First of all, the reason this is happening is likely because unless someone else has hit my Dialogic site, the Heroku servers need to take the site up from its slumber in order to serve it. This is a side-effect of Heroku putting your site into a form of hibernation so as to save resources. To fix this you can change the default timeout interval value. Check out the Asynchronous Support section of the Jasmine documentation.

However, you can also add a specific timeout as the last argument for any it function:

Here I’ve added a timeout value of 12,000 milliseconds, or 12 seconds for this particular test.

Assuming all works, you’ll see browser windows pop open and then close. The console will output the results of the Jasmine execution.

Let’s add one more test:

This is mainly just to show you a little more interaction with the page and give you some more material to play around with. I’ve already provided plenty of links for you to investigate exactly how this is working.

Wrapping Up

A few key points to take from this post is the different components you have and what is being used when. For example, know that Jasmine was the test runner and thus methods that it provided were used to structure and execute the tests (describe, it, beforeEach, and afterEach). Selenium was the test execution tool (using WebDriver) to communicate with a browser and thus methods that it provided were used (findElement, getAttribute, sendKeys, click).

Hopefully this has given you a good start in exploring further on your own. One thing you may note is that these tests look a little busy. It’s kind of hard to read the automation code and figure out what’s happening at a glance. This is due largely to the Selenium API and there are various wrapper libraries around it, as I mentioned before. I will cover one of those in a future post. Also note here that I have done nothing to modularize my logic a bit, such as using the page object pattern. Again, that will be covered in a future post.

Share

About Jeff Nyman

Anything I put here is an approximation of the truth. You’re getting a particular view of myself … and it’s the view I’m choosing to present to you. If you’ve never met me before in person, please realize I’m not the same in person as I am in writing. That’s because I can only put part of myself down into words.

If you have met me before in person then I’d ask you to consider that the view you’ve formed that way and the view you come to by reading what I say here may, in fact, both be true. I’d advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.

This entry was posted in Automation, JavaScript, Test Solutions. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *