As a tester wanting to write test tools in the JavaScript context, you have to get used to the concepts of callbacks and promises. This is one area that is very different from other programming languages when considering automation. So let’s talk a bit about that.

One of the key challenges with JavaScript is that it is asynchronous. Getting familiar with asynchronous behavior can be a little bit of a challenge.

Why is that? Well, when you execute something synchronously, you wait for it to finish before moving on to another task. When you execute something asynchronously, you can move on to another task before the current one finishes. If you think about most testing, you’ll realize that you need things to be done in a certain order. When you get into automated checking, you need to control the order in which code executes, similar to how you control your test tasks.

As I said, JavaScript is asynchronous. This means whatever automated check code you execute will be executing in the context of asynchronous behavior. To tame asynchronous behavior, Javascript provides two mechanisms: callbacks and promises. Knowing how JavaScript handles callbacks allows you to control the order in which your code runs. This is as opposed to simply relying on the vicissitudes of chance. So let’s take a very brief look at that.

Callbacks

Synchronous code is easy to understand because it executes in the order it is written. A good comparison on this point is often made using the synchronous and asynchronous file APIs in Node.js, so I’ll do the same thing here. What follows is a script that writes to a file and reads back the contents synchronously.

Each of console lines will print in order. You would see output like this:

Checking Contents
Sat Mar 05 2016 07:28:19 GMT-0600 (CST)
Script Finished

The script uses the writeFileSync and readFileSync functions of the filesystem (fs) module to write a timestamp to a file and read it back. After the contents of the file are read back, they are compared to the timestamp that was originally written to see if the two values match. The console.assert() displays an error if the values differ. In this example they always match so the only output is from the console log statements before and after the assertion.

Now let’s consider this script:

Here the output will be a bit different:

Script Finished
Checking Contents
Sat Mar 05 2016 07:31:07 GMT-0600 (CST)

This script does the same job as the previous one, but this time using the asynchronous functions writeFile and readFile. Both functions take a callback as their last parameter. Comparing this code to the previous example, you’ll see that the console output appears almost in reverse order. Here the call to writeFile returns immediately so the last line of the script runs before the file contents are read and compared to what was written.

In JavaScript, callbacks are used to manage the execution order of any code that depends on an asynchronous task. A problem here is when you put code that relies on the completion of an asynchronous task outside the appropriate callback. If you do that, it creates problems. As an example, here is a similar script to the previous ones:

Your output will be something like this:

Comparing Contents

assert.js:89
  throw new assert.AssertionError({
  ^
AssertionError: false == true

This code expects the callback given to readFile to be invoked before readFile returns, but when that doesn’t happen the content comparison fails. Make sure you understand that. The callback to readFile is always invoked asynchronously, so readFile is guaranteed to return before invoking the callback. Once that happens, the callback never runs before the log or assert statements on the last two lines because of what are called “run-to-completion semantics” in the JavaScript world. This simply means that the current task is always finished before the next task is executed. Specifically, line 9 in the script above executes after lines 12 and 13.

At the risk or boring you, let me do just one more example here because I want to make sure these points are driven home.

Take a look at the sample code below and think about its output.

From looking at the code you would expect “Avengers Assembled!” to be printed last after displaying all of the contents of the array. But that is not the case. Instead you’ll see this output:

Final List:
Avengers Assembled!
Assemble: Captain America
Currently Assembled: Captain America
Assemble: Iron Man
Currently Assembled: Captain America,Iron Man
Assemble: Thor
Currently Assembled: Captain America,Iron Man,Thor
Assemble: The Vision
Currently Assembled: Captain America,Iron Man,Thor,The Vision

So “Avengers Assembled!” is actually the second line to be printed out. Let’s consider this a little bit.

Here I basically have a loop that is reading from an array and pushing each item from that array into a new array. I’m using setTimeout to trigger a function — in this case, an anonymous function — after a given amount of time. The setTimeout function accepts two arguments: a function to call and the minimum number of milliseconds to wait before calling the function.

That second parameter specifies the minimum amount of time that will lapse before the callback is run as opposed to the exact amount of time. It’s impossible to know exactly when the callback will run because other JavaScript could be executing at that time and the machine has to let that finish before returning to the queue to invoke your callback.

So what I did here is simulate what’s called a non-blocking operation that will take 1000 milliseconds to complete with the setTimeout function. Within that context I do my array push operation. Due to Javascript being asynchronous, it does not wait for the operation to finish and starts the next operation right away. As a result, all array push operations are still running when it reaches line 12 where I print the array contents which is an empty array. Once it has finished with line 12 and 13, all array push operations have been completed and the output of each push are displayed afterwards.

One of the bigger challenges with nontrivial amounts of asynchronous JavaScript handled by callbacks is managing execution order through a series of steps and handling any errors that arise during those callbacks. That gets us into promises.

Promises

Promises address this problem by giving you a way to organize callbacks into discrete steps that are easier to read and maintain. Promises can be combined to orchestrate asynchronous tasks and structure code in various ways. This is a lot easier to show in operation so I’m going to do so in the context of WebDriverJS.

WebDriverJS returns Promises from all of its browser interactions. But this can lead to some confusing behavior as I’ll try to show here. First of all, what’s a promise? A Promise (yes, with a capital P) is “an object that represents a value, or the eventual computation of a value.” You can read up on the Promises/A+ specification or look at some information on Promises. Essentially, you’ll most often hear that a promise is an object that serves as a placeholder for a value. That value is usually the result of an asynchronous operation. When an asynchronous function is called it can immediately return a promise object. Using that object, you can register callbacks that will run when the operation succeeds or an error occurs.

So let’s consider the start of a simple script here:

Now let’s say I wanted to perform some actions:

• On the planets page, enter 200 in the weight field.
• Click the calculate button.
• Find out what the value provided for Mercury was.

Each of these statements will become code, but they have to be executed synchronously. This works in Python, Ruby, Java, C# and so on because the Selenium API when used with those language is blocking: meaning, it does one step and is blocked from doing the next step until the current step finishes. JavaScript is not like that. JavaScript is asynchronous. This means JavaScript will keep on moving on, not waiting for a particular step to finish before moving on to the next one.

WebDriverJS uses Promises to get around this.

First I’m going to show you some code that would utilize the strict callback structure I was talking about above:

Lots of indenting. This is sometimes called “callback hell” in the JavaScript world, where you simply have this ‘tower’ of callbacks that is getting built up as you need actions to take place in a specific order.

Promises make this not as terrible by having a then() method which is used to get the eventual return value of an operation. So promises, simply speaking, are simply another method of dealing with asynchronous code. With Promises, what’s called a “resolution context” is returned at the end and a then() method can be chained for the next operation. So the above callback version of the script would look like this:

As you can see, this leads to a chain of then functions. It doesn’t require as much nesting as the callback example, so that’s something, I suppose. Still, though, it probably feels a bit messy, right?

So here’s where it gets interesting. WebDriverJS provides a Promise Manager that helps with the scheduling and execution of all promises automatically. Specifically, WebDriverJS has a wrapper for a Promise called ControlFlow. What ControlFlow does is maintain a list of actions that have been scheduled to execute. They have been “scheduled” because you wrote them as a series of steps in your script.

When those WebDriverJS actions are called (findElement, click, sendKeys, etc), they don’t actually execute immediately. What they do is push the required action into the scheduled list of actions. ControlFlow then puts every new entry in that list in the then callback of the last entry in that list. This is what ensures the sequence remains as planned.

What that means is I can just write the above script like this:

Notice here that there are no callbacks or manual promise chaining and the code looks very synchronous-like. Each line of code that is a promise gets added to the queue and everything runs top down just like a synchronous language.

That said, the tricky part comes in when you want to extract values from the page. In that case, you have to explicitly handle the Promise yourself:

When JavaScript Breaks Promises

So it all sounds great, right? WebDriverJS (along with other libraries) incorporate promises that let you structure code and some of those libraries (again, like WebDriverJS) provide mechanisms, like promise managers, to make the coding of such mechanics largely transparent. Except — that’s not always a good thing. Let’s consider another example here where I use a test runner (Mocha) which is what you would normally do:

You can see I’ve simply taken the bits of the script that I wrote earlier and put them in specific blocks for the Mocha test runner. If you were to run this, however, you would see it runs quick. As in really quick. Part of that is because the script didn’t bother opening a browser at all. Yet — you would see the script passed. How is that possible? I have an assertion in there!

Having promises broken by machines is, as it turns out, no more fun than having them broken by humans. The problem here is that the test runner has no idea that I want to wait for a browser to appear. Keep in mind that JavaScript is asynchronous which, in this context, means that interactions with the browser are non-blocking. In fact, you might notice that the first step is passing even though the browser hasn’t shown up and navigated to the page yet.

So what do we do to fix all this? Well, Mocha is what they call “promise-aware.” This means that I can — and in fact must — return that promise from the test. The change to the code is very simple. To handle the first issue of the first step passing without the browser even opening yet, just make this simple change:

To fix the assertion problem, you do pretty much the same thing:

All I did in both cases was add an explicit return statement.

Now the script would work as you intended. But what you saw here was the idea of broken promises. This means things can show up as working and yet not actually be working, which leads to a lot of analysis on your part to figure out what’s going on and why.

If you read my previous post on this topic (JavaScript with Selenium WebDriver and Mocha), you’ll probably see that much of the code I presented here was used there as well. But I didn’t seem to have the same issues in that post that I’m showing here. That’s because in that post I used the promise manager that is wrapped around Mocha to get around these issues. I did that by using the following:

And then my describe and it blocks that held my script logic were instead test.describe and test.it. That’s what makes what I did there different from what I did in this post.

So What’s the Takeaway Here?

What all of this means is that you have to be aware of what JavaScript libraries you are using, whether they are promise-aware, and some of the issues with how those promises are dealt with by the library.

This is a very different challenge than you will have likely faced in most other programming languages where you were using WebDriver to drive actions against a browser.