Many testers are working in a DevOps context now or soon will be. This context is often about making a new environment available (virtualization) or taking an environment and making sure it has everything needed (provisioning). Usually these two go hand-in-hand. Here I’m going to show you one of the simplest possible ways to do this using one component of Chef. Make no mistake about it: this kind of automation is critical for the modern tester to learn.

As the discipline of software development has matured, frameworks have emerged with the aim of reducing development time by minimizing the overhead of having to implement or manage low-level details that support the development and testing effort. This allows developers and testers to concentrate on rapid delivery of software that meets customer requirements. One of the key areas here is in the availability of environments for developing or testing.

Tools of the Trade

As with programming languages or automated test tools, opinions are many and there are numerous tools to choose from, all of which have different pros and cons. So let me lay out what I’m going to use.

Vagrant, from HashiCorp, allows you to set up and control environments. By default, Vagrant is a wrapper for Oracle’s VirtualBox but Amazon EC2 and VMWare boxes can be set up and controlled just as easily. Even better, there is native support for provisioning with Chef. (Some alternatives to Vagrant are Docker and Vortex.)

Chef, from Opscode, and was designed to turn provisioning into an automated process. It operates using cookbooks, which are basically self-contained scenarios the installation and configuration of various resources. These cookbooks are written in Ruby and managed as source code. Each cookbook contains recipes that describe what a part of your infrastructure should look like. When you configure your new server, you let Chef know what cookbooks and recipes should be applied to build out that infrastructure. (Some alternatives to Chef are Puppet, Ansible, CFEngine, and Fabric.)

For testers in particular, the use of these tools means you can spin up and provision a test environment with specific databases, specific browsers, specific application servers, and so on. You can essentially a construct an environment that is ready to have tests applied to it, whether those tests be manual or automated.

In two previous posts I talked a lot about using Vagrant for virtualization and a little bit about how to provision. Here I want to focus a little more on Chef simply because it has quite a bit of momentum in the industry.

I definitely recommend you go through the installation of VirtualBox and Vagrant (covered in the Vagrant post) as you will need those for everything you are going to do. For this post, you don’t need to worry about installing Chef on the VM since Vagrant will take care of that automatically for you but you will need it installed locally to run one of the command line tools.

You can just go to the downloads area for the chef-client and install it for your operating system. Once you have done that, make sure you have it installed and working:

$ chef-client --version

Provisioning with Chef Solo

Here I’m going to use one particular component of Chef called chef-solo, which is a fully featured standalone configuration management tool. It allows access to a subset of Chef’s features without using a Chef server. This is suitable for simple deployments. You can read more about it in the chef-solo documentation.

First, let’s create a new Vagrant project by creating a new project directory and then initializing a base environment:

$ mkdir my-test-env
$ cd my-test-env
$ vagrant init precise64 http://files.vagrantup.com/precise64.box

As a result of this, you should have a Vagrantfile in the directory with a bunch of comments and the following active lines:

I don’t want to turn this into a ‘learn Vagrant’ post, but just note that config.vm.box provides the name of the box Vagrant will use when building the guest machine. The config.vm.box_url setting contains the URL to where the box specified by config.vm.box can be found in case it isn’t already installed on the system. This setting is optional and is only used if the box doesn’t already exist on your machine.

I recommend removing all the commented lines for now just to keep things straight and clear.

Now that we have a basic Vagrant project set up, we can add Chef provisioning to it. This will let you install some new software packages to the largely empty box in an automated fashion. In this case, we’ll be quite simplistic and ask Chef to provision Apache httpd for us by adding a few lines to the Vagrantfile:

Here I’m using the config.vm.provision setting, which is used to enable and configure provisioners. Provisioners have one required parameter, which is the name of the provisioner to use. Valid values for this that are built into Vagrant are :shell, :chef_solo, :chef_client, :puppet, and :puppet_server. Additional provisioners are available from plug-ins. After specifying the name, an optional second parameter can configure the provisioner, which I’m not using here. In addition to the optional second parameter, an optional block can be given for more advanced configuration of the provisioner and that I am using here.

The added lines instruct Vagrant to use Chef Solo for provisioning and to provision “apache2” on the box. Specifically, the call to add_recipe loads up a recipe that we’re calling “apache2”. Further, cookbooks often need additional custom configuration. For this you can specify additional JSON data in the Vagrantfile as shown above, which basically just sets up a default site that helps you determine Apache is working. Given these lines, how does Chef actually know how to provision “apache2”? For that, it looks for a cookbook in a cookbooks subdirectory in the project current directory.

A slight problem right now is that you don’t have a cookbooks directory, much less any cookbooks in it. If you want to provision some commonly used package — and the Apache httpd web server certainly counts as commonly used — there is a very good chance that someone has already written a cookbook for it. You can get this cookbook from the OpsCode Community Supermarket and look for it there. You can also create your own cookbooks but that’s a bit beyond what I’ll talk about here.

The goal here is to provision the “apache2” package which is readily available on the OpsCode site. Given that, we can use knife — a command-line tool for Chef — to download the cookbook to the current directory:

$ knife cookbook site download apache2

You’ll see that the cookbook was saved as a tar.gz file. Incidentally, this file is grabbed directly from the Apache2 cookbook in the supermarket.

Now we’ll create the necessary cookbooks directory and extract the downloaded apache2 package into it:

$ mkdir cookbooks
$ tar xvzf apache2-*.tar.gz -C cookbooks

Even if you are on Windows, I recommend getting one of the many available Unix/Linux style command tools, whether via the GNU packages, Git Bash, Cygwin, etc. But, in a pinch, tools like 7-Zip will let you extract from compressed and minified files like tar.gz.

If you were to bring the virtual machine up now — with the command vagrant up — then the apache2 web server module would be provisioned on it. That said, it would be nice if you could verify that it actually works, right? Normally with Apache, you would just open the default website with a browser. However that default website is contained inside the virtual machine. So how do you access it?

Vagrant allows you to map a port on the host machine to a port on the virtual machine, forwarding all the traffic accordingly. So you can, for example, map port 9292 on your localhost to port 80 on the virtual machine. This is as simple as adding just one line to the Vagrantfile, which was similar to one of the lines that was already in place and commented out:

So now let’s bring the virtual machine up which will also automatically provision it since we added in the chef.vm.provision logic.

$ vagrant up

If you had already tried bringing up the machine with that command, it means the vm is already running. In that case, you can simply re-provision it by doing this:

$ vagrant provision

At this point you will likely see an error like this:

ERROR: Cookbook iptables not found.
If you're loading iptables from another cookbook,
make sure you configure the dependency in your metadata

I wanted to show you this because it’s a situation that can come up when a given cookbook has dependencies on other cookbooks. There are various ways to handle this but here I’ll opt for the simple route. To handle this, you need to get the iptables cookbook as well. If you aren’t aware of it, iptables is absically a firewall utility for Linux. Also, to spare you the trouble of figuring this out on your own, you will also need the logrotate cookbook. So, do the following:

$ knife cookbook site download iptables
$ tar xvzf iptables-*.tar.gz -C cookbooks

$ knife cookbook site download logrotate
$ tar xvzf logrotate-*.tar.gz -C cookbooks

Let’s try it again:

$ vagrant provision

Now you can open http://localhost:9292/ on your machine and you should see the standard “It works!” page that Apache provides. You can throw away the freshly built virtual machine:

$ vagrant destroy

If you wanted to recreate it, you could just do this:

$ vagrant up

So with a single command, you now have a freshly built virtual machine, configured and ready to go. As you can see, it is quite easy to use Chef and Vagrant together in order to automatically create virtual machines from scratch. This combination is what enables you to treat your infrastructure as code.

In the next post on this subject, I’m going to use Chef more as it was intended, which is as a client-server style tool. Also, I mentioned that you are now treating your infrastructure as code. The tester in you is probably thinking: “Code! Then I have to test it. But … can I test infrastructure code?” You certainly can and that’s a subject I’ll also tackle in a future post.