Testing Chef Cookbooks. Part 1. Foodcritic.

This post have been in draft for almost a year. At last I have made myself to turn back to my blog and continue writing.

A couple of years ago we have adopted automating our server configuration through chef recipes. In the beginning we didn't have many cookbooks and all the recipes were more or less simple. But as time passed new applications had to be installed and configured, including some not so simple scenarios. Turned out that "hit and miss" method wasn't too good. We came across a lot of errors, such as:

• typo: simple as that. Although knife makes a syntax check on uploading cookbooks to server, but many times it was a wrong path or something like that, which could be revealed only after we tried to provision a node.
• missed dependencies: we ran our scripts on ec2 instances, and to save time on bootstrapping the new instance, we didn't do it every time we changed something in our scripts, but only the first time. When we finally got the recipe working on this node, sometimes it turned out that the recipe will not be working on another clean node, because we forgot some dependencies, that were somehow already installed on our test node.
• interfering applications: some recipes were used for configuring several applications. Although they were very similar, they were not identical, and sometimes changing the recipe for installing one application broke installation of the other one.

At last we came to the idea, that infrastructure code as any other code should be tested. Currently we have built a pipeline using Jenkins that first runs syntax check, then coding conventions tests, then unit tests and finally integration tests on cookbooks. And only if all the tests are passing, Jenkins runs knife cookbook upload, publishing the cookbooks on chef-server.

In my following posts I will share with you how we established this testing architecture for our chef recipes. There will 3 parts in the tutorial, we will start from the easiest checks that only make sure that your ruby code can be parsed and follows some code conventions.

First of all make sure you are running ruby 1.9.2 or newer. (We use Ubuntu on our linux servers, so all the code I provide is tested in Ubuntu 12.04 LTS.)

sudo aptitude install ruby1.9.3

Now we need rake for building our project and foodcritic for testing.

sudo gem install rake foodcritic --no-ri --no-rdoc

Chef cookbook repository has a Rakefile in it. Now when we have rake installed, we can try and run rake inside the folder Rakefile is in.

rake

This should run syntax check on your recipes, templates and files inside cookbooks. Next we can try to run foodcritic tests on your cookbooks. Type

foodcritic cookbooks

and hit enter (assuming that "cookbooks" is the folder your cookbooks are in). If it founds some warnings, it will print them out, otherwise an empty string will be printed.

Of course you may not agree with some of the rules and would like to ignore them. This could be achieved by providing additional options to foodcritic command. You can also write your own rules and add them into checks. For example, there was a FC001 rule, which stated "Use strings in preference to symbols to access node attributes". I actually preferred vice versa using symbols to strings. So I created a new rule:

rule "JT001", "Use symbols in preference to strings to access node attributes" do
 tags %w{style attributes jt}
 recipe do |ast|
  attribute_access( ast, :type => :string )
 end
end

and saved it into foodcritic-rules.rb file. Then it was easy to disable the existing FC001 rule and enable mine with:

foodcritic cookbooks --include foodcritic-rules.rb --tags ~FC001

There are also some 3^rd party rules available, so you have something to start with.

Now we will join rake and foodcritic together by creating a rake task that runs foodcritic tests. Add a new rake task similar to:

desc "Runs foodcritic linter"
task :foodcritic do
  if Gem::Version.new("1.9.2") <= Gem::Version.new(RUBY_VERSION.dup)
    sh "foodcritic cookbooks --epic-fail correctness"
  else
    puts "WARN: foodcritic run is skipped as Ruby #{RUBY_VERSION} is < 1.9.2."
  end
end
task :default => 'foodcritic'

Now when you run rake it should run both syntax and code conventions tests.

Next post will be about unit tests using chefspec.