Our Continuous Integration Setup
When I started at DockYard, Brian tasked me with setting up a continous integration (CI) server so that we could keep an eye on our RSpec test suite. We went with Jenkins since we are writing client code, so travis-ci.org is out of the question (for now).
Our CI server is running on Ubuntu 10.04. We are using nginx as a reverse proxy infront of our Jenkins server. Our basic setup is the same as the presentation I gave at Boston RB in January. There are a few upgrades I have made since then. First, I set up the GitHub authentication plugin. The other plugin I installed was the Campfire notification plugin. Since we are all remote, we use Campfire as our main line of communication. Having Jenkins notify us in Campfire allows us to see when new code is pushed to master, and when someone breaks the build.
RSpec HTML formatter + Jenkins = Every Build is successful (even when it isn't)
As we found out the hard way, using the RSpec HTML formatter from within jenkins is not the best idea. The problem is the HTML formatter returns the same exit code regardless of whether or not the suite passes. This is a huge problem, as you end up with false positives.
The ci_reporter gem provides a rake task that generates a set of xml reports that Jenkins can interpret and give us a more complete picture of our test suite. Jenkins will plot the number of failure over time, display test duration, and provide a number of other stats you can utilize.
I added the following code to our
1 2 3 4 5 6 7 8
config.before(:suite) do @headless = Headless.new @headless.start end config.after(:suite) do @headless.destroy end
The above spippet creates the headless instance and creates the framebuffer at the beginning of the test suite, and destorys it afterwards.
Overall, I'm pretty happy with our set up as it is. The one issue I have with it is the way ci_reporter and jenkins interact. Since Jenkins was origianlly built for Java, builds are BROKEN when they don't build, but UNSTABLE when their tests fail. UNSTABLE builds are seen as successful. I would rather an UNSTABLE build be seen as a failure, since the campfire notification plugin plays the same sound for successful and unstable builds. I may poke around with the plugin or ci_reporter to have jenkins notify us of builds in a way that makes more sense.