03.12. Setting up an own QA Environment for Javascript

Nearly every PHP project comes up with a great set of tools to assure the quality of source code; unit tests are a no longer a „nice to have“ feature, they are common components in new projects. While unit tests help you to provide solid interfaces and proof the functionality of a certain method, there is a long list of tools that check the quality of the source code itself with many different metrics. Each of these mentioned tools reports its results into a XML file, which can easily be interpreted by a continuous integration server like Jenkins. The benefit is enormous: After every commit to your version control system your continuous integration server triggers the execution of the tests and the source code quality analysis and shows the result in meaningful diagrams. As a developer you get a direct feedback and you can make a prediction on the status of the entire project, because if even the smallest units work fine, there is a good chance that the whole system runs stable.

So far for developing with PHP, but what about JavaScript? Are there any tools for quality assurance? Yes! Is it really necessary to do JavaScript QA? Oh yes! Since JavaScript is becoming one of the hottest programming languages for web (and mobile) development, there is no doubt that that we need some way to check the functionality and measure the code quality. Before I will show you some really cool tools and some easy steps on how to set up your own quality assurance environment, let me say a few words on how the increasing popularity of Javascript is affecting the requirements we have on QA for JavaScript and how it is different to QA for PHP. . A couple of years ago, JavaScript was only used for some (unneeded) animation, called DHTML, and for simple form validations. The AJAX hype gave JavaScript a second chance and from this point the lines of JavaScript code where spreading. In other words, most developers haven´t seen the necessity of writing down something similar to unit tests only for some small piece of JavaScript code, they use Firebug for debugging and it works fine for the moment. Another reason why JavaScript QA is different from PHP QA lies in the nature of JavaScript. Whereas PHP is mostly executed on a (Apache) server JavaScript is executed on many different environments and devices that don´t follow common rules and standards. That´s why the requirements for any QA tools are really difficult, because you want to cover a huge amount of environments and devices- otherwise you can only guaranty that your source code runs correctly on a certain device and I´m sure that´s not your intention. In addition, don´t forget the dynamic of JavaScript. It´s easy to extend the range of functions of JavaScript (because of it´s prototypical inheritance) or think of all the functions that come up with the environment, for example the Firefox File API. All these dynamic is really really hard to cover for QA. And last but not least, JavaScript is an interpreted language not a compiled one, this means there is no static code analysis that check the syntax tree like it’s done during compiling in other programming languages. So we have indicated two very simple requirements for JavaScript quality assurance: we need something to check our syntax and we need something that tests our code with many different environments/browser and devices. Ok, enough for the moment, let´s introduce our first tool: jsLint.

jsLint
This is not a completely new tool, invented by Douglas Crockford, jsLint is a JavaScript syntax parser written in JavaScript that does not need a browser for its syntax checks. You can use jsLint in many different ways, I prefer the Java wrapper for jsLint as a command line tool, see jslint4java [http://code.google.com/p/jslint4java/]. JsLint (and also jslint4java) comes up with a lot of possible optional parameters to define how strict your code should be checked.

Here is an first example of jsLint (called from the commandline via jsLint4java):


Not only syntax errors like the missing semicolon at the end of a line or a comma after the last entry of an array are considered, mixing spaces and tabs or a wrong indention will be checked as well. Using jsLint4java allows you to print the results into defined XML files. On the way to our own JavaScript QA environment we have entered our first milestone: We check our syntax tree with jsLint, in other words we do a static code analysis- cool!

From now on our source code quality will be checked by jsLint, but this doesn´t guaranty that our functions work well. This is the point where we should think about unit tests. There are a lots of great unit testing tools for JavaScript and nearly every frameworks comes up with its own implementation. To fit the requirement I have mentioned before, run the tests on many different environments (browsers) and devices, we need not only a standalone unit testing tool, we need something that runs the tests on these environments and devices. This is the point where our second tool enters the stage: please say hello to Googles jsTestDriver.

JsTestDriver

JsTestDriver is pretty easys to use it consist of a single JAR file which contains everything you need to get started. Here is an overview of how JsTestDriver works at runtime:
This tool combines an independent test framework and a testrunner to execute the tests on many different environments and devices.

This is the general architecture of jsTestDriver



Therefore the tool starts a simple HTTP server to connect with your browser. From this point your browser is marked as a “browser to test”, this process is called “capturing”. There are only two steps left to run tests on the captured browsers. Step one is to write down some tests and step two is to trigger the execution of the tests from the command line of your system. Furthermore for jsTestDriver it doesn´t matter whether you are executing your tests from Windows, Mac or any Linux operation system you can capture a browser form any remote system or a mobile device. By the way, it´s not strictly needed to add a browser by visiting the “capture page”, it´s also possible to pass the path to the browser to an optional command line parameter (–browser). In this case jsTestDriver will open the browser, execute the tests and shut down the browser. For writing the tests jsTestDriver provides a huge amount of assertions but it´s also possible to write the tests with nearly every test framework of your choice by using an adapter for it. By starting the tests, all tests will be sent over HTTP to all captured browsers and executed, the results will be sent back to the server and will be displayed by default at the screen. An export of the results into a XML file is also possible.

Here is a short example how a test for jsTestDriver could look like

MathmaticsTest = TestCase(„MathmaticsTest“);

MathmaticsTest.prototype.testAdd = function() {

var mathmatics = new school.Mathmatics();

assertEquals(3, mathmatics.add(1,2));

}

MathmaticsTest.prototype.testSub = function() {

var mathmatics = new school.Mathmatics();

assertEquals(1, mathmatics.sub(4,3));

}

MathmaticsTest.prototype.testDiv = function() {

var mathmatics = new school.Mathmatics();

assertEquals(4, mathmatics.div(8,2));

}

MathmaticsTest.prototype.testMultiple = function() {

var mathmatics = new school.Mathmatics();

assertEquals(9, mathmatics.multiple(3,3));

}

Executing the tests ends in a result like this:

After knowing these two very nice tools we already have two major components for or QA environment. What is still missing? Running a static code analysis and triggering unit tests for different devices from the command line is pretty cool already, but it would be even cooler if we were able to integrate jsLint and jsTestDriver into a continuous integration environment to automate the process of checking and testing our source code. A couple of lines ago, I have mentioned Jenkins as one possible tool for continuous integration and do you remember that jsLint4java and also jsTestDriver have a XML output? Bingo! This is the great benefit of these two tools, they generate a XML file that could easily be read by Jenkins (or any other continuous integration tool). Imagine this workflow: You write some new JavaScript code including tests, you check in your changes into your projects version control system, Jenkins recognizes that there are some changes and triggers the static code analysis and starts the execution of the test and after performing these two tasks the results will be displayed in some nice graphs.Sounds promising, doesn’t it? This is exactly what we will do with the next easy steps, because we only have to configure Jenkins- nothing more! Step one is to create an ANT task for each tool, this will be done in the projects build file (build.xml).
A sample definition of an ANT task could be found at the homepage of jsLint4 [http://docs.jslint4java.googlecode.com/git/2.0.1/ant.html].

Important for this configuration is the node target with it´s attribute name, because this is the value you have to enter into the Jenkins configuration. In addition, see the jslint node, where you can configure the strength or weakness of your analysis via the attribute options. Whereas the node formatter defines where the XML output should be saved, the fileset node defines the directory with the source code file that should be checked. That´s all for jsLint.
The task definition for jsTestDriver looks pretty similar:

This ANT task executes the tests on the defined browsers. Also it writes the output into the defined directory. And that´s all for the moment, all other needed steps have to be done in Jenkins, and that´s also easy.

Here the two needed steps:

And these are the results:

Conclusion

For this blogpost I have introduced two (free available) tools and showed the steps to build your own basic JavaScript QA environment with Jenkins for continuous integration. In summary what you will get with this setup is a solid environment for checking your source code quality and a way for automated unit test execution. Beside the presentation of the syntax violation results (jsLint) and unit tests (jsTestDriver) results I recommend a small subversion post commit hook that will send you an email with the results coming from jsLint. To conclude this blogpost, there is not only jsLint and jsTestDriver for measuring, checking and saving the quality of JavaScript code, for example it´s possible to use a plugin for jsTestDriver that checks the code coverage or define another ANT task that automatically generates a minified version of your source code.