Backstop

This article was originally posted at Manatí

We usually don't want to break already existing functionality. We usually don't want to break already good-looking stuff. That's there where visual regression testing comes to the rescue: it allows you to compare 2 sets of screenshots and find the differences between them.

There are several tools for doing this; however, for this guide, I'll focus on BackstopJS because it's easy to use and easy to start working with it. It's just a matter of npm install backstopjs --save-dev and ./node_modules/.bin/backstop genConfig to get an initial backstop configuration where you only need to change urls and some minor adjustments to make your tests work.

In the generated config, in viewportsyou could add different screen sizes to be tested; and under scenarios you'll have one JSON object for every page you want to test. Under that object, you'll have the url and you could also have a referenceUrl if you want that your reference screenshot and your test screenshot are from different urls (e.g. when you want to test your production site vs your development site). Other important properties in that object are onBeforeScript and onReadyScript. With the first one, you can add javascript that will be executed prior the page load (for example, you could login, add cookies, etc) and with the second one you can interact with the page loaded as if you were an user. We'll come back to this later to demonstrate how you can login in a Drupal site when testing with BackstopJS.

So, given that, you can see it's really really easy to implement this for Drupal testing when you don't need to login to screenshot the pages. When you need to do that, it's a little more complex and that's what we're going to do now.

For our imaginary example we have the following situation:

• A public repo with drupal code inside a folder and some additional stuff in the repo root
• We don't want to write passwords in the repo for existing users (it would be really bad from a security perspective); so, we need to login without password
• We need to ensure we can run this in a CI server hopefully with only one command
• Our live site is hosted somewhere and we have drush aliases to interact with it

While researching for this I found this link that contains a really interesting "library" for cookie management (DCookieManagement.js), so I included it in my casper_scripts folder. Besides that, based on that post, I created a login.js file that receives a drush uli generated url, opens it and save the cookies into a file. It looks like this:

var casper = require('casper').create({
  pageSettings: {
    loadImages: false,//The script is much faster when this field is set to false
    loadPlugins: false,
    userAgent: 'My User Agent String'
  }
});var loginUrl = casper.cli.args[0];
if (typeof loginUrl === 'undefined') {
  casper.exit(1);
}// Create cookie manager object. Cookies will be saved in file called cookies.txt.
var cookiesManager = require('./DCookieManagement').create("./backstop_data/cookies/cookies.txt");
// Cookie file exists, try to read it.
if (cookiesManager.cookieFileExists()){
  // If file exists, nothing to do.
  casper.exit(0);
}// First step is to open the site and instantiate cookiemanager.
casper.start().thenOpen(loginUrl, function() {
  console.log("Website opened");
});// Wait to be redirected to the Home page, and then save cookies.
casper.then(function(){
  console.log("Save cookies.");
  cookiesManager.loadCookies(phantom.cookies);
  cookiesManager.saveCookies();
});casper.run();

And I added an onBeforeLogin.js file to be executed onBeforeScript to inject cookies when needed:

module.exports = function (casper, scenario, vp) {
  // Create cookie manager object. Cookies will be saved in file called cookies.txt.
  var cookiesManager = require('./DCookieManagement').create("./backstop_data/cookies/cookies.txt");
  // Cookie file exists, try to read it.
  if (cookiesManager.cookieFileExists()) {
    cookiesManager.readCookies();
    phantom.cookies = cookiesManager.getCookies();
  }
  console.log('onBeforeLogin.js has run for '+ vp.name + '.');
};

Now, we need to edit the onBeforeScript setting for scenarios that need login and set our new onBeforeLogin.js script for them.

Once done that, it will work; however, the command line to make it work will be really complex:

rm -f backstop_data/cookies/cookies.txt ; casperjs backstop_data/casper_scripts/login.js `drush @live.site.alias uli --name=username` ; backstop reference ; rm backstop_data/cookies/cookies.txt ; casperjs backstop_data/casper_scripts/login.js `drush @dev.site.alias uli --name=username` ; backstop test ; rm backstop_data/cookies/cookies.txt

Let's explain it:

• Remove the cookies file if it exist
• Run login casper script with drush uli for live site result to get cookies from live site
• Run backstop reference to get reference screenshots
• Remove cookies file
• Run login casper script with drush uli for dev site result to get cookies from dev site
• Run backstop test to generate test screenshots and compare them with reference screenshots
• Remove cookies file

It would be nice it we can do this with one or two short commands. For this, we can use the npm scripts section in the package.json file. Add something like this to package.json:

"scripts": {
    "backstop-reference": "rm -f backstop_data/cookies/cookies.txt ; casperjs backstop_data/casper_scripts/login.js `drush @live.site.alias uli --name=username` ; backstop reference",
    "backstop-test": "rm backstop_data/cookies/cookies.txt && casperjs backstop_data/casper_scripts/login.js `drush @dev.site.alias uli --name=username` && backstop test && rm backstop_data/cookies/cookies.txt",
    "backstop": "npm run backstop-reference && npm run backstop-test"
  }

Now, if you run npm run backstop-reference it will get reference screenshots even if some of them needs login and if you run npm run backstop-test it will run the test even if some scenarios need login. To run both of them, just execute npm run backstop.

Now, you can integrate this without major issues to the CI service that you're using or run it locally and it should work as expected.

Hope it helps!

• Drupal,
• Visual Regression,
• Automated Testing