Behat and CSS testing

Behat logoIt often happens that new Drupal website components display differently on different pages, or that CSS changes affect one page differently from other pages. With most coding, regression testing is used to trap for these sorts of issues. Even with the most popular test engines – CasperJS, Karma, Jasmine – you cannot create back-end unit tests. This is why, for testing Drupal sites, our QA team chose the Behat framework. The question is: Can we use Behat to test our site for visual consistency as effectively as we use it to test, say, JavaScript? Can we use Behat to test our CSS?

A lot of factors need to be taken into account for visual styles testing: position, dimension, color, font, background image – it’s a long, long list of potential parameters. The time spent on test creation can as much as actually creating new functionality! And, by the end, the probability of new bugs is not excluded…

I found CSS testing perceptual diff, as in PhantomCSS, very useful. The pros and cons of this method are discussed in depth by software engineer Brett Slatkin in this video:

 

Of particular interest is the case of new features appearing in places where they should not appear. And even an impressive set of tests doesn’t detect a bug!

1px_single_element_changeBecause I plan on using functional unit testing and CSS test in one box, I found a couple of perceptual diff Behat extensions:

https://packagist.org/packages/cevou/behat-screenshot-compare-extension

https://packagist.org/packages/zodyac/behat-failure-screenshot-extension

Both are built around the Imagick PHP extention and using the compare() function. I also used the cropImage() function to compare parts of the page. Creating a perceptual diff test is very simple and effective. The comparison function returns TRUE as if successful, which simplifies the process of test automation.

To make my test more effective, I divided the page into blocks that have the same selectors and compared only those data blocks. Finally, when my test was finished, the QA engineer only had to write a single line feature:

This is another advantage of the Behat: for a better presentation of the results of tests, provide an example result of the comparison block changes from one block size 1px.

Of course, CSS tests should also be conducted so you understand more clearly the reason for a test’s failure. But it’s not necessary to check all styles of the design layout. Time spent writing solid, high quality perceptual diff tests means faster execution speed and more precisely defined errors.

4 thoughts on “Behat and CSS testing

  1. Solid article!
    But you know, those mentioned extensions for behat are so outdated to use.
    They were started and dropped in 2012. And didn’t came out from ‘alpha’.

    So the only option is to write smth simular by yourself.

  2. That’s true Alex. For my task I have wrote my own code. It’s not too much of work and you can use extensions in the article as example.

  3. If you need quick and easy CSS testing automation, I would suggest you trying Screenster (http://screenster.io). It does not require coding and can be regarded an alternative to Selenium for visual/CSS testing tasks.

    Screenster is a UI testing tool which performs screenshot-based comparison of different versions of your web pages. First it creates a visual baseline for a page, taking a screenshot for each user action. During the next run it takes a new screenshot at each step, compares it with the one from baseline and highlights differences. It also has a number of features for easy maintenance of tests.

Comments are closed.