Diffy Visual Regression Testing: Improved confidence and productivity

I sat down with Yuri Gerasymov, founder of Diffy, to discuss how we use Diffy and how it has given us more confidence and productivity in our continous improvement workflows. Here's the case study that resulted, pulled from the Diffy website.

The challenge

Capellic is a Drupal agency focusing on building and maintaining Drupal websites for nonprofits.

The testing process relies a lot on someone very familiar with the website, a historian who would go and manually check the pages. This includes a side-by-side comparison of multiple pages. This is very tedious work and requires a large screen. Not everyone tolerates this kind of work and is so detail-oriented.

After some time, the team realized this wouldn’t hold up. It prevented the Capellic team from doing the work they wanted to do because they just saw the potential impact of changes and how much testing should be involved.

Solution

Team uses Diffy in three main ways:

1. Storybook components

When the new site is being built, the team delivers the components library using Storybook. Each of the components gets approved and then captured with Diffy. So every change gets checked against the baseline, so unexpected changes get picked up very fast.

2. Updates automation

On a service side, the Capellic team has plenty of maintenance sites. The team has built a CI workflow for a script to pull in updates monthly, push them to a dedicated Pantheon multi-dev environment, and get the sites tested with Diffy.

3. Whole pages testing during new development

Similar to the first case, when the new site is being built, Diffy gets trained on the whole pages of the site. When new components are implemented, the team ensures that approved ones do not get broken on the site.