CI for UI: Visual Regression Testing with Percy and Playwright

Overview

Visual regression testing helps you catch unintended UI changes before they reach users. By pairing Percy with Playwright, you can automate snapshot captures during CI runs, compare them against baselines, and surface diffs as part of your pull requests or deploy pipelines. This post walks through a practical setup, from local preparation to a CI workflow, so your UI stays stable across code changes.

Why visual regression testing matters

• It detects visual drift that unit or integration tests miss.
• Percy creates a visual baseline and highlights pixel diffs across themes, layouts, and responsive widths.
• Playwright provides robust browser automation and testing capabilities, enabling end-to-end UI interactions before snapshots.

Prerequisites

• Node.js and npm or pnpm
• A Playwright project with tests (Playwright Test)
• A Percy account and a Percy project to receive snapshots
• Access to CI (e.g., GitHub Actions, GitLab CI, or another CI system)

Set up Percy with Playwright

• Install Percy CLI and Playwright integration:

npm i -D @percy/cli @percy/playwright

• Update test files to capture Percy snapshots:

// tests/home.spec.ts
import { test, expect } from '@playwright/test';
import { percySnapshot } from '@percy/playwright';

test('Homepage renders correctly', async ({ page }) => {
  await page.goto('https://example.com');
  await percySnapshot(page, 'Home');
});

• Ensure you have a Percy token available in CI:

  • In your CI environment, set the PERCY_TOKEN variable (e.g., as a secret in GitHub Actions).
  • The token authorizes Percy to upload snapshots for your project.

• Optional: add a Percy configuration file to tailor snapshot behavior

// percy.config.js
module.exports = {
  version: 2,
  discovery: { requestTimeout: 20000 },
  // Optional: specify snapshot widths and minHeight
  // snapshots: { widths: [375, 1280], minHeight: 768 },
};

• Add a script for ease of use (optional but handy):

"scripts": {
  "test": "playwright test",
  "visual": "npx percy exec -- npx playwright test"
}

• Run locally to verify:

npm run visual

This will start the Playwright tests and upload the captured snapshots to Percy for diffing.

CI integration: GitHub Actions example

A minimal workflow to run visual regression tests on push and pull requests:

name: Visual regression tests

on:
  push:
    branches: [ main, master ]
  pull_request:

jobs:
  percy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
      - run: npm ci
      - run: npx playwright install
      - name: Run Percy visual tests
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}
        run: npx percy exec -- npx playwright test

Notes for CI:

• Ensure PERCY_TOKEN is configured as a repository secret.
• Percy will upload snapshots for every test that uses percySnapshot, and the CI run will fail if diffs exceed your configured thresholds or if snapshots fail to upload.

Best practices

• Start with a curated set of critical pages and interactions to snapshot (home, product pages, checkout, responsive states).
• Use multiple viewport widths to cover desktop and mobile layouts.
• Stabilize selectors and avoid content that frequently changes (dates, dynamic ads) in snapshots.
• Review diffs in Percy promptly to keep baselines meaningful.
• Use CI to gate PRs: failing visual diffs should block merges until reviewed or exceptions are added.

Troubleshooting

• If snapshots don’t upload, verify PERCY_TOKEN is set in CI and the token belongs to the correct Percy project.
• For flaky snapshots, ensure tests wait for dynamic content before percySnapshot is called, or conditionally skip snapshots on known flaky states.
• If Playwright tests fail due to environment issues, run locally with the same browser binaries and versions as CI (e.g., npx playwright install).

Final notes

By integrating Percy with Playwright in your CI, you gain repeatable, automated visual checks that help protect UI integrity across code changes and deployments. This approach keeps your UI stable while enabling rapid iteration.