Accessibility Testing With Pa11y CI

Accessibility is a crucial aspect of modern web development. In this guide, I’ll walk you through setting up Pa11y CI to run automated accessibility tests on your website.

Tools

1. Install Nodejs.
2. Install Chrome.
3. Install ChromeDriver. If you use Homebrew, you can install it with brew install chromedriver.
4. Install the needed pa11y packages with:

npm -g install pa11y-ci pa11y-ci-reporter-html

Configuration

Create a file named .pa11yci and add the following JSON configuration:

{
  "defaults": {
    "reporters": ["pa11y-ci-reporter-html"],
    "runners": ["axe", "htmlcs"],
    "includeNotices": false,
    "includeWarnings": true,
    "standard": "WCAG2AAA"
  }
}

• reporters: Configures the HTML reporter. Alternatively, you can set this to cli if you prefer command-line reports.
• runners: Enables tests to run with both axe and htmlcs. This gives you a comprehensive accessibility check.
• includeNotices: Set to false by default. While most notices are for manual checks, you might consider enabling this setting after focusing on errors and warnings initially.
• includeWarnings: Includes warnings in the report. Set to false if the warnings are too noisy.
• standard: Sets the standard for accessibility to WCAG2AAA, which is the strictest standard. If you’re new to accessibility testing, it may be beneficial to start with a more lenient standard like WCAG2AA or WCAG2A and work your way up. This setting is only used by htmlcs.

Running tests by passing a sitemap

If your site provides an XML sitemap, the sitemap URL can be directly passed to pa11y-ci:

pa11y-ci --sitemap https://mathiaspolligkeit.com/sitemap.xml

Keep in mind that if your sitemap has a large number of pages, the testing process might take some time. Once completed, you can review the generated report:

open pa11y-ci-report/index.html

Running tests by configuring the URLs

In case your site doesn’t publish an XML sitemap, you can specify a list of URLs in the .pa11yci configuration file:

{
  "defaults": {
    "reporters": ["pa11y-ci-reporter-html"],
    "runners": ["axe", "htmlcs"],
    "includeNotices": false,
    "includeWarnings": true,
    "standard": "WCAG2AAA"
  },
  "urls": [
    "https://www.yourwebsite.com",
    "http://localhost:1313//accessibility-testing-with-pa11y-ci"
  ]
}

After adding the URLs, run the following command:

pa11y-ci

Resources

Pa11y CI offers various advanced configurations, including setting up actions to be performed before the tests, such as logging into a webpage. Any configuration option listed in the pa11y readme can be added to the defaults object in the config.

For more information, refer to:

• github.com/pa11y/pa11y
• github.com/pa11y/pa11y-ci
• pa11y.org