Accessibility Testing Automation — Testing Techniques Library

What automated accessibility testing can catch

Automated accessibility testing uses rules engines to analyse page structure and DOM attributes, flagging violations against WCAG 2.1 criteria. Tools scan for mechanical issues that can be reliably detected by code:

Missing or empty alt text on images
Colour contrast failures (foreground/background ratio below 4.5:1 for normal text)
Missing labels on form inputs
Heading hierarchy violations (h1 jumps to h3, skipping h2)
Duplicate ID attributes on the page
ARIA misuse (invalid roles, incorrect attribute values)
Empty buttons or links (no accessible name)
Images used as buttons without alt text or ARIA labels
Form fields without visible labels
Page language not declared (lang attribute missing on <html>)

These are the low-hanging fruit: high-confidence violations that are straightforward to fix.

The automation limit: Automated tools catch about 30-40% of accessibility issues. The rest require human judgment. A page might have perfect ARIA structure but be unusable with a keyboard or screen reader. Automation finds the bugs code can detect; testing finds the bugs only humans can find.

Tools in the automated accessibility ecosystem

Tool	Format	Key strength	Best for
axe DevTools	Browser extension + API	Widely used, excellent explanations and remediation guidance, integrates into test frameworks	First line of automated testing; CI/CD pipelines; starting point for teams new to accessibility
Pa11y	CLI + API, Node.js	Scriptable, reports to JSON, easy to integrate into CI/CD, supports multiple reporters	CI/CD pipelines, bulk scanning many URLs, teams that want full automation control
WAVE	Browser extension, also standalone API	Visual overlays directly on the page, colour-coded icons for errors/warnings/structure	Manual review by testers, visual scanning, design team collaboration
Lighthouse	Built into Chrome DevTools, CLI, API	Bundled with Chrome, runs as part of broader performance/quality audit, free	Integrated into test suites, part of overall quality gates, CI/CD that needs accessibility + performance

Common issues automated tools find (and how to fix them)

Missing alt text

Issue: <img src="button.png"> with no alt attribute. Screen reader announces "image" with no context.

Fix: Add descriptive alt text: <img src="button.png" alt="Close dialog">. For decorative images, use empty alt: <img src="divider.png" alt="">.

Colour contrast failure

Issue: Grey text (#777) on light grey background (#EEEE) has a contrast ratio of 2.1:1. WCAG requires 4.5:1 for normal text.

Fix: Darken the text or lighten the background. The tool reports the current ratio and required ratio, so designers know exactly what to change.

Missing form label

Issue: <input type="email"> with no associated <label>. Screen reader doesn't announce what the field is for.

Fix: Add a label: <label for="email">Email</label><input id="email" type="email">. The for attribute must match the input's id.

Heading hierarchy broken

Issue: Page has h1, then h3, then h2. The outline is confusing to screen reader users navigating by heading.

Fix: Make headings descend in order: h1, h2, h3. If you need bigger text, use CSS, not heading tags.

ARIA misuse

Issue: <div role="button" aria-pressed="true"> but the element isn't actually focusable by keyboard or activatable by Enter/Space.

Fix: Use a real <button> tag. If you must use ARIA, ensure keyboard handling and focus management are implemented correctly. (Better: use native HTML.)

What automation can't catch

Keyboard navigation: An element might have perfect ARIA markup but not be reachable by Tab. Automation can't tell; manual testing can.
Screen reader behaviour: How does a screen reader actually announce this component? Automation doesn't run screen readers; only human testers using NVDA or JAWS can verify.
User experience for disabled users: Is the experience actually usable? Is the page understandable? These require human judgment and preferably testing with real users who have disabilities.
Context and meaning: An image of a chart with alt text "Chart" is technically labelled but not actually accessible. Only a human can judge if the alt text is descriptive enough.
Dynamic content updates: A loading spinner appears, then results load. Did the page announce the update to screen readers? Automation can check for aria-live regions but can't verify they actually work as intended.

Integration in CI/CD pipelines

Automated accessibility testing fits naturally into CI/CD. On every build:

Start a fresh instance of the application.
Run the automated accessibility scanner on critical pages.
Report violations (errors, warnings, or notices).
Fail the build if errors are found (optional: allow warnings to pass).
Archive the report as a build artifact.

Example with axe in Selenium/WebDriver tests

// JavaScript + WebDriver
const results = await runAccessibilityAudit(page);
if (results.violations.length > 0) {
  const errors = results.violations
    .filter(v => v.impact === 'critical' || v.impact === 'serious');
  if (errors.length > 0) {
    throw new Error(`Accessibility violations found:\n${errors.map(e => e.id).join(', ')}`);
  }
}

Example with Pa11y in CI/CD

// pa11y.json config
{
  "runners": ["axe"],
  "chromeLaunchConfig": {},
  "timeout": 10000,
  "wait": 1000
}

// In GitHub Actions
- name: Run Pa11y scan
  run: |
    pa11y http://localhost:3000 \
      --runner axe \
      --standard WCAG2AA \
      --json > a11y-report.json
- name: Check for violations
  run: |
    if grep -q '"level":"error"' a11y-report.json; then
      echo "Accessibility errors found"
      exit 1
    fi

Worked example: axe integration in an automated test suite

Scenario: You want every test to check for accessibility violations. If a functional test passes but accessibility fails, the test fails overall.

// Jest + Puppeteer + axe-core
const { axe, toHaveNoViolations } = require('jest-axe');

describe('Checkout flow', () => {
  it('submits order and is accessible', async () => {
    await page.goto('http://localhost:3000/checkout');

    // Perform the action
    await page.click('input[name="email"]');
    await page.type('input[name="email"]', 'test@example.com');
    await page.click('button[type="submit"]');

    // Wait for success state
    await page.waitForNavigation();

    // Check accessibility
    const results = await axe(page);
    expect(results).toHaveNoViolations();
  });
});

Manual testing as the complement

After automated tests pass, manual testing fills the gaps:

Tab through the page. Can you reach every interactive element with Tab? Is the focus order logical?
Use a screen reader. Open NVDA (Windows) or VoiceOver (macOS) and navigate the page. Does the structure make sense? Are buttons and form fields announced clearly?
Test at 200% zoom. WCAG requires content to remain usable at 200% zoom. Does the layout reflow correctly?
Test with browser zoom and Windows high contrast mode. Do colours remain readable? Do images still make sense?
Verify error handling. When you trigger an error (invalid email, missing field), is the error message clear and linked to the field?

WCAG 2.1 coverage: what automation addresses

Automated tools address a subset of WCAG 2.1 criteria. The breakdown varies by tool, but roughly:

Perceivable (images, colour, captions): ~60% coverage. Automation catches missing alt text and colour contrast. It doesn't verify if captions are accurate or if text is readable.
Operable (keyboard, timing, seizures): ~25% coverage. Automation flags missing focus indicators in code, but can't verify keyboard navigation works in practice.
Understandable (language, labels, errors): ~40% coverage. Automation checks for language tags and form labels, but can't judge if language is simple enough or if error messages are helpful.
Robust (HTML validity, ARIA correctness): ~85% coverage. Automation is excellent at catching malformed ARIA and invalid HTML.

This is why automation is layer one, not the whole solution. You need human testing to cover the gaps.

Best practices and anti-patterns

Automate the mechanical checks; test the rest manually. Don't wait for perfect automation coverage. Run axe or Pa11y on every build to catch regressions, then add manual accessibility testing at key points (new feature launch, design change, before release).

Fail the build on critical violations only. Many tools report errors, warnings, and notices. Set CI to fail on "error" or "serious" severity, but allow warnings to pass (and track them separately). This prevents false-positive build failures from minor issues.
Review violations with a purpose. Don't just auto-approve every fix. Have a designer or accessibility expert review significant changes to ensure fixes are actually accessible, not just compliant.
Test representative pages, not every page. Scanning 500 pages is expensive and noisy. Focus on critical pages (homepage, checkout, forms) and representative component instances (button, input, dropdown).
Pair automation with manual keyboard testing. Keyboard navigation is the most impactful manual test. Spend 10 minutes tabbing through every page and checking focus order. This catches more real-world issues than any automation.
Run automated checks early and often. Accessibility is cheaper to build in than to fix later. Run axe during development (IDE integration, local pre-commit hook) before code review.
Include accessibility in your definition of done. No feature is done if it doesn't pass automated scans and manual keyboard checks. Make it a checklist item.

Related techniques: Accessibility Testing (WCAG), Visual Regression Testing.

← Visual Regression Testing Next: Idempotency Testing →