Mid-Level · Automation Engineer

Cross-Browser Automation

Playwright runs on Chromium, Firefox, and WebKit with one config file. That's the easy part. The hard part is designing a CI strategy that catches browser-specific bugs without drowning your pipeline in false positives.

Mid-Level ISTQB CTAL-TA v3.1.2 — K3 Apply ~14 min read + exercise

1 The Hook — Why This Matters

A Palmerston North SaaS company runs their Playwright suite on Chromium only. Tests pass 100% of the time in CI. The test lead is proud of that green dashboard.

A sales rep takes the product to a large enterprise customer and demos it on their corporate MacBook. The file upload component — built with a custom drag-and-drop implementation — does nothing. Completely broken in Safari. The drag events fire differently in WebKit. The engineering team later find the bug has been in production for six months.

The reasoning that led here: "our users are mostly on Chrome." The problem: the customer's IT policy mandates Safari for all web applications. Deal lost. The irony is that the fix was a six-line change to the event handler. The tests would have caught it in two minutes if anyone had run them on WebKit.

2 The Rule — The One-Sentence Version

Never assume Chromium represents all browsers. Safari (WebKit) has distinct rendering, JavaScript, and API behaviour that Chromium-only suites will never catch.

Browser engines are not interchangeable. Chromium and Firefox share a lot of ground, but WebKit — the engine powering Safari on Mac and iOS — diverges meaningfully on drag-and-drop, clipboard, file inputs, CSS features, and media APIs. Any app that may be used in Safari needs WebKit coverage in the test suite.

3 The Analogy — Think Of It Like...

Analogy

Cross-browser testing is like cutting a key and testing it in multiple locks.

Your test (the key) works perfectly in the Chromium lock. You feel confident. But the WebKit lock has a slightly different cylinder. It won't turn until you notice the subtle difference in the cut. You have to try all the locks to know the key actually works.

The Chromium-only team cut one key and declared success. The customer handed them a different lock.

Senior engineer insight

The most useful mental model I have for cross-browser automation is this: Chromium is the baseline, but WebKit is the deciding vote. In three years of running Playwright suites across NZ government and financial sector projects, the overwhelming majority of browser-specific defects showed up in WebKit — not Firefox. File inputs, drag events, clipboard access, and certain CSS pseudo-selectors all behave differently enough in Safari that a Chromium-passing test suite gives you a false sense of security about the user experience on any MacBook or iPhone.

Once I started tagging tests that touched those APIs with a @webkit-sensitive annotation and filtering them into a dedicated CI gate, the team stopped treating browser failures as noise and started treating them as signal.

The most common mistake: running WebKit in the same sequential job as Chromium and Firefox, then disabling it when pipeline time blows out — instead of parallelising across runners so each browser costs only as much as a single-browser run.

4 Watch Me Do It — Step by Step

Playwright's multi-browser support lives entirely in playwright.config.ts. One file, five project configs — desktop and mobile across all three engines.

TypeScript — playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 4 : undefined,

  projects: [
    // Desktop browsers
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
    // Mobile viewports
    {
      name: 'mobile-chrome',
      use: { ...devices['Pixel 7'] },
    },
    {
      name: 'mobile-safari',
      use: { ...devices['iPhone 14'] },
    },
  ],
});

Running all five projects locally is slow. In CI you want a matrix that parallelises across runners so each browser gets its own job. This GitHub Actions workflow does that:

YAML — .github/workflows/playwright.yml
name: Playwright Tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        browser: [chromium, firefox, webkit]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - run: npx playwright install --with-deps ${{ matrix.browser }}
      - run: npx playwright test --project=${{ matrix.browser }}
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: playwright-report-${{ matrix.browser }}
          path: playwright-report/

The matrix gives you three parallel jobs instead of one sequential job. Total CI time stays close to a single-browser run while giving you three-browser coverage.

PR vs nightly strategy: Run Chromium on every PR for fast feedback. Run all three browsers (including mobile) on a nightly schedule. Exception: if a PR touches drag-and-drop, clipboard, file inputs, or any custom event handler — trigger the full suite for that PR.

When Playwright WebKit emulation is not enough — for example, when you need to test on a real iPhone or verify macOS-specific font rendering — wire up BrowserStack. The integration is a one-line config change:

TypeScript — BrowserStack integration in playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  // Point Playwright at BrowserStack's remote WebDriver grid
  use: {
    connectOptions: {
      wsEndpoint: `wss://cdp.browserstack.com/playwright?caps=${
        encodeURIComponent(JSON.stringify({
          browser: 'safari',
          browser_version: 'latest',
          os: 'OS X',
          os_version: 'Ventura',
          'browserstack.username': process.env.BS_USERNAME,
          'browserstack.accessKey': process.env.BS_ACCESS_KEY,
        }))
      }`,
    },
  },
});

BrowserStack is paid but invaluable for validating on real Safari/iOS before a major release. Use it for smoke tests on real devices, not as a substitute for your local Playwright WebKit coverage.

From the field

A team building an online CoverNZ claim portal assumed their users were on Chrome because that was their internal browser. The assumption was reasonable — their analytics dashboard showed 78% Chrome traffic from internal testers. What nobody had checked was the claimant population: New Zealanders using iPads or iPhones at home, or older Macs running Safari as the default. The cross-browser suite was there; it just hadn't been wired into CI. Tests were run manually once a quarter.

Three months after launch, support tickets spiked around the document upload step. The file picker opened but the upload handler silently swallowed errors in WebKit due to a difference in how Safari resolves the FileReader API inside an async callback. The test suite would have caught it immediately. The fix took two hours; the support triage took two weeks and involved three teams.

What changed: the team moved WebKit tests into a nightly CI schedule as a non-negotiable gate, and added a label to any PR touching file handling that automatically triggered the full browser matrix. Nobody has to remember — the pipeline enforces it.

5 When to Use It — Decision Framework

ScenarioStrategy
Public-facing app, unknown user baseFull three-browser CI (at least nightly)
Government or enterprise customer in NZInclude WebKit — Safari is frequently mandated
App uses custom drag-and-drop, file input, or clipboardAll three browsers on every PR touching those components
Internal tool, Chrome mandated for all staffChromium-only is acceptable — document the decision
Mobile user base > 20%Add iPhone and Pixel mobile projects to the suite
Pre-release smoke on real deviceBrowserStack or Sauce Labs for real Safari/iOS

The decision to run Chromium-only should be explicit and documented, not a default that nobody questioned. Add a comment to playwright.config.ts if you deliberately restrict browsers.

6 Common Mistakes — Don't Do This

🚫 "Playwright WebKit is the same as testing on Safari"

I used to think: Playwright WebKit gives full Safari coverage. Actually: Playwright WebKit uses the WebKit engine but runs in a Linux environment. It catches most WebKit-specific JavaScript and CSS bugs, but not macOS-specific rendering differences or iOS gesture behaviour on real hardware. For complete Safari coverage on a real Mac or iPhone, use BrowserStack or a physical device.

🚫 Running all five browser configs on every PR

I used to think: more coverage on every PR is always better. Actually: running all five projects (Chromium, Firefox, WebKit, mobile Chrome, mobile Safari) on every PR can quadruple CI time and kill developer velocity. Run Chromium on every PR for fast feedback. Run the full suite nightly. Trigger the full suite on PRs only when browser-specific code is changed.

🚫 "If it passes Chromium, it's browser-independent"

I used to think: Chromium coverage means the feature works everywhere. Actually: custom event handling (drag events, touch events, clipboard, file input dialogs) behaves differently in Firefox and WebKit. Form validation messages render differently. Some CSS properties are unsupported or prefixed differently. Always test custom interactive components across all three engines before calling them done.

7 Now You Try — Prompt Lab

Write your answer in the box below and the AI coach will review it.

🧪 Exercise — Cross-Browser Test + CI Config

A Chorus NZ digital service portal uses a custom file upload component. Write a Playwright test that verifies the upload works correctly, then write the GitHub Actions workflow configuration that would run this test across Chromium, Firefox, and WebKit on every pull request.

Why teams fail here

  • Treating browser coverage as optional: Teams defer WebKit testing indefinitely because "our users are on Chrome," then discover the Revenue NZ e-filing portal or a council payments page is Safari-majority when the first support tickets arrive after go-live.
  • Running all browsers sequentially on every PR: Adding Chromium + Firefox + WebKit to a single sequential CI job triples pipeline time. Developers start skipping it or disabling failing browser projects rather than fixing root causes — the opposite of the safety net you wanted.
  • Conflating Playwright WebKit with real Safari: WebKit in Playwright runs on Linux and catches most engine-level bugs, but it does not replicate macOS GPU compositing, real iOS touch gestures, or Safari extension interference. Teams that skip BrowserStack for pre-release validation on high-traffic Safari flows learn this the hard way.
  • No annotation strategy for browser-sensitive tests: Without a tagging convention (e.g. @webkit-sensitive, project filters, or test.skip conditions), the whole suite runs on all browsers by default. This generates noise — tests failing for known, accepted reasons — and teams start ignoring red builds entirely.

Key takeaway

Chromium-only automation tells you your app works; WebKit coverage tells you your app ships.

8 Self-Check — Can You Actually Do This?

Click each question to reveal the answer.

Q1. What is the difference between Playwright WebKit and testing on a real Safari browser?

Playwright WebKit uses the open-source WebKit engine compiled for Linux. It catches the majority of WebKit-specific JavaScript, CSS, and API bugs — file input behaviour, drag events, flexbox quirks. What it does not catch: macOS-specific font rendering, GPU compositing differences, real iOS touch gesture handling, and Safari extensions. For those, you need BrowserStack or a physical Mac/iPhone. Playwright WebKit is a good first filter; real Safari is the definitive check before a major release.

Q2. When would you run full cross-browser CI on every PR rather than running it nightly only?

When the PR touches code that is known to behave differently across browser engines: custom drag-and-drop handlers, clipboard API calls, file input interactions, CSS features with inconsistent support (e.g. :has(), subgrid), or media APIs. For general feature work — button clicks, form submissions, navigation, API calls — Chromium on every PR plus nightly multi-browser is the right trade-off. Document the rule in your contributing guide so the team applies it consistently.

Q3. A test passes on Chromium and Firefox but fails on WebKit. What are the three most likely causes?

(1) Event handling — WebKit fires drag events, pointer events, or input events in a different order or with different properties to Chromium. Check the specific event type and its payload. (2) CSS or layout — WebKit may not support a CSS feature, or interprets a flexbox or grid spec edge case differently. Inspect the rendered layout in the WebKit trace. (3) Web API availability — Clipboard API, Web Animations, File System Access, and some Media APIs have different support or permission models in WebKit. Check MDN compatibility tables for the specific API the test relies on.

9 ISTQB Mapping

CTAL-TA v3.1.2 — Section 3.2.7: Test execution — browser and platform coverage. Cross-browser execution is a test design and scheduling concern for the Technical Test Analyst, covering selection of configurations, execution environments, and CI integration.

CTFL v4.0 — Section 4.1.1: Non-functional characteristics — compatibility testing. Browser compatibility is a non-functional quality attribute. Testers must identify coverage decisions (which browsers, which versions, which devices) and document the rationale for scope inclusions and exclusions.