Specialised · Senior & Automation

GitHub Actions for QA

GitHub Actions is the CI/CD platform most NZ startups and product teams run. This lesson teaches you to build proper QA quality gates: matrix strategies across browsers and environments, test result annotations, branch protection rules, and reusable workflow patterns.

Senior Automation Engineer ISTQB CT-TAS ~16 min read

1 The Hook

A Wellington fintech startup launches a KiwiSaver contribution management tool. The team of six runs Playwright tests locally before every PR merge. Tests pass. Reviews happen. Code ships. Three hours after a Friday release, the on-call developer gets paged: the contribution form is broken for all users on Chrome Linux — which is what every production server runs.

The QA engineer ran tests on Chrome Mac. Everything passed there. The failing combination — Chrome on Linux with a specific timezone offset required for NZ Daylight Saving Time calculations — was never in the test matrix. Nobody noticed because tests ran on developer laptops, not in a consistent, reproducible environment.

The post-mortem produced one recommendation: "Tests must run in CI on every PR before merge, using a matrix that covers production environment configurations." The fix took one afternoon. Not running it for six months cost one very bad Friday.

2 The Rule

"A test that runs on a developer's laptop but not in every PR is a test that will silently miss the environment difference that causes your next production incident."

The laptop is not the production environment. It never has the same OS, the same browser version, the same timezone, the same environment variable set, or the same network configuration. CI is not a luxury — it is the only reproducible testing environment your team has. Quality gates in GitHub Actions block bad code before it merges, not after it ships.

3 The Analogy

Analogy

A GitHub Actions quality gate is an airport security checkpoint for your codebase.

Airports do not let passengers board because they seem fine. Every passenger goes through the same scanner, regardless of their history or how trustworthy they look. GitHub Actions applies the same scrutiny to every PR: type checks, unit tests, integration tests, browser matrix, security scan — all mandatory, all automated. The gate blocks the code from boarding (merging) until every check passes. One exception kills the model: the moment you allow a PR to merge with a failing gate "just this once," the airport starts letting people skip the scanner.

4 Workflow Anatomy

A GitHub Actions workflow is a YAML file in .github/workflows/. Three concepts control everything: triggers, jobs, and steps.

# .github/workflows/qa.yml
name: QA Gate

on:                          # TRIGGER: when does this run?
  pull_request:
    branches: [ main, develop ]
  push:
    branches: [ main ]

jobs:                         # JOBS: parallel units of work
  test:
    runs-on: ubuntu-latest    # always Linux in CI
    steps:                      # STEPS: sequential commands
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm test

Key concepts:

on: — the trigger. pull_request runs on every PR. push runs when commits land on a branch. schedule runs on a cron.
runs-on: — the runner environment. Always use ubuntu-latest for web tests (matches most NZ production environments). Use windows-latest or macos-latest only for platform-specific tests.
uses: — a reusable action from the GitHub marketplace. actions/checkout@v4 checks out your code. actions/setup-node@v4 installs Node.js.
run: — a shell command. npm ci installs exact versions from lock file (not npm install).

5 The QA Workflow Pattern

A complete QA gate for a NZ e-commerce or SaaS product follows this job structure:

# Full QA gate pattern
jobs:

  typecheck:                    # fast fail — 30 seconds
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: {node-version: '20', cache: 'npm'}
      - run: npm ci --ignore-scripts
      - run: npx tsc --noEmit

  unit-tests:                   # fast — 1-2 minutes
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: {node-version: '20', cache: 'npm'}
      - run: npm ci
      - run: npm run test:unit -- --reporter=junit --outputFile=results.xml
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: unit-results
          path: results.xml

  e2e-tests:                    # slower — uses matrix (see next section)
    needs: [ typecheck, unit-tests ]  # only run if fast jobs pass
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: {node-version: '20', cache: 'npm'}
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test
        env:
          BASE_URL: https://staging.mysite.co.nz

The needs: key creates a dependency. E2E tests are expensive — only run them after the cheap checks pass. This pattern cuts wasted CI minutes for NZ teams on paid GitHub plans.

6 Matrix Strategy for Browser and OS Coverage

A matrix runs the same job with different variable combinations. One workflow definition, multiple parallel runs.

# Browser matrix for a NZ retail site
e2e-matrix:
  runs-on: ubuntu-latest
  strategy:
    fail-fast: false          # don't cancel other browsers if one fails
    matrix:
      browser: [ chromium, firefox, webkit ]
      viewport: [ desktop, mobile ]
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with: {node-version: '20', cache: 'npm'}
    - run: npm ci
    - run: npx playwright install --with-deps ${{ matrix.browser }}
    - run: npx playwright test --project=${{ matrix.browser }}-${{ matrix.viewport }}
      env:
        TZ: Pacific/Auckland  # NZ timezone — catches DST bugs
    - uses: actions/upload-artifact@v4
      if: failure()
      with:
        name: traces-${{ matrix.browser }}-${{ matrix.viewport }}
        path: test-results/

This single job definition runs 6 parallel jobs (3 browsers × 2 viewports). Key settings:

fail-fast: false — without this, a Firefox failure cancels the Chrome run before you can see Chrome results. Always use false for diagnostic matrices.
TZ: Pacific/Auckland — sets the NZ timezone in the runner. Catches bugs in date formatting, daylight saving transitions (NZ switches in September and April), and time-relative features like booking cutoffs.
if: failure() — uploads Playwright trace files only when a test fails. Saves storage while still giving you full visual replay for debugging.

7 Test Result Publishing

Raw exit codes tell you "tests passed" or "tests failed" — nothing more. Test result publishing turns CI output into annotations directly on the PR, showing which test file and which assertion failed without leaving GitHub.

# Publish Playwright JUnit results as PR annotations
    - run: npx playwright test --reporter=junit,html
      continue-on-error: true  # let result step run even on failure

    - name: Publish test results
      uses: mikepenz/action-junit-report@v4
      if: always()
      with:
        report_paths: test-results/*.xml
        fail_on_failure: true

    - name: Upload HTML report
      uses: actions/upload-artifact@v4
      if: always()
      with:
        name: playwright-report-${{ matrix.browser }}
        path: playwright-report/
        retention-days: 30

The if: always() condition is critical: it ensures the upload step runs even when tests fail. Without it, test artefacts are never uploaded for failing PRs — the exact cases where you need them most.

Code coverage gating: Add jest --coverage --coverageThreshold='{"global":{"lines":80}}' to fail the job if line coverage drops below 80%. For NZ regulated teams (banking, health), 90%+ branch coverage for payment and data-handling paths is a common requirement.

8 Quality Gates and Branch Protection

CI jobs mean nothing if you can merge without them passing. Branch protection rules enforce quality gates at the repository level.

Setting up branch protection on main

In GitHub: Repository Settings → Branches → Add branch ruleset → Target: main

Require status checks to pass before merging — add each job name: typecheck, unit-tests, e2e-matrix (chromium, desktop), etc.
Require branches to be up to date — prevents a PR from merging if main has moved ahead. Ensures tests ran against the latest code.
Require pull request reviews — combined with CI gates, this is the two-factor approval that NZ enterprise teams and ISO 27001 auditors expect.
Do not allow bypassing — administrators can override by default. Disable this for regulated teams. No exceptions.

Gate type	Blocks merge?	Best for
TypeScript typecheck	Yes	All repos with TypeScript
Unit tests	Yes	All repos
E2E — critical path only	Yes	Payment, auth, core workflows
Full browser matrix	Warn only	Informational — unblock velocity
Security scan (CodeQL)	Yes for critical	Auth, payments, user data paths
Accessibility check (axe-core)	Yes for WCAG AA	Govt, public-facing services

NZ government note: DIA's Digital Service Design Standard mandates automated accessibility testing in CI for government services. Use axe-playwright or pa11y-ci as a required status check if you are building for a government client.

9 Reusable Workflows

When the same CI pattern appears in five different repositories, a bug in that pattern needs five PRs to fix. Reusable workflows define the CI logic once in a shared repo and call it from any other repo.

# In: org/.github/workflows/playwright-qa.yml
on:
  workflow_call:            # makes this workflow reusable
    inputs:
      base-url:
        type: string
        required: true
      browsers:
        type: string
        default: '["chromium"]'
    secrets:
      API_KEY:
        required: false

# In: any-team-repo/.github/workflows/qa.yml
jobs:
  run-qa:
    uses: myorg/.github/.github/workflows/playwright-qa.yml@main
    with:
      base-url: https://staging.kiwipay.co.nz
      browsers: '["chromium","webkit"]'
    secrets:
      API_KEY: ${{ secrets.STAGING_API_KEY }}

For NZ product companies with multiple services (common in fintech and healthtech), this pattern means the QA standards team owns CI quality in one place. When Playwright releases a new version or a security patch changes the scan config, one PR in the shared repo updates all teams simultaneously.

10 Common Mistakes

✗ Using npm install instead of npm ci in CI

npm install may update the lock file or resolve different versions than what's in it. npm ci installs exactly what's in package-lock.json and fails if there is a mismatch. CI should be deterministic: the same commit always produces the same result.

✗ Not setting TZ: Pacific/Auckland in runners

GitHub runners default to UTC. Many NZ applications have time-sensitive features: booking cutoffs, compliance deadlines, daylight saving transitions (September and April in NZ). Tests that pass in UTC will fail in NZST/NZDT if timezone is not explicitly set.

✗ Installing all Playwright browsers instead of only the ones you need

npx playwright install --with-deps installs all three browsers plus system dependencies — 500+ MB per run. Use npx playwright install --with-deps chromium for jobs that only test one browser. Use matrix to parallelise browser coverage, not a monolithic install.

✗ Required status checks that never run on the target branch

Branch protection requires a status check by name. If the workflow is not triggered on pull_request targeting main, the check never appears — and GitHub silently allows the merge. Always verify the trigger matches the branch protection target.

✗ Committing secrets as environment variables in YAML files

Never put API keys, passwords, or tokens directly in YAML. Use GitHub Secrets (secrets.MY_SECRET). For environment-specific credentials, use GitHub Environments with environment-scoped secrets — this also adds approval gates before a job can access production secrets.

11 Now You Try

Three AI-graded exercises. Apply GitHub Actions concepts to NZ team scenarios.

Exercise 1 — Write a QA workflow

Write a complete .github/workflows/qa.yml for a NZ e-commerce checkout page (built with Next.js and Playwright). Requirements:

Runs on every PR targeting main
TypeScript check first, unit tests second, E2E third (only if prior jobs pass)
E2E matrix: Chromium and WebKit, desktop and mobile viewports
NZ timezone set correctly
Playwright traces uploaded on failure

Assessed by a real LLM

Exercise 2 — Design quality gates for a payments team

You are the QA lead for KiwiPay, a NZ payment processing company. Determine which CI checks should block merge vs warn only for their main branch, and explain the reasoning for each decision.

Consider: TypeScript typecheck, unit tests (>85% pass), E2E critical-path (payment flow), full browser matrix (6 combinations), OWASP dependency scan, axe-core accessibility, performance budget (Lighthouse), visual regression tests.

Assessed by a real LLM

Exercise 3 — Debug a broken CI workflow

The following GitHub Actions workflow is failing intermittently on PRs. Identify all the bugs and explain how to fix each one.

name: Tests
on: push
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm install
      - run: npx playwright install
      - run: npx playwright test
      - uses: actions/upload-artifact@v4
        with:
          name: traces
          path: test-results/

Assessed by a real LLM

12 Self-Check

Click each question to reveal the answer.

What is the difference between npm install and npm ci, and which should you use in CI?

npm install may update the lock file and resolve different package versions. npm ci strictly installs what is in package-lock.json and fails if there is a mismatch. Always use npm ci in CI — it ensures the build is reproducible and deterministic across all runs.

Why does fail-fast: false matter in a browser matrix?

Without fail-fast: false, GitHub Actions cancels all remaining matrix jobs as soon as any one of them fails. If Firefox fails, the Chrome and WebKit results are never collected. Setting fail-fast: false lets all matrix jobs complete so you get a full picture of which browser-viewport combinations fail — essential diagnostic information for cross-browser bugs.

A required status check is configured in branch protection but it never appears as passing or failing on PRs. What is the most likely cause?

The workflow is not triggered on pull_request targeting the protected branch. Branch protection required status checks only block merge when a check with that exact name exists on the PR. If the workflow trigger doesn't match (e.g. it only triggers on push, not pull_request), the check never runs and GitHub silently allows the merge.

What is a reusable workflow, and when should a QA team use one?

A reusable workflow is a GitHub Actions workflow triggered by workflow_call instead of push/pull_request. It can be called from any other repository in the same organisation. QA teams should use reusable workflows when the same CI pattern (e.g. Playwright matrix, security scan config) appears across multiple repositories — centralising it means a single update fixes all repos simultaneously.

Why is TZ: Pacific/Auckland important for NZ test runners?

GitHub runners default to UTC. NZ applications often have time-sensitive features — booking cutoffs, tax deadlines, NZ Daylight Saving Time transitions in September and April. A test that passes in UTC may fail in NZST or NZDT because date calculations, displayed times, or time-based conditional logic differ. Setting TZ explicitly makes CI reproduce the environment your users are actually in.

13 ISTQB Mapping

ISTQB CT-TAS (Certified Tester — Test Automation Strategies)

Chapter 4 — Deployment: CI/CD pipeline integration, automated test triggering on PR/push, pipeline stages for testing
Chapter 5 — Reporting and Metrics: Test result publishing, coverage gates, quality metrics from CI

ISTQB CTAL-DevOps (Advanced DevOps Tester)

Chapter 3 — Continuous Integration: Test automation in CI, shift-left quality, pipeline-as-code
Chapter 4 — Continuous Delivery: Quality gates, environment management, deployment validation

ISTQB CTAL-TA (Advanced Test Analyst)

Environment and configuration testing, cross-browser and cross-platform strategies

← Jira Xray Next: Event-Driven Testing →