Automation Architect Answer Key

Expected pass output

Running 2 tests using 2 workers

  ✓ [staging] › staging.spec.ts:4:7 › staging smoke › landing page renders the hero (1.3s)
  ✓ [staging] › staging.spec.ts:10:7 › staging smoke › docs link navigates to docs root (1.7s)

  2 passed (3.2s)

Running TARGET_ENV=staging npx playwright test should produce exactly staging's tests. TARGET_ENV=dev or prod should swap to the other project's specs and URLs without any code edits.

Challenge solution — 1: mobile emulation project

Append to projects in playwright.config.ts:

{
  name: 'mobile',
  use: {
    ...devices['iPhone 14'],
    baseURL: envs.staging.baseURL,
  },
  timeout: envs.staging.timeout,
  retries: envs.staging.retries,
  testMatch: /staging\..*\.spec\.ts/,
},

Run with:

npx playwright test --project=mobile

Open the HTML report. Each test's screenshot should show a ~390px-wide viewport, not desktop. Confirm by looking at the Settings panel in the trace viewer — user-agent should be Mobile Safari, viewport should read 390×844.

Watch for: devices['iPhone 14'] is WebKit. If WebKit didn't install, mobile tests error with "Executable doesn't exist". Run npx playwright install webkit. testMatch here reuses the staging specs but constrains the project to only that file set — otherwise the mobile project would try running dev and prod specs too.

Challenge solution — 2: per-env credentials via dotenv

Install dotenv: npm install -D dotenv.

Create three files (all in .gitignore):

# .env.dev
USERNAME=
PASSWORD=
API_KEY=dev-key-abc123

# .env.staging
USERNAME=
PASSWORD=
API_KEY=stg-key-def456

# .env.prod
USERNAME=standard_user
PASSWORD=secret_sauce
API_KEY=prod-key-xyz789

Update playwright.config.ts:

import { defineConfig, devices } from '@playwright/test';
import * as dotenv from 'dotenv';
import * as path from 'path';

const TARGET_ENV = process.env.TARGET_ENV ?? 'staging';

// Load the env-specific file; default fallback lets CI override with real secrets.
dotenv.config({ path: path.resolve(__dirname, `.env.${TARGET_ENV}`) });

const envs = {
  dev:     { baseURL: 'https://example.com',         timeout: 10_000, retries: 0 },
  staging: { baseURL: 'https://playwright.dev',      timeout: 20_000, retries: 1 },
  prod:    { baseURL: 'https://www.saucedemo.com',   timeout: 30_000, retries: 2 },
} as const;

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  reporter: [['list'], ['html', { open: 'never' }]],
  projects: [
    {
      name: TARGET_ENV,
      use: {
        ...devices['Desktop Chrome'],
        baseURL: envs[TARGET_ENV].baseURL,
        extraHTTPHeaders: {
          'x-api-key': process.env.API_KEY ?? '',
        },
      },
      timeout: envs[TARGET_ENV].timeout,
      retries: envs[TARGET_ENV].retries,
    },
  ],
});

In the prod spec, credentials come from env vars:

test('prod login', async ({ page }) => {
  await page.goto('/');
  await page.getByPlaceholder('Username').fill(process.env.USERNAME!);
  await page.getByPlaceholder('Password').fill(process.env.PASSWORD!);
  await page.getByRole('button', { name: 'Login' }).click();
  await expect(page).toHaveURL(/inventory/);
});

Verify:

TARGET_ENV=prod npx playwright test

Why this pattern beats per-env config files: credentials change more often than URLs. Developers can add a key to .env.* without touching playwright.config.ts (and a code reviewer can enforce "config files get reviewed, env files are gitignored"). In CI, no .env.* files exist — the CI platform injects env vars directly, and the dotenv call silently no-ops.

Sanity check: run git status after creating .env.prod. It should not appear. If it does, add .env.* to .gitignore and run git rm --cached .env.prod.

Common mistakes

• All projects run when you wanted one

  --project staging (space) is a path argument. --project=staging (equals) selects the project. Or use TARGET_ENV=staging with the .filter(p => !TARGET_ENV || p.name === TARGET_ENV) trick from the practice.

• Specs cross-contaminating between projects

  Without testMatch or a folder-per-project layout, every test runs in every project. That's sometimes what you want (cross-browser smoke), but if your dev spec deletes a local fixture, running it in prod is a bad day. Structure tests/dev/*.spec.ts, tests/staging/*.spec.ts, etc., and set testMatch per project.

• Credentials in playwright.config.ts

  Hardcoded strings in config files get committed. Env vars + dotenv is the standard.

• Forgetting .env.* in .gitignore

  "I'll remember not to commit it" survives exactly one hasty git add .. Codify it.

• Prod retries set to 2+ without a "why should I retry?" rule

  Against real prod, a failing test might mean a real incident. Auto-retrying 2 times masks a 66% failure rate. For prod smoke tests, retry network-level errors (timeouts, 5xx), but fail fast on assertion mismatches. That requires a custom retry predicate — typically implemented in a fixture, not the config.

Expected pass output

Playwright (GitHub Actions):

✓ monitor-chromium › monitor.spec.ts:4:7 › playwright.dev uptime monitor › home page loads (2.1s)
✓ monitor-chromium › monitor.spec.ts:12:7 › playwright.dev uptime monitor › docs navigation still works (1.8s)

  2 passed (4.2s)

Newman (local cron):

iterations        1         0
requests          2         0
assertions        7         0
[20260422T101500Z] run complete, exit=0

Check the Actions tab after a scheduled window passes — you should see runs fire on the :00 and :30 marks. For Newman, reports/monitor.log grows a new line every 15 minutes once the cron is active.

Challenge solution — 1: Slack notifications on failure

Step 1 — create the webhook. Slack → your channel → Settings → Integrations → Add Incoming Webhook. Copy the URL (format: https://hooks.slack.com/services/T.../B.../...).

Step 2 — store it as a repo secret. GitHub → Settings → Secrets and variables → Actions → New repository secret. Name: SLACK_WEBHOOK. Paste the URL.

Step 3 — append a failure step to monitor.yml:

      - name: Notify Slack on failure
        if: failure()
        uses: slackapi/slack-github-action@v1.27.0
        with:
          payload: |
            {
              "text": ":red_circle: Playwright monitor failed",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "*Playwright monitor failed* on `${{ github.ref_name }}`\n<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View run>"
                  }
                }
              ]
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

Verify: temporarily break an assertion in monitor.spec.ts (toHaveTitle(/WillNeverMatch/)), push, wait for the next scheduled run. Slack should ping. Revert.

Lead the charge on noise management: one failed monitor → one ping. A cascade of transient failures during a playwright.dev outage → one ping, not 48. Add dedup logic — compare to the previous run's status via the GitHub API, only ping on red-from-green transitions. That's architect-level monitoring, not just "I hooked up a webhook."

Challenge solution — 2: multi-OS matrix

Convert the single job to a matrix:

jobs:
  playwright:
    timeout-minutes: 10
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci || npm install
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test --reporter=list,html
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report-${{ matrix.os }}-${{ github.run_id }}
          path: playwright-report/
          retention-days: 14

Push. Actions tab shows three parallel jobs per scheduled trigger. Three artifacts upload per run.

Is this worth the minutes cost? On free tier: yes, public repo = unlimited; private repo = 3× the minutes. For a monitor that runs 48 times a day × 3 OSes × ~4 minutes each = 576 minutes/day just for monitoring. That's 17k/month — past the free tier. Architects justify this with "how often does playwright.dev break only on Windows?" (Rarely.) Usually, ubuntu-latest only for monitors; matrix OS for the CI test suite.

The right tradeoff in one sentence: monitor on the OS your users run; CI-test on every OS you support.

Common mistakes

• Cron fires but the log stays empty on Linux

  cron has a minimal PATH. newman lives in /usr/local/bin (via nvm) or /usr/bin (via apt) — neither is in the default cron PATH. Fix: absolute path in the script (/usr/local/bin/newman), or PATH=/usr/local/bin:/usr/bin:/bin at the top of crontab.

• Windows Task Scheduler exits 0x1 with no output

  PowerShell's execution policy blocked the script. Task action arguments must include -ExecutionPolicy Bypass -File "C:\path\to\script.ps1". Or sign your script, but nobody does that for in-house monitors.

• GitHub scheduled workflow never fires even though it's green on push

  Three common causes: default branch isn't main; repo has had zero activity for 60+ days (GitHub disables inactive crons); cron syntax is wrong. crontab.guru is your friend. Also: scheduled workflows run against the default branch only — editing the workflow on a feature branch won't affect the schedule until merged.

• Treating a monitor like a CI test

  CI tests should be deterministic and fast. Monitors tolerate transient failures (network blips, CDN restarts), retry generously, and alert on persistent failure. Don't copy-paste your CI config as a monitor — you'll page someone on every random timeout.

• Secrets in the YAML

  SLACK_WEBHOOK: https://hooks.slack.com/... in monitor.yml is a public URL that can post to your channel. Use secrets.SLACK_WEBHOOK. Always.

Expected pass output

Running 2 tests using 2 workers

  ✓ [chromium] › smoke.spec.ts:3:1 › playwright.dev home page loads (1.9s)
  ✓ [chromium] › smoke.spec.ts:9:1 › jsonplaceholder API returns a known post (0.4s)

  2 passed (3.0s)

Inside the container, same as outside. docker compose run --rm tests or docker run --rm --ipc=host resync-playwright-architect:latest should both produce this.

Challenge solution — 1: push to ghcr.io

# Tag with your GitHub username
docker tag resync-playwright-architect:latest \
  ghcr.io/YOUR-USERNAME/resync-playwright-architect:latest

# Create a classic PAT with write:packages scope at
# https://github.com/settings/tokens

# Log in
echo $GITHUB_PAT | docker login ghcr.io -u YOUR-USERNAME --password-stdin

# Push
docker push ghcr.io/YOUR-USERNAME/resync-playwright-architect:latest

# Verify it's pullable on a different machine
docker system prune -af
docker pull ghcr.io/YOUR-USERNAME/resync-playwright-architect:latest
docker run --rm --ipc=host ghcr.io/YOUR-USERNAME/resync-playwright-architect:latest

Default visibility is private. Go to https://github.com/users/YOUR-USERNAME/packages/container/resync-playwright-architect/settings, scroll to Danger Zone, change visibility to Public if you want anyone to pull without auth.

Why ghcr.io over Docker Hub: free unlimited public images, no pull-rate limit for authenticated users, and it lives alongside your code — repo permissions carry over. Docker Hub's pull limit (100/6h anonymous) will throttle a busy CI pipeline without warning.

Challenge solution — 2: multi-stage build

Rewrite Dockerfile:

# -------- builder stage: install node deps in a slim image --------
FROM node:20-slim AS builder

WORKDIR /build

COPY package.json package-lock.json ./
RUN npm ci --no-audit --no-fund --production=false

# -------- runtime stage: playwright image with only what we need --------
FROM mcr.microsoft.com/playwright:v1.49.0-jammy

WORKDIR /work

# Pull in just the installed dependencies, not the builder's toolchain
COPY --from=builder /build/node_modules ./node_modules

COPY package.json package-lock.json ./
COPY playwright.config.ts ./
COPY tests ./tests

RUN chown -R pwuser:pwuser /work
USER pwuser

ENV CI=1

CMD ["npx", "playwright", "test", "--reporter=list,html"]

Compare image sizes:

docker build -t resync-playwright-arch:single -f Dockerfile.single .
docker build -t resync-playwright-arch:multi  -f Dockerfile.multi  .

docker images resync-playwright-arch

REPOSITORY                TAG      SIZE
resync-playwright-arch    single   1.85GB
resync-playwright-arch    multi    1.52GB

About 300 MB saved. Where the saving actually comes from: the single-stage image carries npm's build toolchain (gcc, make, python3 — pulled in as transitive deps for native modules). The multi-stage build leaves that behind in the builder.

Be honest about the ceiling: the Playwright runtime image is 1.3 GB of its own. Chromium + Firefox + WebKit + system libs. You can't multi-stage your way past that — the browsers have to be in the final image. If you want truly small test images, use mcr.microsoft.com/playwright:v1.49.0-jammy-arm64 (smaller on Apple Silicon) or switch to a chromium-only base like mcr.microsoft.com/playwright/chromium:... (if it exists at your version — Microsoft doesn't ship per-browser variants consistently).

Common mistakes

• "browser disconnected" / SIGBUS mid-test

  Chromium's default /dev/shm inside a container is 64MB. Real Chromium wants 2GB+ for its render process. Fix: docker run --ipc=host (shares host's /dev/shm) or --shm-size=2gb. The docker-compose template uses ipc: host; don't drop it.

• Playwright version mismatch between package.json and image tag

  package.json pins @playwright/test@1.49.0 but the Dockerfile uses :v1.48.0-jammy. The browsers in the image are 1.48; the test runner expects 1.49. Symptoms are subtle: flaky click events, locator behaviour mismatch. Pin both sides together, and when you upgrade the lib, also bump the tag.

• Cold cache on every build — takes 4 minutes to rebuild after a spec change

  You're copying source before npm deps. Docker layer cache is invalidated whenever any earlier COPY changes. Correct order: package manifests first, npm ci, then source. The deps layer stays cached until package.json changes. A good .dockerignore (excluding node_modules, playwright-report, .git) prevents bust-on-noise.

• Permission denied writing the HTML report

  Host directory is owned by root; container runs as pwuser (UID 1000). The bind mount preserves host ownership. Fix on the host: mkdir -p playwright-report && sudo chown 1000:1000 playwright-report. Or, pragmatically, let the container write and chown back to you as a post-step.

• Building for x86 on Apple Silicon, then failing to run on CI's x86 runner

  Apple Silicon builds produce linux/arm64 by default; GitHub's runners are linux/amd64. Use docker buildx build --platform linux/amd64 on your M-series Mac, or build in CI. This one bites architects who forgot their laptop is ARM.