Checklist-Based Testing
Structured lists of what to verify — built from experience, standards, and known failure modes. More flexible than formal test cases; more structured than ad hoc testing.
1 The Hook
A Dunedin team does an accessibility pass on a new council form before launch. Their best tester, who knows accessibility cold, runs it from memory. Looks great — ships. The next sprint a different person does the next form, also from memory, and forgets to check keyboard focus order. A screen-reader user files a complaint: they can’t tab through the form in a sensible order, so they can’t complete it.
The knowledge existed. It just lived in one person’s head and walked out the door the moment someone else did the work. There was nothing written down that said “check tab order follows the visual layout” — so on the form that mattered, nobody did. The fix wasn’t a smarter tester; it was a one-page list that every tester runs every time.
That list is checklist-based testing. It captures hard-won experience as a set of things to verify, so quality doesn’t depend on who happens to be testing today or what they remember under pressure.
2 The Rule
Capture what must be verified as a list of conditions — the what, not the how — so every tester checks the same things every time. It’s more flexible than scripted test cases, more reliable than testing from memory, and it must be kept alive as the product changes.
3 The Analogy
A surgical team’s pre-op checklist.
Before an operation at Auckland Hospital, the team runs a short checklist out loud: right patient, right site, allergies known, instrument count correct. These are experienced surgeons and nurses — they know all of this. The checklist doesn’t tell them how to operate; it just makes sure no critical item is skipped because someone was busy, tired, or assumed a colleague had it covered. It is the floor, not the ceiling, of safe practice.
A test checklist works the same way. It doesn’t script the tester’s every move — it names the conditions that must be verified so none slips through. Judgement and skill still do the work; the list just guarantees nothing important is forgotten.
The failure mode nobody warns you about is tick-box theatre. I have inherited checklists on government projects where every item was marked passed, yet the live site had obvious failures — broken tab order, missing error messages, the lot. When I dug in, testers were ticking items they didn't understand because the checklist gave no context and no one ever asked questions. A vague item like "accessibility checked" gets ticked in three seconds under sprint pressure. The fix isn't a longer checklist — it's a shorter one where every item is specific enough that a tester who hasn't seen the feature before can still make an unambiguous call. Ruthlessly cut anything that can't be verified in under sixty seconds or that requires insider knowledge to interpret. Ten items your team can genuinely verify beats thirty items that get rubber-stamped.
What it is
A checklist is a list of conditions, questions, or items to verify. Unlike formal test cases, checklists don’t specify exact steps or expected results — they specify what to check, leaving the tester to determine how.
This flexibility makes checklists valuable when testing requires judgment (accessibility, UX quality, cross-browser compatibility) or when the exact steps can’t be scripted in advance.
Example checklists
| # | Check |
|---|---|
| 1 | Required fields marked with visible indicator |
| 2 | Required fields validated on submit — not mid-typing |
| 3 | Error messages are specific ("Enter a valid email address") not generic ("Invalid input") |
| 4 | Valid data accepted, form submits successfully |
| 5 | Field length limits enforced and communicated to user |
| 6 | Tab order follows visual layout, logical sequence |
| 7 | All labels correctly associated with their inputs |
| 8 | Successful submission provides clear confirmation |
| # | Check |
|---|---|
| 1 | Layout intact on Chrome, Firefox, Safari, Edge |
| 2 | No horizontal scroll on mobile (320px – 768px) |
| 3 | Touch targets ≥ 44px on mobile |
| 4 | Images load and aren’t stretched or cropped unexpectedly |
| 5 | Fonts render correctly — no invisible or fallback text |
| 6 | Navigation works at all breakpoints |
Maintaining checklists
A checklist is a living document. Every time you find a bug that wasn’t on the list, add it. When an item becomes irrelevant or is now covered by automation, remove it. Review and prune checklists at least once per release cycle.
At senior level, you should own and maintain checklists for your area. At lead level, you’ll have team-wide checklists that become the baseline quality standard.
Checklists vs formal test cases
- Checklists are faster to create and maintain. Good for experienced testers who know how to check the thing.
- Formal test cases are more precise. Required when tests will be re-run by different people, when compliance evidence is needed, or when automated.
- Many teams use checklists for exploratory and regression testing, and formal test cases for critical paths and audit trails.
ISTQB mapping
| Ref | Topic |
|---|---|
| 4.4.2 | Checklist-Based Testing — high-level conditions, structured but flexible |
| 4.4.2 | Checklists derived from experience, standards, requirements, and defect data |
Practice this technique: Try Grad Practice 07 — Team & about page, Senior Practice 01 — Accessibility audit.
4 Industry Reality
- Checklists are often created once and never touched again. A form-validation checklist written in 2022 still gets run in 2026 on a completely different tech stack. No one added items for the new file-upload widget; no one removed the IE11 entry. The most valuable skill is not writing a checklist but keeping it alive — and that requires discipline most teams don’t enforce.
- Requirements don’t always tell you what the checklist should contain. In practice you build checklists by reading the specs, then cross-referencing past defect logs, then asking a senior tester what keeps biting them. A requirement says “user can reset password”; it does not say “check that the reset link expires after 24 hours and can only be used once”. That knowledge lives in previous incidents, not the spec.
- Time pressure compresses checklists in the wrong direction. Under sprint pressure, teams cut items they assume are fine rather than items they actually covered elsewhere. The checkbox that gets skipped is usually the one that catches the bug. Senior testers resist this by flagging which items are genuinely low-risk vs. which ones feel low-risk because they haven’t bitten anyone recently.
- Checklists blur into scripted test cases on compliance projects. On NZ government projects subject to the Digital Service Design Standards or the Privacy Act 2020, stakeholders start adding “expected result” columns and specific step sequences to checklist items. That’s fine — the hybrid provides both the flexibility of a checklist and the auditability of a formal test case — but recognise when you’ve crossed the line and name it appropriately in your test plan.
- The same checklist gets used on features it was never designed for. A cross-browser checklist written for a static marketing site gets applied to a React SPA with dynamic routing. Some items don’t translate; some critical SPA-specific checks (e.g. back-button behaviour, shallow vs. full page load) are missing entirely. A good tester adapts the checklist to the feature, not the other way around.
5 When to Use It — and When Not To
✓ Use it when
- Testing requires tester judgement and the “how” varies — accessibility reviews, UX checks, cross-browser passes
- The same area is tested repeatedly across releases and you need consistent coverage without scripting every run
- You are doing regression testing on stable functionality and want a fast, reliable baseline
- Knowledge lives in one or two people’s heads and you need to externalise it before that person leaves or changes team
- You want to codify past defects so the same category of bug can never slip through again
✗ Skip it when
- The test must be reproducible step-for-step by someone who has never seen the feature — use a formal test case instead
- Compliance or audit evidence is required: a tick-box checklist does not satisfy the traceability demands of the NZ Privacy Act 2020, PCI-DSS, or health data standards
- The feature is so novel there is no existing knowledge to capture — use exploratory testing first, then distil the learnings into a checklist
- The area will be fully covered by automated tests — a manual checklist for something CI already checks every commit is wasted effort
- The team is too junior to interpret open-ended items correctly — vague checklist items become inconsistent coverage; add enough context to make each item unambiguous or write formal test cases
Context guide
How the right level of checklist-based testing effort changes based on project context.
| Context | Priority | Why |
|---|---|---|
| Revenue NZ myIR portal — KiwiSaver, income tax, student loan features with Privacy Act 2020 obligations | Essential | High-volume citizen use, financial accuracy requirements, and audit obligations under the Tax Administration Act demand checklists seeded from past defect logs. NZ-specific items (NZD formatting, BB-bbbb-AAAAAAA-SS bank account structure, macron names) will never appear on a generic overseas checklist. |
| CoverNZ online claims — injury notification and medical certificate upload flows | Essential | Claimants are often injured and under stress — error messages must be specific and actionable, file-upload validation must be unambiguous, and session-timeout behaviour must be explicitly checked. A checklist externalises this knowledge so every tester covers it regardless of CoverNZ domain familiarity. |
| Harbour Bank / Pacific Bank — internet banking feature releases (payments, transfers, alerts) | High | Checklists capture recurring payment edge cases (duplicate submissions, session token expiry, NZD rounding) without over-scripting every regression run. Pair with formal test cases only for the critical payment-processing paths that require RBNZ-traceable evidence. |
| HealthNZ / HealthNZ — patient-facing appointment and referral portals | High | Health data under the Health Information Privacy Code 2020 demands consistent coverage across every release. Checklists are the practical tool for ensuring te reo Māori name support, WCAG 2.1 AA accessibility, and mobile usability are checked every sprint without scripting every scenario from scratch. |
| Pacific Air / TeleNZ — high-traffic e-commerce flows (booking, plan changes, top-ups) | Medium | Much of the core flow is automated in CI. Use checklists for the judgment-intensive layers automated tests miss: cross-browser rendering, error-message quality, and mobile usability at NZ-common screen sizes. Don’t duplicate coverage that already exists in the regression suite. |
| Internal tooling — staff-only admin dashboards at Benefits NZ, TransitNZ, or similar agencies | Low | Low public exposure and trained users reduce the risk of missed checklist items causing widespread harm. A lightweight checklist covering critical data-integrity and permission checks is usually sufficient; invest the saved effort in exploratory testing of novel business logic instead. |
Trade-offs
What you gain and what you give up when you choose checklist-based testing.
| Advantage | Disadvantage | Use instead when… |
|---|---|---|
| Externalises hard-won knowledge so quality is independent of who is testing today — any tester can pick up the list and cover the same ground as the domain expert. | Only covers known risks. A fully-ticked checklist says nothing about bugs the author never anticipated — novel failure modes remain invisible until an exploratory session or a production incident surfaces them. | The feature is genuinely new with no prior failure history — use exploratory testing first, then distil learnings into a checklist once patterns emerge. |
| Faster to create and maintain than scripted test cases — writing conditions rather than step sequences reduces the authoring burden and keeps the artefact lightweight enough that teams actually maintain it. | Not reproducible step-for-step across testers. Two testers running the same checklist may interpret items differently, making defect reproduction inconsistent — a problem on NZ government projects where audit evidence must be traceable. | Compliance, PCI-DSS, or Privacy Act 2020 audit evidence is required — switch to formal test cases so each check is traceable with exact steps and expected results. |
| Preserves tester judgement — the list specifies what to check, not how, so an experienced tester can apply context-sensitive skill rather than following a script that may not match the current build state. | Relies on tester skill to interpret open-ended items correctly. A vague item like “accessibility checked” is ticked in seconds by an under-informed tester; the checklist provides false assurance rather than real coverage. | The team is predominantly junior or new to the domain — write formal test cases with explicit steps so each check is unambiguous, and evolve toward checklists as team knowledge matures. |
| Accumulates institutional memory — every defect found gets added as a new item, so the checklist grows smarter after each release and past failure modes can never silently recur. | Decays without active maintenance. Stale items (IE11 rendering, removed fields, superseded payment APIs) erode tester trust until the whole list is treated as optional — at which point it offers no protection at all. | The area is fully covered by an automated regression suite on every commit — a manual checklist duplicating CI checks is wasted effort; redirect that time to exploratory testing of uncovered risk areas. |
Enterprise reality
At 200–300-developer scale, checklist-based testing stops being a team habit and becomes an auditable compliance artefact — the difference between a QA practice and a quality management system.
- Checklist authoring and review become separate roles: platform engineers generate checklists from infrastructure templates (Terraform modules, GitHub Actions workflows) rather than testers writing them by hand — a small team can't afford this overhead, but TechServNZ and TeleNZ run dedicated test architecture teams whose entire job is maintaining the canonical checklist library.
- Governance and legislation make checklists non-negotiable: organisations under the Privacy Act 2020, NZISM (NZ Information Security Manual), PCI DSS, or HISF (Health Information Security Framework) must demonstrate evidence of testing against specific controls — a checklist tick with a tester's name and timestamp is the audit trail; skipping it is a compliance failure, not a process shortcut.
- Tooling shifts from spreadsheets to integrated platforms: Xray for Jira, TestRail, or Zephyr Scale replace shared Excel files once a programme hits ~5 squads, because you need traceability from requirement to checklist item to test run result to release sign-off in one linked chain — not a folder of CSVs.
- Cross-squad coordination requires a checklist federation model: at 10+ squad scale (common at Harbour Bank, Revenue NZ, or HealthNZ programme level) each squad owns a domain checklist, but a release train lead aggregates a "programme-level" checklist that gates the combined release — without this, squads independently sign off components that break each other at integration.
◆ What I would do
Professional judgement — when to reach for checklist-based testing, when to skip it, and what to watch for.
The bottom line: A checklist is only trustworthy when it is short enough to run completely, specific enough that any tester can judge each item unambiguously, and recent enough that its last review date is still within the current product era. If it fails any of those three tests, fix the checklist before you trust the coverage it claims to provide.
6 Best Practices
- ✓ Write items as conditions, not steps. “Error messages are specific, not generic” is a checklist item. “Type invalid data, click Submit, read the error” is a test step. If you catch yourself writing keystrokes, stop and extract the condition.
- ✓ Make every item objectively tickable. Two testers who have never spoken should be able to independently run the checklist and agree on pass/fail for every item. If they wouldn’t, the item is too vague. Sharpen it.
- ✓ Seed new checklists from defect history. Before writing from scratch, pull the last 12 months of bugs filed against this area. Every distinct failure mode should be a checklist item. This is faster, more accurate, and immediately useful.
- ✓ Review and prune every release cycle. Set a calendar reminder. Remove items now covered by automation, items no longer relevant to the current tech stack, and IE11 entries (yes, they’re still there in every checklist you inherit). Add items for anything new.
- ✓ Tag items by risk level. Mark items as P1/P2/P3 or Critical/High/Medium so that when time is short, testers know which items are the floor they must not skip and which ones they can defer. A checklist with no priorities is a checklist that gets cut randomly under pressure.
- ✓ Separate “always run” from “run on change”. A full regression checklist and a change-specific checklist are different documents. Using the full list for every small fix wastes time; using the small list for a major refactor misses coverage. Keep them distinct.
- ✓ Store checklists in version control alongside the code. A checklist in a shared drive gets out of sync. Markdown checklists in the repo are reviewed in pull requests, updated when the feature changes, and have an audit trail. NZ government teams in particular benefit from this traceability.
- ✓ Run exploratory testing alongside the checklist, not instead of it. The checklist is the floor — it covers known risks. Exploratory testing uncovers the unknowns. Senior testers do both: structured pass first, then 15–30 minutes of unscripted exploration to find what the checklist doesn’t anticipate.
- ✓ Capture what the checklist misses. When you find a bug during exploratory testing that the checklist didn’t cover, add it before you close the session. The checklist should grow smarter after every test run.
- ✓ NZ-specific: include te reo and macron handling as standard items. Any checklist for a customer-facing NZ application should include “names and place names with macrons (e.g. Tāmaki Makaurau, Pōneke) display correctly throughout”. This is a genuine defect vector that generic checklists always omit.
7 Common Misconceptions
❌ Myth: A checklist is just a dumbed-down test case for junior testers.
Reality: Checklist-based testing is an experience-based technique (ISTQB 4.4.2) — it is harder to do well than scripted testing, not easier. Writing a checklist requires knowing which conditions matter, which failure modes to anticipate, and how to phrase items so any tester can judge them. Junior testers can execute a good checklist; it takes a senior tester to write and maintain one. The surgical pre-op checklist is used by experienced surgeons, not because it’s simple, but because it makes expert practice reliable.
❌ Myth: Once a checklist is written, it’s done.
Reality: A static checklist is a decaying asset. Every release that changes the product without updating the checklist is a release where the checklist becomes less accurate. Items go stale (IE11), get automated, become irrelevant, or miss new failure modes. ISTQB explicitly calls checklists “living documents” for this reason. If you haven’t reviewed your checklist in the last release cycle, you are running coverage theatre — the appearance of rigour without the substance.
❌ Myth: Checking every item on the checklist means you’ve done thorough testing.
Reality: A checklist only covers known risks and failure modes. Ticking every box means you haven’t missed the things you knew to check — it says nothing about what you didn’t anticipate. Experienced testers treat the checklist as the structured baseline, then follow it with exploratory testing to probe the unknowns. Over-reliance on a checklist is one of the main reasons testers miss novel bugs: the checklist confirms the expected, and exploration finds the unexpected.
8 Now You Try
Three graded exercises — spot, fix, then build. Write your answer, run it for AI feedback, then compare to the model answer.
Below are six lines someone added to a “login form” checklist. A good checklist item names what to verify and is judgeable; a bad one is either a scripted step, too vague to judge, or not a check at all. Mark each line good or bad and say why.
(b) Click the email box, type test@example.com, press Tab, type a password, click Submit
(c) The form works
(d) Error messages are specific, not generic
(e) Password field masks input by default
(f) Make it good
Show model answer
(a) GOOD — names a verifiable condition (visible required-field indicator); leaves the "how" to the tester. (b) BAD — that is a scripted test case (exact steps). A checklist names what to check, not the keystrokes; this belongs in a formal test case. (c) BAD — too vague to judge. "Works" how? Can’t be ticked objectively. (d) GOOD — a clear, judgeable condition (specific vs generic error messages). (e) GOOD — specific and verifiable (password masked by default). (f) BAD — not a check at all; meaningless. The rule: a good checklist item is the WHAT, is specific enough that two testers would agree whether it passed, and leaves the HOW to the tester’s judgement. Scripted steps and vague slogans both fail that test.
A team’s cross-browser checklist below has gone stale. One item is now handled by automation, one is no longer relevant, one is too vague, and a known bug type is missing. Rewrite the checklist: prune, sharpen, and add the missing item.
1. Layout intact on Chrome, Firefox, Safari, Edge
2. Page renders correctly in Internet Explorer 11
3. Unit tests pass in CI (now run automatically on every commit)
4. Looks fine on mobile
5. Fonts render correctly
Rewrite the checklist:
Show model answer
Revised cross-browser / responsive checklist: 1. Layout intact on current Chrome, Firefox, Safari, and Edge. 2. No horizontal scroll at mobile widths (320px–768px). 3. Touch targets are at least 44px on mobile. 4. Fonts render correctly — no invisible or fallback-only text. 5. Images load and aren’t stretched or cropped unexpectedly. What I removed: - IE 11 rendering — no longer relevant; IE is end-of-life and not in the supported browser set. - "Unit tests pass in CI" — now handled by automation on every commit, so it doesn’t belong on a manual checklist. What I sharpened / added: - "Looks fine on mobile" was too vague to tick — replaced with judgeable items: no horizontal scroll in a stated range, and a 44px touch-target minimum. - Added "images load and aren’t distorted" — a common cross-browser/responsive bug the original list missed. The point: a checklist is a living document. Prune what automation now covers or what’s irrelevant, sharpen anything you can’t objectively tick, and add new failure modes as you find them.
Build a checklist of 6–8 items for verifying a NZ government online payment form (e.g. paying a fine or a council rates bill). Make each item specific and judgeable, and include at least two NZ-specific conditions a generic checklist would miss.
Show model answer
NZ government payment-form checklist (example): 1. Required fields are marked and validated on submit, not mid-typing. 2. Amount is shown in NZD with a $ symbol and exactly two decimal places. 3. Error messages are specific enough for a user to self-correct (e.g. "Enter a 4-digit postcode"). 4. The form is keyboard-navigable and tab order follows the visual layout (accessibility). 5. NZ bank account in BB-bbbb-AAAAAAA-SS format is accepted; an invalid structure is rejected with a clear message. 6. A confirmation/receipt is shown after a successful payment, with a reference number. 7. The form meets the relevant NZ Digital Service Design / accessibility expectations (e.g. WCAG). 8. Content is available in te reo Māori where required, and names with macrons (e.g. Māori) display correctly. NZ-specific items: the NZD formatting, the BB-bbbb-AAAAAAA-SS bank-account format, macron/te reo handling, and alignment to NZ government service standards — all things a generic, overseas checklist would miss and that cause real failures here. The point: a good checklist names verifiable conditions (the what), stays judgeable, and is tailored to the actual context — including local rules a borrowed checklist wouldn’t contain.
Why teams fail here
- Writing items as steps instead of conditions — “click Submit and read the message” instead of “error messages are specific and actionable” — turns a checklist into a poor man’s test case that gives you the worst of both: no repeatability, no flexibility.
- Never reviewing or pruning: stale items (IE11, removed fields, old API versions) erode trust until testers start skipping items they assume are irrelevant — and those are exactly the items that catch bugs.
- No priority labels: under sprint pressure every team cuts checklist items, but without P1/P2/P3 markers they cut randomly rather than cutting the genuinely low-risk items first, and the critical checks are the ones that disappear.
- Treating a ticked checklist as evidence of thorough testing: a checklist only covers known risks. A fully-ticked checklist with no accompanying exploratory testing means you verified everything you expected and nothing you didn’t — which is precisely where novel bugs live.
Key takeaway
A checklist is only as reliable as its last review date — write it from defect history, own it by name, and prune it every release cycle or it becomes a false sense of coverage dressed up as rigour.
How this has changed
The field moved. Here is how Checklist-Based Testing evolved from its origins to current practice.
Aviation and military checklists predate software — the concept that experienced practitioners need reminders comes from high-stakes domains where memory errors kill people. Software testing inherits this discipline, initially for configuration and release verification.
Software testing checklists proliferate in waterfall organisations as a documentation artefact. Pre-release checklists, regression checklists, and configuration verification checklists are standard deliverables. Checkbox compliance becomes a goal rather than a means.
Agile Manifesto pushes back on documentation overhead. Many teams abandon checklists as "waterfall waste." The useful discipline of structured reminders is lost along with the bureaucratic overhead of maintaining thousand-line checklist documents.
The testing community finds the middle ground. Lightweight heuristic-based checklists (OWASP, WCAG, browser compatibility grids) prove their value. Test session charters in exploratory testing function as structured checklists without bureaucratic overhead.
AI tools can generate contextual checklists from requirements and system descriptions — reducing the burden of maintenance. Compliance-driven domains (health, finance, government) use checklists as audit evidence. The skill is knowing which checklist to use and when a checkbox hides risk rather than eliminates it.
Self-Check
Click each question to reveal the answer.
Interview Questions
What NZ hiring managers ask about Checklist-Based Testing — and what strong answers look like.
What makes a testing checklist effective, and what makes it become a liability?
Strong answer: An effective checklist captures non-obvious items that experienced testers would check but might forget — OWASP security categories, WCAG criteria, cross-browser rendering gotchas. It is short enough to use without taking longer than the thing it is checking. It is reviewed and updated after each release. A checklist becomes a liability when it grows to hundreds of items checked mechanically without thinking, when items are ticked without being genuinely verified, or when it gives false confidence — a passed checklist does not mean a tested system.
Junior/Mid
How do you prevent checklist compliance from replacing actual test thinking?
Strong answer: By positioning the checklist as a reminder, not a substitute for exploration. After completing the checklist, I ask: "What is this checklist not covering?" Every system has unique risks that a generic checklist misses. I combine the checklist with exploratory sessions focused on areas of highest risk. I also review the checklist after each bug found in production — if the bug was in the checklist domain but was missed, either the checklist item is too vague or the verification step was ineffective.
Mid/Senior
Q1: What does a checklist specify that a formal test case does not?
A checklist specifies what to verify — the conditions or questions — and leaves how to the tester. A formal test case specifies the exact steps and expected results. That makes checklists faster and more flexible, and formal test cases more precise and reproducible.
Q2: Why does the council-form story show the value of a checklist over testing from memory?
Because knowledge in one person’s head leaves with them, and even experts forget items under pressure. A written checklist that every tester runs every time makes quality independent of who is testing today and what they happen to recall — nothing critical gets skipped.
Q3: What makes a checklist item good rather than bad?
It names a specific, verifiable condition (the what) that two testers would agree on, and leaves the how to judgement. Scripted keystrokes belong in a formal test case, and vague items like “the form works” can’t be objectively ticked — both are poor checklist items.
Q4: Why is a checklist called a living document, and how do you maintain it?
Because the product and risks change. You add any new bug type you find, remove items now covered by automation or no longer relevant, and sharpen anything too vague to tick. Review and prune at least once per release cycle so it stays accurate and useful.
Q5: When would you choose a formal test case over a checklist?
When the test must be re-run identically by different people, when compliance or audit evidence is needed, or when it will be automated — for example a payment-gateway charge that must be traceable to NZ standards. Judgement-based checks by an experienced tester are faster as a checklist.
Q6: Your team is testing the new KiwiSaver voluntary contribution form on the Revenue NZ myIR portal. There is no existing checklist. How do you decide what goes on it, and in what order do you do the work?
A: Start by pulling defect history for related Revenue NZ forms — past failures become your first items. Cross-reference the product spec for stated rules (e.g. contribution limits, NZD formatting, NZ bank-account structure in BB-bbbb-AAAAAAA-SS format), then add NZ-specific conditions a generic checklist would miss: macron handling in names, RealMe identity-verification step, Privacy Act 2020 data-minimisation items, and WCAG 2.1 AA accessibility. Risk-rank everything so the P1 items (payment accuracy, identity, session timeout) are at the top. Run the checklist first, then follow with exploratory testing to catch what the list doesn't anticipate.
Q7: What is the key difference between checklist-based testing and exploratory testing, and when should you use both together?
A: Checklist-based testing is structured and covers known risks — it guarantees you check the same things every time, but only the things you already know to check. Exploratory testing is unscripted and targets unknown risks — it finds novel bugs the checklist never anticipated, but can miss known failure modes if time runs short. Use them together on the same feature: run the checklist first to nail the known risks, then spend 15–30 minutes in unscripted exploration to probe the edges. The checklist is the floor; exploration is what raises the ceiling. This combination is especially common on NZ government portals, where known compliance items (WCAG, Privacy Act) are handled by the checklist and exploratory sessions find the edge cases in business logic.
Q8: A developer tells you: "We don't need a checklist — we have unit tests in CI that run on every commit, so all those checks are automated." What is wrong with this reasoning and how do you respond?
A: Unit tests verify isolated code logic; checklists cover end-to-end user-facing conditions that unit tests rarely reach — things like "error messages are specific and helpful," "tab order follows the visual layout," "the page works on Safari mobile," or "te reo Maori macrons display correctly." CI catching a null-pointer exception is not the same as a tester verifying that a confused Benefits NZ beneficiary can self-correct from an error message. A good response: agree to remove any checklist item that is demonstrably covered by automation (and remove it — don't leave dead weight), but keep items that require judgement, cross-browser observation, or user-experience assessment. The checklist and the test suite cover different risk surfaces.
Q9: You inherit a checklist for an CoverNZ online claims form. It was last updated in 2021, still contains "renders correctly in Internet Explorer 11," and has no priority labels. Before the next sprint, what three actions do you take and why?
A: First, prune stale and redundant items: remove the IE 11 entry (end-of-life, not in the supported browser set) and any items now fully covered by automated regression. Second, add missing risk items by pulling defect logs from the past 12 months — every distinct bug category that isn't on the list gets added. For an CoverNZ form, add macron and te reo handling, file-upload size and type validation, and session-timeout behaviour for users who leave the form mid-completion. Third, tag every remaining item with a risk priority (P1/P2/P3) so that when sprint time is short, testers know which items are the non-negotiable floor and which can be deferred. A checklist without priorities gets cut randomly under pressure, which is the worst possible outcome.
Try It — Checklist or formal test case?
For each testing scenario below, decide whether a checklist or a formal test case is the more appropriate tool.
| Scenario | Your choice |
|---|---|
| Verifying that a payment gateway integration charges the correct amount to the cent — evidence required for financial audit | |
| Quickly checking that a refreshed marketing landing page looks right across 4 browsers before go-live | |
| Confirming a bug fix for a specific edge case — needs to be re-run by another tester in a different timezone | |
| General accessibility review of a new feature — checking WCAG criteria with judgment calls required | |
| Testing a government portal's identity verification flow that must comply with NZ Digital Service Design Standards |
Answers
| Scenario | Best tool | Why |
|---|---|---|
| Payment gateway audit | Formal test case | Exact steps, precise expected results, traceable audit evidence required. |
| Marketing page cross-browser check | Checklist | Experienced tester knows how to check; judgment-based; no exact steps needed. |
| Bug fix re-run by different tester | Formal test case | Reproducible across testers and time zones requires precise steps. |
| Accessibility review | Checklist | WCAG criteria map naturally to checklist items; requires tester judgment throughout. |
| Government identity verification | Formal test case | Compliance requirement — traceability to NZ standards and audit trail needed. |
The key decision factor: does someone else need to replicate this exactly, or produce audit evidence? If yes, formal test case. If it’s a judgment-based check by an experienced tester, a checklist is faster and just as effective.
Senior engineer insight
The checklist only has value if the tester actually trusts it — and testers stop trusting checklists the moment they notice items that haven’t been updated since the last tech stack migration. I changed how I think about checklist authorship when I realised that the maintainer matters more than the original author: a checklist written by an expert in 2021 and never touched since is now dead weight wearing a safety harness. The most effective checklists I’ve used on NZ government projects are short, owned by a named person, and have a “last reviewed” date on them — those three things alone tell you whether to trust it before you even read item one.
Most common mistake: treating the checklist as done once it’s written. Teams run stale checklists on changed systems and call it regression testing. The checklist says nothing failed because it never checks the things that changed.
From the field
On a release gate for a NZ Ministry of Education online enrolment form, we had a four-page checklist that had been copy-pasted between projects for two years. Nobody had noticed that three items still referenced a field that was removed in a redesign eight months earlier, and the entire accessibility section was a single line: “accessibility OK.” We found out on go-live week when a screen-reader user reported they couldn’t complete the form — keyboard focus was jumping into a hidden element on page three. The checklist had been “passed” on every release. After that incident we introduced a mandatory annual checklist review as part of our onboarding process for new QA staff: every new tester ran the checklist for real and flagged anything they couldn’t objectively verify. Within two reviews we’d halved the list and doubled its reliability. The lesson that generalises: your onboarding QA is your best checklist auditor, because they can’t coast on familiarity the way incumbents can.