Test Design
Aroha answered your questions. The contribution rates are 3, 4, 6, 8 and 10%, a failed save keeps the old rate, and the flow must be accessible. Now you turn those clarified requirements into test cases — the ones you will execute next week.
1 The Hook
Your clarifying questions from Week 1 paid off. Aroha confirmed the contribution rates a Tūāpapa member can choose are 3%, 4%, 6%, 8% and 10%, that a failed save to the fund administration system keeps the old rate and shows an error, and that the whole flow must be operable by keyboard and screen reader. The developers are building it now.
A new tester on the team, keen to get going, writes their test cases like this: “Test 1: change rate to 3% — works. Test 2: change rate to 4% — works. Test 3: change rate to 6% — works.” Forty test cases later, every one of them sets a valid rate and checks it saved. The suite is large, it looks thorough, and it would pass a build that crashes the moment a member tries to type 5% or 100%.
The problem is not effort — it is design. Testing all five valid rates over and over tells you almost nothing new each time, while the dangerous behaviour lives in the inputs nobody tested: the invalid rate, the edge of the range, the failed save, the screen reader. Good test design is not about writing more tests. It is about choosing the few inputs that each tell you something different, so a small suite covers a large risk.
This week you learn the techniques that turn an infinite space of possible inputs into a tight, deliberate set of test cases: equivalence partitioning, boundary value analysis, and the negative and non-functional paths that catch the defects valid inputs never will.
2 The Rule
You can never test everything, so do not try. Group inputs that behave the same into partitions, test one representative of each plus the boundaries between them, and always test the invalid and failure paths. A small set of well-chosen test cases finds more defects than a large set of repetitive ones.
3 The Analogy
A WoF inspector, not a panel-beater.
When your car goes in for a Warrant of Fitness, the inspector does not drive it 10,000 km to prove the brakes work. They check the few things that actually tell them about safety — brake performance, tyre tread depth at the legal limit, lights, the edge cases where a car fails. They have a finite checklist that covers the real risks, chosen so each check tells them something the others do not.
That is test design. You are not trying every possible input — you are choosing the handful that sit at the boundaries and in each distinct partition, the ones where failure actually hides. A tester who runs the same valid input forty times is the inspector driving laps of the car park instead of testing the brakes at the legal limit.
4 This Week’s Brief
Here are the clarified acceptance criteria for the story you are designing tests against this week. These are the answers Aroha gave to your Week 1 questions.
• A member may set their rate to one of: 3%, 4%, 6%, 8%, 10%. No other value is valid.
• The selector must reject any value outside that set; a free-typed value like 5% or 100% must not be accepted.
• On a valid change, the portal shows a confirmation naming the new rate and effective date, and sends the new rate to the fund administration system.
• If the fund administration call fails, the member sees an error and the previous rate is retained.
• Every change is written to the audit log: member ID, old rate, new rate, timestamp.
• The flow must be operable by keyboard and announced by a screen reader (NZ Government Web Accessibility Standard).
This is one requirement with five valid inputs, a large space of invalid ones, an integration that can fail, an audit obligation, and an accessibility rule. Your job: cover all of that without writing a hundred cases.
5 Equivalence Partitioning
An equivalence partition is a group of inputs the system should treat the same way. If 3% works, you have good reason to believe 4% works the same way — they are in the same partition (“valid rate”). Testing every member of a partition is wasteful; testing one representative is enough.
For TUA-103 the input space splits into clear partitions:
- Valid rates — {3, 4, 6, 8, 10}. One representative tells you the happy path works; you do not need all five for the same check (though boundaries differ, below).
- Invalid in-range values — {5, 7, 9, 2, 1} — numbers that look plausible but are not allowed. The system must reject them.
- Out-of-range values — {0, 100, −5} — clearly outside any sensible range.
- Non-numeric input — letters, symbols, blank — if free entry is possible at all.
Each partition is a different behaviour, so each earns at least one test case. The mistake the new tester made was testing the “valid rates” partition five times and the other three partitions zero times.
6 Boundary Value Analysis
Defects cluster at boundaries. Off-by-one errors, the wrong comparison operator, an inclusive range coded as exclusive — these all live at the edge of a partition, not in the middle. So once you have partitions, you test the values right at and just past each boundary.
TUA-103’s valid set is not a continuous range, so its boundaries are the lowest and highest valid values and the values immediately around them:
• 2% — just below lowest valid (must be rejected)
• 10% — highest valid rate (must be accepted)
• 11% — just above highest valid (must be rejected)
• 5% — an in-range gap value (must be rejected — proves the set is discrete, not a range)
That last one matters. A developer who implemented “between 3 and 10” instead of “one of 3, 4, 6, 8, 10” would accept 5% and 7%. Only a boundary/gap test catches that, and it is exactly the defect the Week 1 clarification was guarding against.
7 Negative and Non-Functional Paths
Partitions and boundaries cover the input field. They do not cover what happens when the system around it misbehaves, or the qualities the requirement demands. Those need their own deliberate cases.
- Integration failure: the fund administration call fails mid-save. Expected: error shown, old rate retained, nothing half-written. This is a requirement, so it is a test case.
- Audit trail: after a successful change, the audit log holds member ID, old rate, new rate and timestamp. Invisible to the member, mandatory for the FMA, easy to forget.
- Accessibility: the whole flow works by keyboard alone, and a screen reader announces the rate selector, the confirmation, and any error. Required by the NZ Government Web Accessibility Standard.
- Concurrency/state: what if the member opens the page, the rate is changed elsewhere, and they submit a stale value? Worth a case for anything touching a member’s money.
A suite that tests only valid and invalid rates passes a build that silently fails to write the audit log or is unusable with a screen reader. The negative and non-functional cases are where the regulator’s interest lives.
8 The Anatomy of a Test Case
A test case a teammate can run without asking you anything has a fixed shape: an ID, a clear title, any preconditions, numbered steps, the exact test data, and a specific expected result. “Check the rate change works” is not a test case — it is a wish.
Title: Reject in-range but disallowed contribution rate (5%)
Requirement: TUA-103
Priority: High
Precondition: Member “Test01” logged in via RealMe; current rate 3%.
Test data: Attempted rate = 5%
Steps: 1. Open Contributions page.
2. Attempt to set the rate to 5% (select or type).
3. Attempt to confirm.
Expected: 5% cannot be selected or submitted; a clear message states the
allowed rates; the member’s rate remains 3%; no call is sent to
the fund administration system.
Notice: a specific data value (5%), a precondition that sets the starting state (current rate 3%), steps anyone can follow, and an expected result precise enough that pass or fail is never a judgement call. That precision is what lets you hand the case to anyone — including yourself in Week 3 — and get a reliable result.
9 Common Mistakes
🚫 Writing many cases that all test the same partition
Why it happens: Adding another valid input feels like more coverage, and it grows the case count quickly.
The fix: Inputs in the same partition behave the same, so the extra cases add effort without adding information. Cover each partition once, then spend your effort on boundaries and invalid inputs — that is where defects live.
🚫 Only testing valid inputs
Why it happens: Requirements describe what should happen, so the natural cases are the ones that work.
The fix: For every valid partition there is at least one invalid one. List them on purpose — the disallowed rate, the out-of-range value, the blank field — because a build that handles valid input perfectly can still crash on the first bad one.
🚫 Vague expected results
Why it happens: “It works” or “the change saves” feels obvious when you wrote the case.
The fix: A vague expected result turns pass/fail into an opinion. State exactly what should be visible, stored, and sent — the confirmation text, the retained rate on failure, the audit-log entry — so anyone running the case reaches the same verdict.
🚫 Forgetting the non-functional cases
Why it happens: Accessibility, the audit log and integration failure are not in the visible UI, so they slip off the list.
The fix: These are real requirements with real regulators behind them. Add explicit cases for the audit trail, the failed-save behaviour, and keyboard/screen-reader operability — they fail far more often than the happy path.
10 Now You Try
Three graded exercises designing tests for the Tūāpapa portal. Write your answer, run it for feedback from your test lead, then compare to the model answer.
A teammate’s test set for TUA-103 is below. Identify 3 coverage gaps — partitions or behaviours the set fails to test — and name the technique that would close each.
TC-1: set rate to 3% — expect saved.
TC-2: set rate to 4% — expect saved.
TC-3: set rate to 6% — expect saved.
TC-4: set rate to 8% — expect saved.
TC-5: set rate to 10% — expect saved.
List 3 coverage gaps and the technique for each:
Show model answer
The set tests one partition (valid rates) five times and nothing else. Any three of these gaps earn full marks. 1. Invalid in-range values untested — 5%, 7%, 9% are inside the 3–10 range but not allowed. A developer who coded "between 3 and 10" would accept them. Closed by: equivalence partitioning (the "invalid in-range" partition) plus a boundary/gap test. 2. Boundaries untested — nothing checks 2% (just below the lowest) or 11% (just above the highest). Off-by-one and wrong-operator defects live here. Closed by: boundary value analysis. 3. Failure and non-functional behaviour untested — no case for the fund administration call failing (error shown, old rate kept), no audit-log check, no keyboard/screen-reader check. Closed by: negative testing and non-functional (accessibility) test cases. Bonus: every expected result is just "saved" — too vague to give a reliable pass/fail. The trap: five green test cases on a build that accepts 5% and never writes the audit log. Volume of cases is not coverage.
The test case below is too vague to run reliably. Rewrite it as a complete test case with these fields: Test ID, Title, Requirement, Priority, Precondition, Test data, Steps, Expected result. It should test the failed-save behaviour for TUA-103.
“Test that if the system is down the rate change handles it properly.”
Rewrite as a complete test case:
Show model answer
Test ID: TC-103-12 Title: Rate change is rejected gracefully when the fund administration system is unavailable Requirement: TUA-103 Priority: High Precondition: Member "Test01" logged in via RealMe; current rate 4%; the fund administration system stubbed/forced to return an error on save. Test data: New rate = 8% Steps: 1. Open the Contributions page. 2. Select the new rate 8% and confirm. 3. Observe the response while the fund administration call fails. Expected result: The member sees a clear error message that the change could not be saved and to try again; the displayed rate remains 4%; no partial/half-written change is committed; the failed attempt is logged for support; no success confirmation is shown. What makes this complete: a precondition that forces the failure, a specific data value, runnable steps, and an expected result precise on all four points (error shown, old rate retained, nothing half-written, no false success). The original named neither the data, the starting state, nor what "handles it properly" means.
Design a test suite of at least 6 test cases that gives strong coverage of TUA-103. Each case needs at least an ID, what it tests, the test data, and the expected result. Cover valid rates, invalid in-range, boundaries, integration failure, audit log, and accessibility.
Show model answer
TC-01 | Tests: valid rate accepted (valid partition) | Data: 6% | Expected: rate saved; confirmation names 6% and effective date; sent to fund administration system TC-02 | Tests: lowest valid boundary | Data: 3% | Expected: accepted and saved TC-03 | Tests: highest valid boundary | Data: 10% | Expected: accepted and saved TC-04 | Tests: invalid in-range value (discrete-set gap) | Data: 5% | Expected: rejected; allowed rates shown; rate unchanged; no call sent TC-05 | Tests: just outside range boundary | Data: 11% | Expected: rejected; rate unchanged TC-06 | Tests: integration failure | Data: change 4% to 8% with fund administration system forced to fail | Expected: error shown; old rate (4%) retained; nothing half-written TC-07 | Tests: audit log written | Data: change 3% to 8% successfully | Expected: audit log holds member ID, old rate 3%, new rate 8%, timestamp TC-08 | Tests: accessibility | Data: keyboard only + screen reader | Expected: selector, confirmation and errors are reachable by keyboard and announced by the screen reader A strong suite covers every partition once, hits the boundaries (3, 10, 11, plus the in-range gap 5%), and includes the failure, audit and accessibility cases. A weak suite lists valid rates over and over — that is the difference being marked.
11 Self-Check
Click each question to reveal the answer.
Q1: What is an equivalence partition, and why does it save you work?
A group of inputs the system should treat the same way. If one member of the partition works, the others very likely do too — so you test one representative instead of all of them. It lets a small suite cover a large input space without wasting cases on inputs that behave identically.
Q2: Why do you test at boundaries specifically?
Because defects cluster at the edges of partitions — off-by-one errors, the wrong comparison operator, an inclusive range coded as exclusive. The middle of a partition rarely surprises you; the value right at and just past the boundary is where the code makes its riskiest decision.
Q3: TUA-103’s valid rates are 3, 4, 6, 8 and 10%. Why is 5% an important test value?
Because it is in range but disallowed. A developer who coded “between 3 and 10” instead of “one of {3,4,6,8,10}” would wrongly accept 5%. Only a gap/boundary test catches that — and it is exactly the defect the Week 1 clarification was guarding against.
Q4: Name two non-functional or negative cases TUA-103 needs that valid-input tests would miss.
Any two of: the fund administration call failing (error shown, old rate retained, nothing half-written); the audit log being written with member ID, old/new rate and timestamp; and the flow being operable by keyboard and announced by a screen reader. These fail more often than the happy path and are where the FMA’s interest lies.
Q5: What turns “check the rate change works” into a runnable test case?
A fixed structure: an ID, a clear title, preconditions that set the starting state, specific test data, numbered steps anyone can follow, and an expected result precise enough that pass or fail is never a judgement call. Without those, two testers could reach different verdicts from the same words.
12 Interview Prep
Real questions asked in NZ QA interviews. Read the model answers, then practise your own version.
“You can’t test every possible input. How do you decide what to test?”
I split the input space into equivalence partitions — groups that should behave the same — and test one representative of each, then add boundary values at the edges of those partitions, because that is where off-by-one and wrong-operator defects live. I make sure to list the invalid partitions on purpose, not just the valid ones, and I add explicit negative and non-functional cases for failure paths, audit and accessibility. The goal is a small set where every case tells me something different, rather than a large set that repeats the same check.
“Walk me through designing tests for a field that accepts a fixed set of values.”
Take the KiwiSaver contribution rate — valid values are 3, 4, 6, 8 and 10%. The valid set is one partition, so I test a representative plus the lowest and highest values, 3% and 10%. Then the invalid partitions: an in-range gap like 5% to prove it is a discrete set and not a range, a just-outside value like 11%, and out-of-range or non-numeric input if free entry is possible. Then the behaviours around the field — a failed save keeping the old rate, the audit log, and keyboard/screen-reader operability. That is far fewer cases than testing every number, but it covers the real risk.
“What makes a test case good enough to hand to someone else?”
Someone who has never seen the feature should get the same result I would. That needs a clear title, preconditions that set the exact starting state, specific test data rather than “some value”, numbered steps, and an expected result precise enough that pass or fail is not a judgement call — the exact confirmation text, the retained rate on failure, the audit entry. If the expected result is “it works”, two testers can disagree, and a flaky verdict is worse than no test.