Risk-Based Testing
Time is always limited. Risk-based testing uses risk exposure to decide what to test first, how deeply to test it, and what to deprioritise when time runs out.
1 The Hook
A Wellington council ships a new online rates portal two weeks before the annual rates instalment is due. The team had four weeks of test time and a backlog twice that size, so they tested in the order the tickets happened to be filed: help text first (easy wins), then the FAQ, then login, and they ran out of time before they got to the rate-calculation engine.
On go-live day the portal looked polished. Then ratepayers started phoning: the new calculation code applied last year’s rating valuation to this year’s charges, over-billing thousands of households. The most dangerous part of the system — brand-new code that touches every ratepayer’s money — got the least testing, because nobody had ranked the work by how much damage a failure would cause.
This is the pattern behind a huge share of public quality failures: limited test time spent evenly, or worse, spent on whatever was easiest, instead of on what would hurt most if it broke. The team had the hours to test the calculation engine — they just spent them on the FAQ.
Risk-based testing is the discipline of spending your limited test time proportional to risk exposure — likelihood of failure multiplied by impact if it fails — so that when time runs out, it is always the low-consequence work left untested, not the critical paths. Use it on virtually every real project: time is almost never sufficient to test everything, and risk-based testing gives you a defensible, documented rationale for every coverage decision. The most common mistake is scoring impact by how hard the code is to write rather than by the consequence of failure — a technically tricky animation has low impact; a simple tax-calculation function that every ratepayer relies on has high impact, regardless of how easy the code looks.
The most dangerous moment in risk-based testing is when the developer scores their own module's likelihood. I have seen this on government projects in Wellington where a developer rates freshly rewritten payment code as likelihood 2 because they are proud of it. That score then drives the test depth — and the module gets light coverage. What developers signal as low likelihood is often "I'm confident in this" not "the code has low complexity and stable history." Keep likelihood scoring separate: pull it from code metrics, commit frequency, integration points, and defect history. Never let the author of the module score its own likelihood. That is not distrust — it is the same reason you don't peer-review your own work.
2 The Rule
You never have time to test everything — so rank every area by risk exposure (likelihood × impact) and spend your test effort top-down, testing the highest-exposure areas first, deepest, and earliest.
Common Mistake vs What Works
Teams score each area on a likelihood × impact matrix, sort the backlog by exposure — and then test everything anyway when the sprint runs long. The risk register becomes a report artefact rather than a decision tool. This is exactly what happened on the Wellington council rates portal: someone knew the help text was low-risk, but tested it first because it was easy, and the rate-calculation engine got whatever time was left.
Build the risk register before test planning — not after — and use the scores to make explicit de-scope decisions upfront. When time runs out, the items with the lowest exposure scores are the ones not tested, not the items nobody got to. Write it into the test plan: “If we run out of time before Friday, loyalty points (exposure 4) are deferred and the residual risk is documented. Payment processing (exposure 20) is non-negotiable.” Risk-based testing only works if low-score items can actually be dropped; otherwise the register is decoration.
3 The Analogy
A WOF inspection at a VTNZ station.
A vehicle inspector has a fixed slot to check your car. They do not spend it polishing the wing-mirror glass and admiring the paint. They go straight for the things that kill people if they fail: brakes, steering, tyres, seatbelts. A cracked tail-light gets a note; worn brake pads stop the car passing. The inspector is ranking checks by consequence — how likely is this to fail, and how bad is it if it does — and spending the limited time where a miss matters most.
Risk-based testing is running your test plan like a WOF inspector runs the bay: brakes before bodywork. The high-exposure areas get the deep checks; the cosmetic ones get a glance, and if the clock runs out they are the things you can defend leaving untested.
What it is
Risk-based testing (RBT) is an approach to test planning where you use risk assessment to prioritise test activities. The areas with the highest risk get the most test effort; low-risk areas get less or none. This is how experienced leads make defensible decisions about coverage when there isn’t time to test everything.
Product risk vs project risk
- Product risk — risks that the software will fail to meet stakeholder needs. These become test targets. Examples: payment processing errors, data loss, security vulnerabilities.
- Project risk — risks that affect the testing process itself. Examples: late delivery of test environments, incomplete requirements, team capacity issues.
Risk-based testing primarily addresses product risk. Project risk is managed in the test plan.
Risk exposure = likelihood × impact
Risk exposure is the product of two factors:
- Likelihood — how likely is this to fail? (technical complexity, code churn, new developer, known defect history)
- Impact — what’s the consequence if it fails? (financial loss, user harm, reputational damage, regulatory breach)
| Feature area | Likelihood (1–5) | Impact (1–5) | Exposure | Priority |
|---|---|---|---|---|
| Payment processing | 3 | 5 | 15 | High |
| User authentication | 2 | 5 | 10 | High |
| Product search | 4 | 3 | 12 | Medium |
| Wishlist feature | 2 | 2 | 4 | Low |
| Email preferences | 1 | 1 | 1 | Very Low |
Building a risk register
- Identify risks — brainstorm with stakeholders, developers, and the team. Review defect history. Check industry-specific risks.
- Assess likelihood — consider: code complexity, number of changes, developer experience, module age, integration points.
- Assess impact — consider: user importance of the feature, business criticality, regulatory requirements, number of users affected.
- Calculate exposure — multiply the scores. Sort descending.
- Map to test effort — high risk = deep testing + automation + early testing. Low risk = light testing or none.
How risk drives test decisions
- Test order — test high-risk areas first so critical bugs are found early.
- Test depth — high-risk areas get full EP/BVA/decision table coverage. Low-risk areas may get smoke tests only.
- Automation priority — high-risk, frequently run paths should be automated first.
- Exit criteria — if time runs out, you can demonstrate that high-risk areas were fully tested and low-risk areas were consciously deprioritised.
ISTQB mapping
| Ref | Topic | Level |
|---|---|---|
| CTFL 5.2 | Risk management overview — product vs project risk | Foundation |
| CTAL-TM Ch. 2 | Risk-based testing — full framework, risk register, mitigation | Advanced / Lead |
| CTAL-TA 2.3 | Risk-based testing tasks for the Test Analyst | Advanced / Senior |
Practice this technique: Try Test Lead Practice 02 — Risk-based prioritisation, Senior Practice 09 — Cross-level regression.
NZ example — risk register for a NZ e-commerce platform
For a New Zealand e-commerce platform, a practical risk register might include:
- Payment processing via POLi (NZ bank-to-bank payment) — High likelihood if newly integrated, High impact (revenue loss, customer trust) = exposure 25, test first and deepest.
- NZ address validation (RD addresses, apartment formats) — Medium likelihood (complex input rules), High impact (failed deliveries) = exposure 15.
- GST calculation correctness — Low likelihood (stable code), High impact (tax compliance breach, Revenue NZ penalty) = exposure 10.
- Product image display — Low likelihood, Low impact = exposure 4, test last or not at all if time is cut.
Risk-based testing means: when the sprint is shortened, you can show the stakeholder exactly which risks were fully tested and which were consciously deferred. “We tested payment and address validation; we deferred product image testing — here is the residual risk.”
4 Industry Reality
- Risk registers get stale fast. You build one at sprint planning, then requirements shift, a developer rewrites the payment module mid-sprint, and nobody updates the scores. Senior testers treat risk registers as living documents and re-score after every significant change — not just at kick-off.
- Stakeholders fight the scores. A product owner will insist their pet feature is "high impact" even when a failure is trivially recoverable. You need to be ready to explain the difference between business visibility and actual consequence-of-failure, diplomatically, in a meeting where the PO outranks you.
- Legacy codebases skew likelihood completely. In a 10-year-old NZ insurance policy system, the riskiest modules are often the ones nobody touches — undocumented, no tests, one developer who understands them and they left three years ago. Likelihood isn't just about code churn; it's about knowledge gaps too.
- Time pressure inverts the model. Under crunch, teams default to testing what's easiest, not what's riskiest — exactly the opposite of risk-based testing. You'll spend as much time defending the approach as applying it. "But we haven't tested the home page yet" is a real conversation you'll have.
- The residual-risk conversation is harder than the register. Building the register is the easy part. The hard part is sitting in a release meeting and saying "we consciously did not test X, and here is the residual risk we are accepting" — and having a stakeholder sign off on it. That requires trust, clarity, and documentation you can defend at 9pm before go-live.
Senior engineer insight
The moment that changed how I use risk-based testing was an CoverNZ claims-processing release in 2022. We had a solid risk register — payment routing scored exposure 20, everything was correctly ranked. Then three days before go-live a developer patched the bank-account validation logic to handle a new format. Nobody re-scored it. We shipped. The patch introduced a silent truncation bug that sent partial account numbers to the payment gateway. We caught it in the first post-go-live batch run, not in testing. The register was right when we built it; it was wrong by release day. Since then I re-score after every code change that touches a high-exposure module — not just at kick-off, and not just at sprint boundaries.
The most common mistake I see graduates make is treating the risk register as a one-time artefact — they build it at planning, file it, and then test in whatever order Jira tickets happen to land. The register is only useful if it actively drives the test queue every day, not just the first day.
From the field
On a HealthNZ (Health New Zealand) patient data migration project in 2023, the team built a risk register that correctly ranked the patient-matching algorithm at exposure 20 — new code, high stakes if two patient records merged incorrectly. What nobody re-scored mid-sprint was the address normalisation module, which a developer quietly refactored to handle Māori place names correctly. The register still showed it as exposure 6 (mature code, low churn). The refactored module shipped with a silent truncation bug that stripped suburb fields from South Auckland addresses, causing appointment letters to fail delivery for hundreds of patients. The lesson that generalises: any code touched mid-sprint restarts the clock on its likelihood score, regardless of what the register said on day one.
5 When to Use It — and When Not To
✓ Use it when
- Test time is constrained and you need a defensible way to cut scope — which is almost every real project.
- The system has areas of wildly different consequence: financial transactions next to help-text pages.
- You need to communicate coverage trade-offs to non-technical stakeholders (executives, clients, regulators).
- A go/no-go decision is imminent and you need to demonstrate what was tested and what wasn't.
- The codebase has had significant churn (new features, rewrites, new integrations) that raises likelihood in specific modules.
✗ Skip it when
- Every component has genuinely equal risk — rare, but on small greenfield tools with no financial or safety consequences, flat coverage may be simpler.
- You are writing a unit-test suite for a single isolated function — risk scoring adds overhead with no benefit at that granularity.
- Regulatory compliance requires exhaustive coverage regardless of likelihood (e.g. some NZ health IT standards mandate 100% scenario coverage — risk-based deferral is not permitted).
- The team is so new to testing that a risk-register discussion will stall the sprint — start with checklists and introduce RBT once they're comfortable.
Context guide
How the right level of Risk-Based Testing effort changes based on project context.
| Context | Priority | Why |
|---|---|---|
| Regulated government or health system (NZISM, AoG standards, HealthNZ mandates) | Essential | High-consequence failures (patient harm, incorrect entitlements, privacy breaches) demand an explicit, auditable record of what was tested and what residual risk was accepted. Regulators will ask for it. |
| Legacy system migration (mainframe to cloud, or replacing a 10–15 year-old NZ insurance/banking core) | Essential | Undocumented business logic and unknown integration dependencies mean defect likelihood is high across many modules; risk scoring is the only defensible way to allocate the limited test time available. |
| Enterprise system with complex integrations (ERP, core banking, nationwide benefits platform) | High | Integration points multiply failure modes; risk scoring prevents teams from spreading effort evenly across dozens of connections when only a handful have both high churn and high consequence. |
| Agile sprint with a time-constrained NZ team (two-week sprints, shared tester across squads) | High | A 15-minute risk-ranking at sprint planning is enough to ensure the highest-exposure stories are tested first; without it, test order defaults to Jira ticket sequence and critical paths are the last to be covered. |
| Small startup or greenfield product (pre-revenue, low user base, no financial transactions yet) | Medium | Consequence of failure is still low for most features, so full risk-register formality is overkill; a lightweight mental model (“what would embarrass us most if it broke at launch?”) captures most of the value with none of the overhead. |
| Unit-testing a single isolated function or micro-service with no downstream dependencies | Low | At this granularity the scope is already scoped to a single responsibility; the overhead of exposure scoring adds no value. Test all branches and be done. |
Trade-offs
What you gain and what you give up when you choose Risk-Based Testing.
| Advantage | Disadvantage | Use instead when… |
|---|---|---|
| Defensible scope decisions — you can explain every coverage cut to a stakeholder with a number, not a feeling. | Scoring is inherently subjective. Two testers on the same project can produce radically different exposure tables without shared criteria. | A regulatory standard mandates exhaustive coverage regardless of likelihood — use structured coverage criteria instead. |
| Critical defects surface early because high-exposure modules are tested first, while there is still time to fix them. | Low-exposure areas that are deferred can still contain defects. If a "low-risk" area is wrong in an unexpected way, it won't be caught until production. | All components have genuinely equal consequence — use a flat coverage checklist to keep overhead low. |
| Gives test leads a clear, visual way to communicate coverage trade-offs to executives and clients — a heat map beats a status spreadsheet. | Maintaining a live risk register across sprints adds overhead. Teams under crunch stop updating it, and a stale register is worse than none. | The team is brand-new to testing — the overhead of scoring will stall the sprint. Start with checklists and introduce RBT once coverage habits are established. |
| Aligns automation investment rationally — high-exposure, frequently-run paths get automated first, not whichever feature the developer liked building. | Developers or product owners can game impact scores to protect their pet features, inflating exposure for low-consequence work if the scoring session isn't facilitated carefully. | You are writing unit tests for a single isolated function — risk scoring adds no value at that granularity; just test all branches. |
| Documents residual risk explicitly, protecting the test team when a deferred area breaks post-go-live — you have a signed-off record, not silence. | The residual-risk conversation is politically difficult. Saying "we did not test X" in a release meeting requires confidence and senior support — junior testers often cave and test everything anyway, defeating the approach. | Time is genuinely unconstrained and a complete regression suite already exists — just run it all and skip the scoring overhead. |
6 Best Practices
- Score impact by consequence, not by difficulty or importance to the PO. The question is: if this fails in production, what happens? Financial loss, user harm, regulatory breach, or a cosmetic glitch?
- Score likelihood by code facts, not gut feel. New code, complex integrations, high churn, no unit tests, single developer knowledge — these raise likelihood. Mature, well-tested, unchanged modules lower it.
- Update scores after every significant change. A new developer picked up the payment module? Re-score likelihood. A regulatory deadline moved? Re-score impact. Risk is not static.
- Test the highest-exposure items first in every sprint. Not first on day one — first in every test cycle, so critical defects surface while there is still time to fix them before the release.
- Separate product risk from project risk in documentation. A late test environment is a project risk; it goes in the test plan's risk section. The product risk register drives test coverage decisions only.
- Document residual risk explicitly for every deferred area. "We did not test X because it scored exposure 2. If it fails, the worst case is Y. The business has accepted this risk." This is what protects you post-go-live.
- Map exposure scores to test depth thresholds. Define upfront: exposure 15+ = full EP/BVA/decision-table coverage; 8–14 = happy path + key boundary cases; 1–7 = smoke test only. This makes depth decisions automatic, not negotiated case by case.
- Align automation investment with risk. High-exposure paths that run on every build are the automation priority. A smoke-test-only area is not worth the automation investment until the risk score changes.
- Involve developers in likelihood scoring. They know where the skeletons are — the module nobody touches, the integration that has never been regression-tested, the API with no error handling. Their input makes your likelihood scores far more accurate than you could achieve from test specs alone.
- Present the register visually to stakeholders. A heat map (red/amber/green by exposure) lands in a governance meeting far better than a raw score table. The goal is sign-off on the risk-based test strategy — you need it to be legible in 30 seconds.
7 Common Misconceptions
❌ Myth: Risk-based testing means skipping tests — it's a shortcut for lazy testers.
Reality: Risk-based testing means skipping low-value tests when time is constrained — which is not laziness but professional judgement. It requires more rigour upfront (building and defending a risk register) than ad-hoc testing does. The discipline is in making a conscious, documented, stakeholder-approved decision about what to defer, not in avoiding work. Senior testers use it precisely because exhaustive testing is never possible and pretending it is creates a false sense of coverage.
❌ Myth: The highest-risk item is the one that's hardest to code or most technically complex.
Reality: Technical complexity contributes to likelihood of failure, but risk exposure also requires impact — the consequence if it fails. A technically trivial function that processes Revenue NZ tax payments has low likelihood but extreme impact, giving it high exposure. A technically fiendish animation on a marketing page may be likely to glitch but has near-zero impact. Scoring by technical difficulty alone produces a register that tests the hard stuff, not the dangerous stuff.
❌ Myth: You build the risk register once at the start of the project and it covers you throughout.
Reality: Risk scores change as the project evolves. A feature that was low-risk in sprint 1 (mature code, no changes) may become high-risk in sprint 6 if a developer rewrites it to meet a new requirement. Requirements change, integrations are added, deadlines shift — all of which alter both likelihood and impact. An effective risk register is reviewed and updated at the start of each sprint or test cycle, not treated as a one-off artefact.
8 Now You Try
Three graded exercises — spot, fix, then build. Write your answer, run it for AI feedback, then compare to the model answer.
A NZ health-board patient portal has these four feature areas. For each, assign a likelihood (1–5) and impact (1–5), calculate exposure, and rank them in test order. Areas: (a) prescription repeat ordering — new code, touches medication; (b) appointment booking — mature code, high traffic; (c) news/notices page — static content; (d) login via RealMe — external dependency, gates all access.
Show model answer
A defensible ranking (exact scores can vary, the reasoning matters): - Prescription repeat: Likelihood 4 (new code) x Impact 5 (patient safety, medication) = 20 — test FIRST and deepest. - Login (RealMe): Likelihood 3 (external dependency) x Impact 4 (blocks all access if broken) = 12 — test early; mature dependency lowers likelihood. - Appointment booking: Likelihood 2 (mature code) x Impact 4 (high traffic, but a missed booking is recoverable) = 8. - News/notices: Likelihood 1 (static) x Impact 1 = 1 — test last, or defer and document the residual risk. Test order: prescription repeat, login, appointment booking, news/notices. The key call is that new code touching medication outranks high-traffic-but-mature booking. Likelihood is driven by code churn and complexity; impact by harm if it fails. A safety-of-life consequence pushes impact to 5.
A junior tester built the risk register below for a KiwiFirst Bank-style transfers feature. It is wrong: it confuses project risk with product risk, scores impact by how hard the code is rather than the consequence of failure, and ranks a cosmetic item above a money-movement one. Rewrite it as a correct product-risk register with sensible exposure scores and test order.
Test environment is late — Likelihood 5, Impact 5, Exposure 25 (test first)
Inter-account transfer — Likelihood 2, Impact 2, Exposure 4
Dashboard colour theme — Likelihood 4, Impact 4, Exposure 16
Transaction history export — Likelihood 2, Impact 1, Exposure 2
Rewrite as a correct product-risk register:
Show model answer
Correct product-risk register (project risks removed, impact scored by consequence): - Inter-account transfer: Likelihood 3 x Impact 5 (money movement, loss and trust) = 15 — test first and deepest. - Transaction history export: Likelihood 2 x Impact 3 (users rely on records, but errors are visible and recoverable) = 6. - Dashboard colour theme: Likelihood 2 x Impact 1 (cosmetic) = 2 — test last or defer. What was wrong with the original: - "Test environment is late" is a PROJECT risk, not a product risk. It belongs in the test plan's risk section, not the product risk register that drives test coverage. Remove it from this register. - Impact was scored by how hard the code is, not by the consequence of failure. A colour theme cannot be Impact 4 — nobody is harmed by the wrong shade of teal. - The ranking was upside down: a cosmetic theme outranked moving customer money. Money movement is the highest-impact area and must be tested first.
A NZ supermarket is launching online click-and-collect with two weeks of test time. Build a short risk-based test plan: list at least four feature areas, give each a likelihood × impact = exposure score, then state for each how that exposure changes your test depth, test order, and automation priority. Finish with what you would defer if the two weeks were cut to one.
Show model answer
A strong plan for click-and-collect (scores indicative):
- Payment and order creation: Likelihood 4 x Impact 5 = 20 — deepest coverage, test first, automate the critical path.
- Stock/availability sync (shown in-store vs online): Likelihood 4 x Impact 4 = 16 — deep testing, early; over-selling damages trust.
- Collection slot booking: Likelihood 3 x Impact 3 = 9 — moderate depth, automate the happy path.
- Loyalty points on the order: Likelihood 2 x Impact 2 = 4 — light testing.
- Marketing banner on the confirmation page: Likelihood 1 x Impact 1 = 1 — smoke only.
How exposure drives the plan:
- Depth: exposure 20 areas get full EP/BVA/decision-table coverage; exposure 1 areas get a single smoke test.
- Order: highest exposure first, so critical defects surface while there is still time to fix them.
- Automation: high-exposure, frequently-run paths (payment, stock sync) are automated first because they will run on every build.
If cut to one week: defer loyalty points and the marketing banner; document the residual risk explicitly ("loyalty point accrual untested — worst case is a points correction, not a failed order"). Keep payment, stock sync, and slot booking fully tested. The whole value of the approach is being able to show stakeholders exactly what was tested and what was consciously left out.
Why teams fail here
- They build the register once and never touch it again. On a Coastal Bank core banking release, the risk register was accurate at sprint planning and irrelevant by release day — three modules had been rewritten since the scores were set. A stale register is worse than no register: it creates false confidence that high-risk areas are covered when they are not.
- They let developers score their own module's likelihood. The developer who rewrote the Revenue NZ tax calculation engine rates it likelihood 2 because they are proud of it. The objective signal — new code, complex integration, no unit tests — says likelihood 4. Developer confidence and code risk are different things, and conflating them quietly drains coverage from the modules that need it most.
- They test the register rather than use it. The register gets built, filed in Confluence, and the team then tests in Jira ticket order because that is what the sprint board shows. On an Benefits NZ benefit-processing release, the risk register correctly ranked entitlement calculation at exposure 20 — and the team still tested the dashboard redesign first because it was at the top of the board. A risk register that does not actively change the test queue every day is decoration.
- They cave on the residual-risk conversation. When a release meeting gets tense and a product owner says "just test everything," junior testers agree and abandon the deferral decisions. On an CoverNZ claims portal go-live, a tester dropped the documented deferral of the PDF receipt generator under stakeholder pressure and spent the last two test days on it — leaving the bank account validation with one smoke test. The value of risk-based testing collapses the moment the low-exposure deferrals become optional.
Key takeaway
The risk register does not protect users — the conversation it forces you to have in the release meeting does: "here is exactly what we tested, here is exactly what we didn't, and here is the worst thing that can happen if we're wrong."
Self-Check
Click each question to reveal the answer.
How this has changed
The field moved. Here is how Risk-Based Testing evolved from its origins to current practice.
Risk-based approaches documented in testing literature but rarely formalised. Experienced testers implicitly prioritise risk — they just do not call it that. High-risk areas get more attention because practitioners know the domain.
ISTQB Foundation syllabus formalises risk-based testing. Product risk and project risk are distinguished. Risk matrices enter testing plans. The language becomes standard even if the practice varies widely.
Agile forces risk assessment into refinement sessions. "What could go wrong with this story?" replaces formal risk documents for many teams. Risk-based test selection becomes a daily conversation rather than a planning artefact.
ISO 31000 risk management framework and IEC 61508 functional safety standards push regulated industries toward formal risk-based test evidence. NZ government programmes adopt gateway reviews with risk-based test coverage as a sign-off criterion.
AI tools can surface risk signals from code change history, defect data, and usage patterns — helping teams identify high-risk areas more systematically. Human judgement still determines risk appetite and trade-offs. ISO/IEC 42119 introduces risk classification specifically for AI systems.
Q1: What is the difference between product risk and project risk, and which one does risk-based testing primarily address?
Product risk is the risk that the software fails to meet stakeholder needs (payment errors, data loss, security holes) — these become test targets. Project risk affects the testing process itself (late environments, incomplete requirements, capacity). Risk-based testing primarily addresses product risk; project risk is managed in the test plan.
Q2: How is risk exposure calculated, and what drives each factor?
Exposure = likelihood × impact. Likelihood is driven by technical factors — code complexity, churn, integration points, defect history, developer experience. Impact is driven by consequence — financial loss, user harm, reputational damage, regulatory breach. You multiply the two and sort descending.
Q3: A cosmetic dashboard theme is technically tricky to get right. Should it score a high impact?
No. Impact measures the consequence of failure, not how hard the code is to write. A cosmetic feature has low impact regardless of difficulty, because nobody is harmed if it fails. Confusing difficulty with impact is a classic scoring mistake that inflates low-value work up the test order.
Q4: Beyond test order, name two other test decisions that risk exposure should drive.
Test depth (high-risk areas get full EP/BVA/decision-table coverage; low-risk areas get smoke tests or none) and automation priority (high-risk, frequently run paths are automated first). It also informs exit criteria — you can demonstrate the high-risk areas were fully tested.
Q5: What is the practical payoff of a risk register when the schedule is cut?
It lets you make a defensible, transparent call about coverage. You can show stakeholders exactly which high-risk areas were fully tested and which low-risk areas were consciously deferred, along with the residual risk — instead of cutting tests at random and hoping.
Q6: CoverNZ is releasing a new online injury claim form two weeks before end of financial year. The form has five sections: personal details, employer information, injury description, medical evidence upload, and payment bank account. You have three days of test time. Which section do you test first and deepest, and why?
A: The payment bank account section should be tested first and deepest — it has the highest risk exposure because an error directly causes financial harm to injured claimants (high impact) and it involves new form logic touching sensitive data (elevated likelihood). Medical evidence upload is second: a failure blocks the claim from proceeding (high impact) and file-handling code is a common defect hotspot. Personal details and employer information are mature, low-consequence fields and can be covered with a quick smoke test. Injury description is free text with no business rules, so it tests last or is deferred entirely.
Q7: What is the key difference between risk-based testing and exploratory testing, and when would you use one over the other?
A: Risk-based testing is a planning and prioritisation strategy — it decides what to test, how deeply, and in what order based on exposure scores before testing begins. Exploratory testing is an execution technique where a tester simultaneously designs and executes tests, learning as they go. They complement rather than compete: you use risk-based testing to allocate time across feature areas, then use exploratory testing as one of the execution methods within a high-risk area when requirements are incomplete or when you want to uncover unexpected defects. For a brand-new TransitNZ licensing system, risk-based testing tells you to spend most time on licence calculation logic; exploratory testing is how you attack that module once you're in it.
Q8: A developer tells you: "We don't need a risk register — we're agile. Risk-based testing is a waterfall concept that doesn't fit sprints." What is wrong with this and how do you respond?
A: This is a common misconception. Risk-based testing is not a phase or a document tied to waterfall — it is a decision-making principle that applies within any sprint. In agile, a lightweight risk register can be built at sprint planning in 15 minutes: list the stories by exposure, decide which get full coverage and which get a smoke test. The register is then updated each sprint as requirements and code change — which is actually more aligned with agile's iterative nature than a one-time waterfall risk plan. Rejecting risk scoring in agile usually means teams default to testing whatever is easiest rather than whatever would hurt most if it broke.
Q9: When should you NOT use risk-based testing, and what approach should you use instead?
A: There are three main situations. First, when a regulatory standard mandates exhaustive coverage regardless of likelihood — for example, some Health New Zealand IT standards and certain Medsafe-adjacent systems require 100% scenario coverage, so risk-based deferral is not permitted. Second, when you are writing unit tests for a single isolated function — the overhead of scoring exposure adds no value at that granularity; just test all branches. Third, when the team is new to testing and a risk-register discussion will stall the sprint — start with simple checklists, build the habit of coverage first, and introduce risk-based prioritisation once the team is comfortable. In these cases, use exhaustive test design techniques, coverage criteria, or structured checklists instead.
Enterprise reality
Enterprise programmes, regulated environments, multi-team delivery
- Risk registers become formal governance artefacts signed off by named risk owners — typically a programme manager or business sponsor — not informal spreadsheets the QA team maintains alone. Residual risk acceptance requires a signature, not a verbal nod in standup.
- Risk appetite thresholds (what counts as High, Medium, Low exposure) are set at programme-board level, not by the QA team. A government agency may mandate that anything above exposure 12 requires executive sign-off before it can be deferred — your job is to score accurately and escalate, not to set the threshold yourself.
- Test coverage must trace to risk items in project governance documentation — Treasury Better Business Cases, OIA-sensitive audits, or ISO-mandated control evidence. Auditors ask: “Show me which test cases covered risk item 7.” A register with no traceability to test cases fails that check.
- Multi-team delivery (squads, systems integrators, offshore vendors) means risk registers must be consolidated across workstreams. Each squad scores their own module, but a test lead or QA architect reconciles scores across the programme — otherwise a critical integration point between Squad A and Squad B falls into a gap that neither scored.
★ Interview Questions
What NZ hiring managers ask about Risk-Based Testing — and what strong answers look like at each level.
Q: What is risk-based testing and why do we use it?
Strong answer: Risk-based testing is an approach where you prioritise test effort based on risk exposure — the product of how likely an area is to fail and how serious the impact would be if it does. We use it because time is almost never sufficient to test everything; risk-based testing gives you a defensible, documented rationale for every coverage decision so that when time runs out, the untested areas are always the low-consequence ones, not the critical paths.
Grad / Junior
Q: What is the difference between product risk and project risk, and which does a risk register address?
Strong answer: Product risk is the risk that the software fails to meet stakeholder needs — payment errors, data loss, security vulnerabilities — and these become test targets. Project risk affects the testing process itself, such as a late test environment or incomplete requirements, and belongs in the test plan. A risk register drives test coverage decisions and should only contain product risks; mixing in project risks (like "the environment will be late") is a common mistake that inflates scores and distorts priorities.
Grad / Junior
Q: You're testing an Revenue NZ income tax calculator. The developer scored their own module's likelihood as 2 out of 5 because they're confident in the code. How do you handle that?
Strong answer: I'd keep likelihood scoring separate from the module's author. Developer confidence is not the same as low defect risk — it reflects pride of ownership, not code history. I'd pull likelihood from objective sources: code churn metrics, commit frequency over the last sprint, integration points with external APIs, and historic defect counts on that module. For a tax calculation touching every taxpayer's return, I'd also weight impact at 5 regardless of likelihood, which keeps the exposure high and ensures it receives deep coverage. The rule is: never let the author score their own module's likelihood.
Senior
Q: When would you NOT use risk-based testing, and what would you use instead?
Strong answer: Three situations come to mind. First, when a regulatory standard mandates exhaustive coverage regardless of risk — some Health New Zealand IT systems and Medsafe-adjacent platforms require 100% scenario coverage, so risk-based deferral isn't permitted; you use structured coverage criteria instead. Second, for unit tests on a single isolated function — the overhead of scoring exposure adds no value at that granularity; just test all branches. Third, when a team is new to testing and a risk-register discussion will stall the sprint — start with checklists and introduce risk-based prioritisation once they're comfortable with basic coverage thinking.
Senior
Q: CoverNZ is two weeks from launching a new online injury claim form. The schedule just got cut by four days. How do you use risk-based testing to manage the cut?
Strong answer: I'd convene a quick risk-scoring session with the team and re-rank the outstanding test areas by exposure. The payment bank account section and medical evidence upload would stay non-negotiable — financial harm to injured claimants and claim-blocking failures both hit exposure 20. Personal details and static help text would be the explicit deferrals, with residual risk documented: "personal details fields untested — worst case is a data-entry correction, not a failed payment." I'd bring that documented list to the release meeting so stakeholders can sign off on exactly what risk we're accepting, rather than cutting test time silently and hoping. The whole value of the approach is that the conversation happens before go-live, not after.
Senior
Q: How would you introduce risk-based testing to a development team that has never used it, and how would you measure whether it's working?
Strong answer: I'd start lightweight: at the next sprint planning, spend 15 minutes scoring the sprint's stories on a simple 1–5 likelihood × impact matrix and use the output to set test depth explicitly — "these three stories get full coverage, these two get a smoke test." Make it visible, not bureaucratic. Over successive sprints I'd track two things: are critical defects surfacing during testing rather than in production, and when the team de-scopes test work under time pressure, are the items dropped always the low-exposure ones? If escape rates on high-exposure areas fall and stakeholders can articulate what residual risk they accepted at each release, the approach is working. If the register is being built but ignored when the sprint runs long, I'd address the culture gap — risk-based testing only works if low-score items can actually be dropped.
Lead
◆ What I would do
Professional judgment — when to reach for Risk-Based Testing, when to skip it, and what to watch for.
The bottom line: Risk-based testing is not a document you produce — it is the discipline of always being able to answer "if we ship tomorrow with what we've tested so far, what is the worst thing that could go wrong?" If you cannot answer that question clearly, your risk register is not doing its job.