Home/ Senior/ Learning/ Risk-Based Prioritisation
Senior · Test Strategy Technique

Risk-Based Prioritisation

You can't test everything equally. Risk-based prioritisation tells you where to aim first, how deep to go, and when to stop.

Senior ISTQB CTAL-TA 2.3 (Risk-Based Testing) — K4 Analyse ~12 min read + exercise

1 The Hook — Why This Matters

In 2021, a Wellington fintech startup shipped a major update to their lending platform. The team had two weeks to test. They spent the first week on UI polish and form validation, leaving only three days for payment integration. On launch day, POLi transfers above $10,000 silently failed. No error message reached the user. The startup processed $2.3 million in loans that never actually transferred.

The root cause wasn't a lack of testing. It was equal weighting. The team treated "wishlist icon alignment" with the same urgency as "bank transfer settlement." A risk-based approach would have flagged payment flows as exposure 25 (likelihood 5, impact 5) and allocated the deepest testing there first.

Time is always finite. Risk tells you where to spend it.

2 The Rule — The One-Sentence Version

Test effort must be proportional to risk exposure, which is likelihood multiplied by impact.

A feature that is complex and business-critical deserves deep, multi-layered testing. A feature that is simple and low-impact deserves light testing or none at all. Equal effort across both is not fairness. It's negligence.

3 The Analogy — Think Of It Like...

Analogy

A hospital triage nurse on a Saturday night.

A patient with chest pain goes straight to resus. A patient with a sprained ankle waits. The nurse doesn't assess everyone for the same length of time. They assess urgency and severity, then allocate resources accordingly. Risk-based testing is triage for your product. If you spend twenty minutes on the ankle while the chest pain patient waits, people get hurt.

Senior engineer insight

The shift that changed how I approach risk scoring was realising that a likelihood of 5 doesn't mean a defect is inevitable — it means your team has the least institutional knowledge in that area. When a squad inherits a legacy payment integration they didn't write, "likelihood" encodes uncertainty as much as probability. That reframe made me push harder for developers to co-own the register; they know exactly which modules have undocumented behaviours hiding in them. Once we started scoring together, our register stopped being a tester's artefact and became the team's shared map of where the ground was soft.

Most common mistake: treating the exposure number as a final decision rather than a conversation starter — then never having the conversation.

From the field

On a central government programme modernising the Revenue NZ's tax debt reconciliation module, the team assumed the highest-risk area was the new user-facing dashboard because it was the most visible deliverable. The risk register was scored in week one by a single analyst and never revisited. Eight weeks in, an automated batch reconciliation job — scored exposure 8 because it was "just moving existing data" — began silently writing incorrect write-off amounts when debt records had been split across tax years. The error ran undetected for three weeks because no one had mapped it to a test suite. The NZ Government's risk management framework (NZGF) explicitly requires that risk registers reflect the consequence of systemic failures, not just user-visible ones; the team had treated compliance risk as a checkbox rather than a scoring input. After the incident, we rebuilt the register collaboratively with the Revenue NZ's enterprise risk team, added batch processes as a mandatory scoring category, and linked every high-exposure item to a named exit criterion. The lesson that generalises: silent automated processes almost always carry higher impact than their technical simplicity suggests.

4 Watch Me Do It — Step by Step

Here is a real NZ e-commerce risk register. Follow these steps to build your own.

  1. Identify risks across four categories Brainstorm functional risks (user journeys), technical risks (integration complexity), business risks (revenue, reputation), and regulatory risks (compliance, data privacy).
  2. Assess likelihood and impact on a 1–5 scale Likelihood: how probable is a defect here? Impact: what happens if one slips through? Be specific. Use historical defect data where you have it.
  3. Calculate risk exposure: Likelihood × Impact This gives you a score from 1 to 25. It is a prioritisation input, not a prophecy.
  4. Map exposure to test effort
    • 20–25 (Deep): exhaustive functional, edge case, regression, and exploratory testing
    • 10–16 (Moderate): standard functional and regression testing
    • 1–9 (Light / None): smoke test or skip, depending on capacity
  5. Review and adapt throughout the project Risk is not static. New code, new requirements, and defect clusters change the picture. Update the register at sprint boundaries.
NZ e-commerce risk register
Feature area Likelihood (1–5) Impact (1–5) Exposure (L×I) Test effort
Payment (POLi/Stripe)5525DEEP
GST calculation4312MODERATE
Address validation4312MODERATE
Product search236LIGHT
Wishlist122NONE
Pro tip: In New Zealand, health apps must consider HISF (Health Information Security Framework) compliance risks. Financial systems must consider RBNZ FMI (Financial Market Infrastructures) requirements. Regulatory risk often has an impact of 5 regardless of likelihood, because a breach can shut down the business.
Pro tip: Use a risk matrix heat map to make the register visual. Involve the whole team in scoring — developers often know technical likelihood better than testers do. Link risk scores to exit criteria: "No open defects in exposure 20+ areas before release."

Second Example: Christchurch Insurance Broker App

A Christchurch-based insurance broker is releasing a major update: new policy quote engine, integration with Revenue NZ for income verification, and a customer dashboard redesign. The team has identified the following risks:

AreaLikelihoodImpactExposureTest Effort
Quote engine calculation (insurance premium logic)4520DEEP — exhaustive; test edge cases (high income, dependants, pre-existing conditions); verify calculation accuracy against actual quotes from competitors
Revenue NZ income verification (automated data sync)5525DEEP — integration testing with Revenue NZ test harness; verify end-to-end data flow; test timeout and retry logic for network failures; verify no customer data is cached insecurely
Dashboard redesign (UI only)224LIGHT — smoke test across browsers; check basic responsive behaviour

Rationale: The quote engine and Revenue NZ integration are both high-likelihood and high-impact. A calculation error means the broker issues incorrect policies, creating legal and compliance liability. Revenue NZ integration failures could lock customers out of the application. The dashboard redesign is purely cosmetic; if it breaks, users can still access quotes and policies through the old interface.

5 When to Use It / When NOT to Use It

✅ Use risk-based prioritisation when...

  • Time or budget is constrained (which is always)
  • You are testing a new feature with unknown stability
  • The system has regulatory or compliance requirements
  • You need to justify test coverage to stakeholders
  • Defect history shows clustering in specific modules
  • You are defining entry and exit criteria for a release

❌ Don't use it when...

  • The project is tiny and every feature is equally critical
  • You have no access to business or technical context
  • The team uses it as an excuse to skip testing entirely
  • You treat the first risk register as final and never revisit it
  • Scores are assigned by one person without team input

⚠ When It Fails

Risk-based prioritisation fails when low-exposure features cause high-impact failures. You skipped testing the login reset flow (exposure 6) and a user locked themselves out of their $50,000 investment account. It also fails when risk scoring is too subjective or outdated. A module you marked as low-risk has shipped three critical defects in the past two quarters, but you're still allocating it light testing. Re-validate your register every sprint using actual defect data. If predicted risk and actual defects diverge, update the scoring before making test allocation decisions.

📋 Decision Checklist: Should You Use Risk-Based Prioritisation?

  • Do you have limited testing time or budget? (Almost always yes.) Risk-based prioritisation helps you allocate effort where it matters most.
  • Can you access business and technical context? You need developers, product owners, and compliance leads to score likelihood and impact accurately.
  • Will you update the risk register throughout the project? If you create a register in week one and ignore it for twelve weeks, you'll miss emerging risks and defect clusters.

6 Common Mistakes — Don't Do This

🚫 Equal weighting everything

I used to think: Every feature deserves the same attention. It's fairer that way.
Actually: Equal effort means your payment flow gets the same three test cases as your "About Us" page. Risk is not about fairness. It's about probability and consequence. A defect in POLi processing costs revenue and trust. A defect in the wishlist costs almost nothing.

🚫 A static risk register that never updates

I used to think: We made the risk matrix in sprint zero. Job done.
Actually: Risk changes as code changes. A module that was stable in sprint two may become risky in sprint six after a major refactor. A module with zero historical defects may suddenly produce a cluster. Review the register every sprint. Treat it as a living document.

🚫 Only scoring technical risk

I used to think: Complex code = high risk. Simple code = low risk.
Actually: A one-line GST rounding rule is technically trivial, but a $0.01 error in every transaction is a business catastrophe. You must score business impact and regulatory exposure separately from technical complexity. Involve product owners and compliance leads in scoring sessions.

⚠ When It Fails

Risk-based prioritisation fails when low-exposure areas mask high-consequence bugs. You scored address validation as exposure 6 and tested it lightly. In production, an edge case (PO Box with no street number) caused all rural claims to be routed to the wrong office. It also fails when the risk register becomes obsolete. You built it in sprint zero and never updated it. By sprint eight, the payment module has a history of three critical defects, but you're still allocating it moderate effort because that's what the old register says. Review and re-score every two sprints based on actual defect data.

7 Now You Try — Build a Mini Register

🎯 Interactive Exercise

Scenario: You are testing a NZ health appointment booking app. You have one week. Identify the test effort for each area using L×I scoring.

  • Patient data sync with GP systems (HISF regulated)
  • Appointment reminder SMS
  • Clinic profile photo upload

Assign likelihood and impact scores, then reveal the suggested register.

Suggested risk register:

Feature areaLikelihoodImpactExposureEffort
Patient data sync4520DEEP
Appointment SMS326LIGHT
Clinic photo upload212NONE

Why: Patient data sync is HISF-regulated. A breach or data loss has regulatory and reputational impact of 5. SMS reminders are useful but easily fixed if broken. Photo uploads are cosmetic. Spend your week on data sync.

Why teams fail here

  • Scoring sessions are run by the test lead alone — developers and product owners never see the register until it's presented as done, so likelihood scores reflect tester intuition rather than engineering knowledge of code fragility.
  • The register is created at project kick-off and treated as a fixed document; when the architecture changes in sprint four, nobody updates the scores, and the team keeps allocating light effort to modules that have become high-risk after a major refactor.
  • Regulatory impact is conflated with technical complexity — a one-line GST rounding rule gets a low impact score because it's simple to write, when in fact an error affects every transaction and triggers an Revenue NZ audit obligation.
  • Risk-based prioritisation is used to justify skipping testing entirely on low-exposure items without documenting the residual risk, so the project manager approves a release believing all areas are covered when several have had zero test execution.

Key takeaway

Risk-based prioritisation is not about testing less — it is about being honest that you cannot test everything and choosing, deliberately and with evidence, where a miss would hurt most.

8 Self-Check — Can You Actually Do This?

Click each question to reveal the answer. If you got all three, you're ready for the interview prep.

Q1. What is the difference between product risk and project risk?

Product risk is the risk that the software will fail to meet user or business needs (defects, poor performance, security holes). Project risk threatens the delivery itself — schedule slip, skill gaps, tool failures. Risk-based testing focuses on product risk, but project risk can amplify it (e.g., time pressure increases the likelihood of defects).

Q2. A feature scores likelihood 2 and impact 5. Another scores likelihood 5 and impact 2. Both have exposure of 10. Should they get the same test effort?

Not necessarily. Exposure is a guide, not a rule. The high-impact / low-likelihood feature (2×5) might warrant deeper testing because the consequence of a miss is severe. The high-likelihood / low-impact feature (5×2) might be cheaper to fix if it fails. Use exposure as a starting point, then apply judgment based on context and fix cost.

Q3. Why should predicted vs actual defect cluster analysis inform the next risk register?

Because risk assessment is a prediction, and predictions improve with data. If you predicted high defects in module A but found none, while module B produced a cluster you didn't anticipate, that tells you your likelihood model was wrong. Update the register with this evidence before the next cycle.

Related techniques

  • Risk-Based Testing — the full technique — likelihood/impact matrices, risk registers, and coverage allocation
  • Test Estimation — risk priority feeds effort estimation — higher risk areas get more testing time and different techniques
  • Test Planning — risk-based prioritisation is the core input to the test plan scope and coverage decisions

9 Interview Prep — Questions You Might Get

Click each question to reveal a strong answer structure.

Q1. How do you handle a stakeholder who wants everything tested equally?

Show them the risk matrix. Explain that equal effort means a critical payment flow receives the same attention as a cosmetic wishlist icon. Use historical defect data to demonstrate where bugs actually cluster. Frame it as resource protection: "We have finite time. Risk-based prioritisation ensures the features that could damage revenue or compliance get the deepest coverage first." Offer to review the register together so they feel ownership of the scores.

Q2. What NZ regulatory frameworks affect risk scoring in software testing?

For health applications, HISF (Health Information Security Framework) governs data security and privacy. For financial systems, RBNZ FMI (Financial Market Infrastructures) sets resilience and continuity expectations. Both frameworks can force an impact score of 5 regardless of technical likelihood, because a compliance breach can result in loss of licence or operational shutdown.

Q3. How do you communicate residual risk to a project manager?

Residual risk is the risk that remains after testing. I communicate it explicitly: "We have tested the payment flow deeply. Residual risk is low-medium. We have done light testing on the photo upload feature. Residual risk is medium-high." I document it in the test summary report with the risk register attached, so the PM makes an informed go/no-go decision rather than assuming everything is equally covered.

Q4. When would you skip risk-based prioritisation and test everything exhaustively?

Almost never, because exhaustive testing is theoretically impossible for non-trivial systems. The only scenario close to it is safety-critical embedded systems where failure means loss of life, and even then engineers use model checking and formal verification, not manual exhaustive testing. For commercial software, risk-based prioritisation is the professional standard. The real question is how granular and evidence-driven your risk model is.

← All Senior learning Full reference (ISTQB deep-dive) → Defect Analysis →