Exploratory Testing Charters
Unstructured exploratory testing is ad hoc. Charter-based exploratory testing is structured, time-boxed, and produces a defensible record of what was explored and what was found.
1 The Hook
A QA tester does 4 hours of "exploratory testing" before a release. Finds 3 bugs. Manager asks: "What did you cover? What did you miss?" She can't answer precisely.
Another tester runs the same 4 hours with 4 charters — one per major area. At the end, she can show exactly: what the mission was, what she explored, what she found, and what she ran out of time for. Both found bugs. Only one produced evidence.
When a defect slips to production, "I did exploratory testing for 4 hours" does not hold up. "Charter ET-03 covered the payment flow using boundary values; the session note records that the 3-decimal-place amount was not tested due to time constraint" does.
Most teams write charters but skip the debrief — and that is exactly when the technique breaks. I have reviewed post-incident reports on NZ government projects where charters existed, session notes existed, and the defect still slipped. Every time, the same pattern: the "Not covered" section was left blank or filled in after the fact to look tidy. Testers were finishing sessions under release pressure and not honestly recording what they ran out of time on. In a CCCFA or Benefits NZ audit, it is the session note that gets forensically read — not the charter. Write the "Not covered" row first, before you start the session, as a forcing function. It sounds backwards, but it makes you honest about scope before the clock starts.
Senior engineer insight
The charter format feels like overhead until the first time a defect slips to production and you can pull up a session note that says exactly what was covered and what was not — suddenly the 5 minutes it took to write the charter looks like the cheapest insurance you ever bought. What changed how I think about this: I stopped treating the information goal as a throwaway line and started treating it as the actual test exit criterion. "To discover how the system handles amounts with 3 decimal places" is both the mission and the done condition; when you can answer that question, the session is over.
The most common mistake: teams write charters but fill in the session note after the debrief to make it look tidy, rather than recording honestly what they ran out of time on. An audit-ready session note is written during the session, not reconstructed from memory an hour later.
From the field
On a NZ insurance platform mid-sprint, the team had four testers and three hours before the release cut-off. Rather than four people loosely exploring the same screens, the Test Lead wrote four charters in ten minutes — one per high-risk area — and assigned them in the standup. At the sprint review the Test Lead could show the product owner a session-coverage matrix: four charters, four session notes, twelve bugs found, two areas explicitly flagged as not fully covered and accepted as known risk. The product owner signed off on the release not because everything was tested, but because the gaps were named and owned.
The lesson that generalises: charter-based ET does not just find bugs — it produces the coverage evidence that lets a product owner make an informed go/no-go decision, which is exactly what sprint-end stakeholders at NZ agile teams need but rarely get from ad hoc testing.
2 The Rule
Write a charter before you start exploring. A charter is not a script — it's a mission statement. It defines the scope, the approach, and how you'll know when you're done.
3 The Analogy
A charter is like a dive plan for a scuba diver.
You're not told exactly where to swim — you decide that as you explore. But you know your maximum depth, your time limit, your buddy, and your objective. The plan isn't restrictive; it's what makes the dive safe and purposeful. A diver without a plan is a risk to themselves. A tester without a charter is a risk to the release.
4 Watch Me Do It
Charter format: Explore [area] using [approach] to discover [information]
Three charters for a NZ banking app pre-release session:
Explore the international payment flow using boundary values on the amount field (minimum NZD transfer, maximum daily limit, amounts with 3 decimal places) to discover how the system handles edge cases in amount validation.
Time box: 60 min Tester: Aroha M Risk area: Financial calculation correctness
Explore the login flow using multiple concurrent sessions from different devices to discover how the system manages session conflicts and logout behaviour.
Time box: 45 min Tester: James T Risk area: Security — session hijacking, ghost sessions
Explore the transaction history view using different date ranges and search terms to discover filter accuracy, pagination behaviour, and performance under a large transaction set.
Time box: 60 min Tester: Aroha M Risk area: Data display accuracy, performance
Session notes template:
| Field | What to record |
|---|---|
| Charter | The mission statement (verbatim) |
| Tester | Name and date |
| Duration | Actual time spent |
| Areas covered | What you actually explored (may diverge from charter) |
| Bugs found | Bug ID or description + severity |
| Issues / questions | Unclear behaviour, risks, questions for the team |
| Not covered | What the charter intended but time ran out |
5 When to Use It
- Before each release — write charters the sprint before to focus on the highest-risk areas
- When testing a new feature with unclear requirements — the charter defines what you're trying to learn, not what you're trying to confirm
- When time is limited and you need to maximise coverage — time-boxed charters force prioritisation
- When you need to document what was explored for sign-off — charters + session notes are the audit trail
- When onboarding a new tester to a complex area — give them a charter, not a blank slate
6 Common Mistakes
❌ I used to think: exploratory testing means testing without documentation.
Actually: undocumented exploratory testing is indistinguishable from random clicking. Charters and session notes are the minimum viable documentation. They don't slow you down — a charter takes 5 minutes to write and saves an hour of explaining when something slips to production.
❌ I used to think: a charter should specify exactly what to test.
Actually: a charter sets the mission, not the steps. "Click button A, then B, then C" is a script, not a charter. The tester decides how to explore within the defined area and time box. The charter gives freedom within a boundary — that's the point.
❌ I used to think: exploratory testing is for when you don't have time for proper testing.
Actually: charter-based exploratory testing is proper testing. It's the technique that finds defects scripted tests miss — because it combines tester skill and intuition with systematic coverage documentation. It's not a fallback; it's a primary technique for high-risk areas.
7 Industry Reality
- Most teams call ad hoc clicking "exploratory testing." You'll often be the first person on a team who writes an actual charter — expect to explain the difference between a charter and a test case at least twice before it sticks.
- Charter writing is frequently squeezed out by sprint pressure. In practice, senior testers write charters in 5–10 minutes on a sticky note or Confluence page, not polished documents — the discipline matters more than the format.
- Session notes almost always reveal scope creep: you set out to explore the payment flow and end up 30 minutes deep in an unrelated date-picker bug. That's not failure — the session note records the detour and its justification. Experienced testers debrief honestly rather than retro-fitting the note to the original charter.
- In NZ government and regulated finance projects (RMA, CCCFA, NZFMA), charter-based ET is increasingly expected as evidence of due diligence — not just a technique choice. You may be asked to hand over session notes as part of a UAT sign-off package or audit trail.
- Remote and offshore team members often receive charters as onboarding scaffolding — a charter assigns an area and an approach without requiring the new tester to already know the full system. This is one of the most underused practical benefits.
8 When to Use It — and When Not To
✓ Use it when
- The feature is new, poorly specified, or the requirements are ambiguous — a charter defines what you're trying to learn, not confirm
- You need an audit trail: sign-off packages, compliance reviews, or post-incident investigations benefit directly from session notes
- Time is limited and you must prioritise — charter time-boxes force the team to agree on what matters most before anyone starts clicking
- You're onboarding a new tester to an unfamiliar area — a charter gives them a mission and a boundary without needing full system knowledge
- Regression suites aren't catching enough — ET with charters targets the seams automated regression misses: edge cases, integration points, unexpected user paths
✗ Skip it when
- The test objective is fully deterministic and repeatable — pure regression confirmation is better served by a scripted test case or automated check
- The area has zero ambiguity and a complete happy-path spec — writing a charter adds overhead for no additional insight
- You're smoke-testing a build in under 10 minutes — informal sanity checks don't warrant charter formality; a simple checklist suffices
- The output requires exact pass/fail reproducibility for a certification test — ET findings are exploratory by nature and may not reproduce on demand
- You have no tester judgement available — charters rely on skilled, adaptable testers; assigning a charter to someone with no domain context without a debrief produces low-value sessions
Context guide
How the right level of exploratory testing charter effort changes based on project context.
| Context | Priority | Why |
|---|---|---|
| Benefits NZ, Revenue NZ, or CoverNZ benefit/entitlement systems — any sprint touching payment calculation or eligibility logic | Essential | Financial errors affect vulnerable people and trigger CCCFA or Social Security Act obligations. Session notes are the audit trail that demonstrates due diligence — not a nice-to-have. |
| HealthNZ or FamiliesNZ clinical/case management platforms — new feature or changed workflow | Essential | Patient safety and child welfare decisions depend on data correctness. Charters with documented session notes provide the evidence base that clinical governance and Privacy Act 2020 obligations require. |
| Harbour Bank or Pacific Bank transactional banking — pre-release sprint with changed payment or authentication flows | Essential | RBNZ prudential requirements and internal audit expect evidence of structured testing on any change to payment rails. Charters + session notes are the artefacts that satisfy that expectation. |
| TransitNZ (TransitNZ) driver licensing or vehicle registration — new integration with third-party identity or payment provider | High | Integration seams between AoG shared services and agency systems are where defects cluster. Charters targeting those boundaries surface failures scripted regression misses. |
| Spark or Pacific Air customer portal — mature product, stable sprint with UI copy and styling changes only | Medium | Visual-only changes carry low logical risk. One short charter targeting cross-browser rendering and responsive layout is proportionate; full SBTM overhead is not justified for a label rename. |
| Internal tooling or admin panel — low-traffic, staff-only CRUD interface with no financial or health data | Low | Risk surface is narrow and impact is contained to internal staff. A simple checklist or smoke test suffices; the charter overhead outweighs the benefit in low-stakes, reversible contexts. |
Trade-offs
What you gain and what you give up when you choose exploratory testing charters.
| Advantage | Disadvantage | Use instead when… |
|---|---|---|
| Produces a defensible audit trail — session notes record coverage and gaps explicitly, which satisfies sign-off requirements at NZ government agencies and regulated entities without requiring scripted test cases for every scenario. | Coverage is tester-dependent — two equally experienced testers running the same charter will not explore the same paths. Reproducibility is limited; a charter session cannot be handed off and replayed like a scripted test. | The objective is exact pass/fail confirmation of a known requirement — a scripted test case or automated check is more appropriate because it is repeatable and independent of tester skill. |
| Finds the defects scripted tests miss — charters are specifically designed for the seams, edge cases, and unexpected user paths that nobody thought to write a test case for, which is where real-world production defects at NZ fintechs and insurers tend to cluster. | Results are harder to measure — a charter session produces qualitative findings rather than a pass/fail count; stakeholders accustomed to scripted test metrics may push back on coverage claims that rely on session notes. | Stakeholders require a quantified pass/fail percentage for compliance reporting — scripted test cases with explicit expected results give the unambiguous numbers an audit or certification body expects. |
| Adapts in real time — when a session reveals an unexpected risk, the tester can redirect and record the detour; no change request, no re-approval cycle. This makes charters ideal for late-sprint sprints where requirements drift up to the last day. | Requires skilled testers — a charter with a vague approach ("explore the payment screen") produces a vague session. The value of the technique scales directly with the tester's domain knowledge, heuristic vocabulary, and risk intuition. | The tester has no domain context and there is no time for a debrief — a checklist-based test guides a less experienced tester through known checks without relying on intuition the tester has not yet developed. |
| Fast to set up — a well-formed charter takes 5–10 minutes to write; no test case authoring, no step-by-step documentation until the session note. This gives teams a structured testing approach even under tight sprint timelines. | Session notes require discipline under pressure — when release pressure peaks, testers shortcut the "Not covered" row or write the note from memory after the debrief. This destroys the audit value and introduces optimism bias into coverage claims. | The team has no testing culture around documentation — a checklist-based approach builds the documentation habit first; charters are most effective once the team already values recording what was and was not tested. |
Enterprise reality
How exploratory testing charters change when you are coordinating 200–300 developers across 10+ squads in a NZ enterprise programme
- Charter authoring moves from individual testers to a dedicated test lead or QA chapter lead — at scale, squads write charters independently and a test coordinator runs a weekly charter review to prevent duplication, catch coverage gaps between squads, and escalate cross-squad risk areas. Ad hoc charter writing by solo testers, fine in a 5-person team, creates invisible blind spots across a 30-squad programme.
- Revenue NZ's transformation programme (moving ~12 million taxpayer accounts to START) requires charter-produced session notes as mandatory artefacts for every sprint touching payment calculation, entitlement rules, or data migration logic — not optional documentation but gate criteria. The Privacy Act 2020 obligation to protect individual taxpayer data and the NZISM requirement for evidence-based assurance mean "we did exploratory testing" without session notes does not satisfy audit. Notes are stored in Jira alongside the associated Epic, not in personal drives.
- Tooling at volume means Xray or Zephyr Scale integrated into Jira for charter and session note management, PractiTest or TestRail for cross-squad coverage matrices, and Confluence for shared charter template libraries — not Google Docs. Teams that use ad hoc tools cannot aggregate session notes into a programme-level coverage view, which is the artefact executives and auditors ask for at release gate reviews.
- Cross-squad coordination requires a daily or twice-weekly charter sync between squad testers when multiple squads own different layers of the same end-to-end flow — common in microservices or SAFe PI structures. Without it, Charter A (Squad 1: payment API) and Charter B (Squad 4: payment UI) cover the same happy path twice while the integration seam between them goes unchartered. Assigning integration-boundary charters explicitly to a named squad is the only reliable fix.
◆ What I would do
Professional judgement — when to reach for exploratory testing charters, when to skip them, and what to watch for.
The bottom line: A charter is not bureaucracy — it is the difference between saying "I tested that" and being able to show what you tested, how, and what you ran out of time on. In any NZ organisation where an audit, incident review, or stakeholder sign-off could question your testing, the charter and session note are the only evidence that survives memory.
9 Best Practices
- ✓ Write charters before the sprint starts, not the morning of the release — good charters come from risk conversations with devs and POs, not last-minute guesswork
- ✓ Keep each charter to one area, one approach, one information goal — if you're exploring payment validation AND session management, that's two charters, not one
- ✓ Set a time box and honour it — when time expires, stop, write the session note, and hand over any uncovered items explicitly; a 90-minute overrun is not a completed charter
- ✓ Record the "Not covered" section every single session, even when you finish on time — there are always edge cases you didn't reach, and naming them is a decision input for the Test Lead
- ✓ Debrief verbally after each session — the 10-minute debrief with a Test Lead or peer surfaces insights that never make it into written notes, and keeps the testing strategy adaptive
- ✓ Use risk-based prioritisation to sequence charters — the area most likely to fail or most costly if it fails gets the first charter; don't start with the happy path
- ✓ Make the approach specific, not generic — "using boundary values on NZD transfer amounts (min $1, max $100,000)" is useful; "testing the amount field" is not
- ✓ Cross-reference charters with your test coverage matrix so you can show what percentage of the risk surface each charter addressed
- ✓ When a session produces an unexpected finding outside the charter scope, note it and spawn a new charter rather than spiralling off-course — this keeps sessions accountable and ensures the detour gets proper coverage
- ✓ Store session notes alongside the charter in your test management tool (Jira, Zephyr, TestRail) — loose notes in Slack or email are invisible to auditors and future testers
10 Common Misconceptions
❌ Myth: A charter is just a test plan by another name.
Reality: A test plan documents scope, schedule, resources, and strategy for an entire test effort. A charter is a 2–3 sentence mission statement for a single 45–90 minute exploration session. The level of granularity is completely different. A test plan might reference 20 charters; a charter never references a test plan. Conflating them causes testers to over-engineer charters into multi-page documents — which defeats the purpose of the technique.
❌ Myth: If you deviate from the charter during a session, the session is invalid.
Reality: Deviation is expected and often valuable — that's the difference between exploratory testing and scripted testing. What matters is that deviations are recorded in the session note so coverage is transparent. A tester who spots a critical defect outside the charter scope and investigates it has done exactly the right thing; the session note captures the detour and the Test Lead can spawn a follow-up charter for the original uncovered area.
❌ Myth: Charters only work for manual testing; they're irrelevant once you have automation.
Reality: Automation covers the paths you know about. Charters are specifically designed to discover what you don't know — the edge cases, integration seams, and emergent behaviours that fall between automated test cases. High-performing teams that use extensive automation still run charter-based exploratory sessions because their automated suites can only check what was anticipated. At NZ fintechs and insurers post-incident reviews regularly show that the defect was in an area no automated test covered — and that no charter had been written for it either.
11 Now You Try
Write 3 charters for a pre-release exploratory session on a NZ government online driver licence renewal service. The service allows: address update, payment, photo upload, and document download. You have 3 hours total. Each charter should follow the format: Explore [area] using [approach] to discover [information], and include a time box and the risk area being targeted.
Why teams fail here
- Writing the charter after the session — retro-fitting scope to what was actually done destroys the audit value and hides real gaps from the Test Lead and product owner.
- Making the approach too vague — "explore the payment screen" is not an approach; "explore the payment screen using boundary values on NZD transfer amounts (min $1, max $100,000, 3-decimal edge cases)" is. Vague charters produce vague sessions.
- Skipping or blanking the "Not covered" row — this single omission turns a useful session note into an optimistic fiction; it is the row that gives the next tester and the Test Lead actionable information.
- Treating a charter as a script — if you find yourself listing numbered steps inside a charter, you have written a test case; stop, delete the steps, replace them with an information goal, and let the tester decide the path.
Key takeaway
A charter does not constrain exploration — it makes exploration accountable: you start with a mission, you record what you found and what you did not reach, and you hand the Test Lead something they can actually use to make a release decision.
How this has changed
The field moved. Here is how Exploratory Testing Charters evolved from its origins to current practice.
Session-Based Test Management (SBTM) developed by James and Jonathan Bach at ST Labs. The session report is the first formal accountability mechanism for exploratory testing — captures what was explored, what was found, and how time was spent. Charters are implicit in SBTM sessions.
Jonathan Bach publishes SBTM methodology. The test charter becomes an explicit artefact — a short mission statement that focuses a session without scripting the path. "Explore the payment module using valid and invalid credit cards with a focus on error handling" is a charter, not a script.
Ministry of Testing and the broader community begin publishing charter libraries, heuristic mnemonics, and session documentation templates. Charters become the accepted vocabulary for collaborative exploratory testing.
Rapid Software Testing (RST) by James Bach and Michael Bolton refines the relationship between charters, missions, and sessions. Charters are distinguished from scripts: a script specifies steps, a charter specifies a mission. The distinction matters for test design thinking.
AI tools can suggest charters from risk areas, user stories, and past defect patterns — automating the starting point for exploratory sessions. The experienced tester's value is in following the evidence wherever it leads, recognising anomalies, and maintaining intellectual curiosity — none of which a charter can prescribe.
12 Self-Check
Click each question to reveal the answer.
Interview Questions
What NZ hiring managers ask about Exploratory Testing Charters — and what strong answers look like.
Write a test charter for exploring a new patient appointment booking system at a NZ DHB.
Strong answer: Charter: "Explore the appointment booking flow for patients booking their first appointment after referral, focusing on how the system handles edge cases in the referral pathway — including expired referrals, referrals to the wrong specialty, and urgent vs routine referral priority handling. Time-box: 90 minutes. Report: defects found, questions for clinical SME, and areas that need more investigation." A good charter has a clear mission (what to explore), a focus area (what to pay most attention to), and a time box. It does not prescribe steps — that is left to the tester's judgement during the session.
Mid/Senior
How do you document an exploratory testing session without losing the freeform nature of the exploration?
Strong answer: I use a lightweight session sheet: mission (the charter), time box (start/end), areas explored (brief notes as I go), defects found (linked to bug reports), questions raised (things I could not verify), and a debrief note (what I found, what I skipped, what needs more investigation). I take notes in real time rather than reconstructing afterwards — a 30-second note while exploring preserves the cognitive state. I use screen recording for complex interactions so I can prove reproduction without relying on memory. The session sheet is not a test script — it is a map of where I went and what I found.
Junior/Mid
Q1: What is the difference between a charter and a test script?
A test script specifies exactly what steps to take and what to check at each step. A charter defines the mission — the area to explore, the approach, and the information goal — but leaves the specific steps to the tester's judgement during the session. A charter enables adaptation; a script does not.
Q2: Why is the "Not covered" section of a session note important?
It documents the gaps explicitly rather than leaving them invisible. When a defect slips through, the question is always "was this area tested?" A session note that records "3-decimal-place amounts not covered — ran out of time" means the team can make an informed decision: accept the risk, schedule another session, or block the release. Undocumented gaps are invisible risks.
Q3: You have 4 hours before a release. How many charters should you write, and how long should each time box be?
As a guide: 60-minute time boxes work well — they're long enough for productive exploration but short enough to keep focus. Four charters of 60 minutes each. Spend 15 minutes upfront writing charters and reviewing with the team to prioritise highest-risk areas, then 45 minutes per session. Leave the last 15 minutes for writing up session notes across all four.
Q4: Your team is preparing a release for an Benefits NZ benefit payment portal. The payment calculation logic was recently changed by a developer. Which charter would you prioritise first and why?
A: Prioritise a charter targeting the changed payment calculation logic using boundary values and equivalence partitioning — for example, exploring payment amounts at minimum entitlement, maximum entitlement, and known edge cases such as part-week periods or benefit rate changes. Changed code carries the highest regression risk, and financial calculation errors in an Benefits NZ context can cause incorrect payments to vulnerable people, creating both legal exposure and public trust damage. A second charter on the payment summary screen should follow to verify the displayed figures match the calculated values end to end.
Q5: What is the key difference between a testing charter and a checklist-based test?
A: A checklist prescribes a fixed set of items to verify — the tester works through the list and confirms each item passes or fails. A charter sets a mission (area, approach, information goal) and then trusts the tester's judgement to decide what to explore within that boundary. Checklists are best for ensuring known coverage; charters are best for discovering unknown defects. In practice, a checklist confirms what you already know to check; a charter is designed to surface what you didn't know to check in the first place. Many experienced testers use both: a checklist for regression confirmation and a charter for risk-based discovery in the same release cycle.
Q6: A developer tells you "we already have 80% automated test coverage, so we don't need exploratory charters for this release." What is wrong with this reasoning and how do you respond?
A: Automated coverage measures the paths the team already anticipated and encoded as tests — it says nothing about the paths nobody thought to automate. Charters are specifically designed to discover the defects that fall outside known test cases: integration seams, unexpected user paths, edge cases in new or changed logic, and emergent behaviour under real-world conditions. At NZ fintechs and government agencies, post-incident reviews regularly show the defect was in a gap between automated tests, not in a covered path. A good response: "The 80% coverage is great for regression confidence. Charters give us 90 minutes targeting the 20% of risk your automated suite wasn't built to find — especially the payment calculation changes from this sprint."
Q7: In an interview, you're asked to describe a time you used exploratory testing. You describe 3 hours of ad hoc clicking that found a bug. The interviewer asks "how did you document your coverage?" What should you have done differently, and what would a strong answer sound like?
A: Ad hoc clicking without a charter produces no defensible evidence of coverage — you found one bug, but you cannot say what you covered or what you missed. A strong answer describes charter-based exploratory testing: "I wrote three 60-minute charters before the session — one for the payment flow using boundary values, one for session management across devices, and one for data display accuracy. Each session produced a note recording what I explored, bugs found, and what I didn't reach. At the end I could tell the Test Lead exactly what was covered and what needed another pass." This shows you understand the technique as a structured discipline, not just free-form clicking, which is the distinction most interviewers at NZ-based testing roles are probing for.
13 ISTQB Mapping
ISTQB CTFL v4.0, Section 5.1.3 — Charter-based exploratory testing (also known as Session-Based Test Management, SBTM). Foundation-level testers are expected to understand the charter format, session notes, and the role of exploratory testing within a test strategy.
Related: CTAL-TA v3.1.2, Section 3.1 — Exploratory testing as a dynamic technique; integration with risk-based test planning.