Exploratory Testing
Simultaneously design, execute, and learn. Exploratory testing isn’t random clicking — it’s structured investigation guided by a charter and the tester’s growing understanding of the system.
1 The Hook
A team in Christchurch ships a school-enrolment portal with a thick set of scripted test cases, all green. Every field validated, every form submitted, every box ticked. Two days after go-live, a parent rings the school in tears: she added her second child, went back to edit the first child’s details, and the portal silently overwrote child one with child two’s information. Both records now showed the same kid.
No scripted test caught it, because no script told anyone to add a child, then go back and edit an earlier one. The script tested each step in isolation; the bug lived in the messy real-world sequence a human actually does. A tester following their nose — "what happens if I do these out of order?" — would have found it in ten minutes.
That is the gap exploratory testing fills. Scripts check the paths you already thought of. Exploratory testing finds the paths you didn’t — by letting a thinking tester design the next test based on what the last one just revealed.
2 The Rule
Design and run tests at the same time, steering each new test by what the last one taught you — but keep it disciplined with a charter, a time box, and notes. It is structured investigation, not random clicking.
3 The Analogy
A detective working a case, not a clerk reading a form.
A clerk processes the same checklist for every file: tick, tick, tick, done. A detective starts with a brief — "find out how the intruder got in" — and lets each clue decide the next question. A scuff on the windowsill sends them to the garden; a footprint in the garden sends them to the fence. They are not wandering aimlessly; they have an objective and they take notes, but the path is shaped by what they keep finding.
Exploratory testing is detective work on a build. The charter is the brief, the notes are the case file, and every surprising result is a clue that points you at the next test. A scripted test plan is the clerk’s checklist — useful, but it never follows a clue it wasn’t told about.
Common Mistake vs What Works
Treating exploratory testing as unstructured "clicking around" — no charter, no time box, no notes. The session ends, management asks what was covered, and the tester can only say "I found a bug with the checkout." Nobody can reproduce the path that led there. On NZ government and banking projects this is a dead end: if you can't show what you tested, it didn't happen.
Write a one-sentence charter before you open the app — "Explore the checkout flow to discover how the system handles concurrent sessions" — set a 90-minute time box, and take live notes as you go. At the end, write a brief debrief: what you covered, what you found, and what you deliberately skipped. The charter is lightweight; the debrief is what makes the session auditable and lets the next tester pick up where you left off.
Exploratory testing is structured detective work — you write a charter, time-box the session, take live notes, and debrief — not random clicking. Reach for it when specs are incomplete, after a major refactor, or any time you need to find bugs that scripted tests cannot see. The mistake most testers make is running sessions without a session sheet, which turns disciplined investigation into an invisible black box that management cannot audit or reproduce.
The failure mode nobody warns you about is charter theatre. Teams write a charter, feel organised, then spend 90 minutes clicking wherever curiosity takes them anyway — because the charter was too broad to actually constrain anything. “Explore the payment flow to find bugs” is not a charter; it is a blank cheque. I have reviewed session sheets on Revenue NZ and CoverNZ portal projects where the charter looked fine on paper but the tester covered the same happy-path screens three times and never touched the edge cases named in the information goal. A real charter forces you to say no to interesting distractions. If your charter does not make you reluctant to follow at least one shiny thing mid-session, it is not specific enough.
What it is
Exploratory testing (ET) is an approach where test design and test execution happen at the same time, guided by the tester’s real-time understanding of the system. It’s most powerful when specifications are incomplete, the system is complex, or you want to find bugs that scripted tests miss.
The core idea: you learn about the system by testing it, and you use what you learn to guide your next test. Unlike scripted testing, you’re not following a predetermined script — you’re investigating.
Not unstructured: exploratory testing is often confused with "ad hoc" testing. Ad hoc has no structure. Exploratory testing has a charter, a time box, and a session report. It’s disciplined investigation, not random clicking.
The charter
A charter is a short statement of what you’re exploring and what you’re looking for. It gives the session focus without scripting every step.
Format: "Explore [target] using [resources/approach] to discover [information]"
| Charter | Focus area |
|---|---|
| Explore the checkout flow using edge-case payment data to discover how the system handles declined cards and partial payments | Payment error handling |
| Explore user profile settings using a newly registered account to discover what can be changed and how data is validated | Account management |
| Explore search functionality using unusual characters and very long strings to discover input handling | Input validation |
Running a session
- Write your charter — define scope and focus before you start.
- Set a time box — typically 60–90 minutes. Long enough to go deep, short enough to stay focused.
- Explore and take notes — what you tried, what you noticed, what surprised you. Notes are key evidence.
- Log bugs as you find them — don’t just mentally note them; write them up properly during the session.
- Debrief — at the end, review what you found, what you didn’t cover, and what new charters to create.
Session reporting (SBTM)
Session-Based Test Management (SBTM) formalises exploratory sessions. Each session produces a Session Sheet with: the charter, duration, tester, bugs found, coverage notes, and a ratio of time spent on setup vs exploration vs bug investigation.
This gives management visibility into exploratory work — it’s no longer a black box.
When to use it
- Early in a project when specs aren’t final — learn the system
- After major refactors — systematic regression may miss emergent bugs
- When you suspect bugs that scripted tests won’t find
- New product areas — before writing formal test cases
- Time-constrained testing — maximum value per hour
ISTQB mapping
| Ref | Topic |
|---|---|
| 4.4.3 | Exploratory Testing — simultaneous design and execution |
| 4.4.3 | Charter-based sessions, time-boxing, session reporting |
| CTFL-AT | Exploratory testing in Agile teams — sprint-level application |
Practice this technique: Try Senior Practice 10 — Exploratory charter, Senior Practice 03 — Async & loading states.
NZ example — Charter for testing a NZ government service
NZ government services (Revenue NZ myIR, TransitNZ TransitNZ portal, RealMe) have specific testing considerations that automated tests won’t have.
| Charter element | Detail |
|---|---|
| Charter | Explore the payment submission flow using NZ bank account formats (including Rural Bank, KiwiFirst Bank, TSB) to discover whether account validation accepts all valid NZ account structures and rejects invalid ones |
| Session note: account format | Does the system accept the BB-bbbb-AAAAAAA-SS format? Does it accept accounts from all NZ banks (KiwiFirst Bank, TSB, SBS have different prefix ranges)? |
| Session note: currency | Does it display amounts in NZD correctly (two decimal places, $ symbol)? |
| Session note: public holidays | Does it handle the NZ public holiday edge case — payments submitted on Waitangi Day, ANZAC Day? |
The charter keeps the session focused — you’re not randomly clicking, you’re investigating a specific quality risk with NZ-specific knowledge that automated tests won’t have.
4 Industry Reality
- Charters are written on the fly, or not at all. Textbooks show polished charters; on real projects you often scribble a two-line focus area in a sticky note 30 seconds before you start. Senior testers still write something down — even a messy one forces you to articulate what you're looking for.
- Time boxes get crushed by context-switching. Your 90-minute session gets interrupted by a Slack message, a standup, and a developer wanting a quick look at a bug. Experienced testers reset the clock, note what they covered before the break, and treat each uninterrupted stretch as its own mini-session.
- Legacy codebases make session notes essential. You click through a decade-old NZ payroll system and find something odd — but if you didn't note exactly how you got there, you'll never reproduce it. The habit of live note-taking saves you hours of re-investigation.
- SBTM session sheets rarely match the theory. Most teams use a simplified version: a charter, a rough time split, a list of bugs, and what wasn't covered. Full DEBRIEF meetings are rare. What matters is that someone can look at your notes and understand what you tested and what you skipped.
- Exploratory testing is often what saves a release. The scripted regression suite is green, QA signs off, and then one tester does a 30-minute charter on "anything that feels weird" — and finds the showstopper. This happens regularly enough that high-performing NZ QA teams schedule unstructured ET time as a standing line item, not an afterthought.
Senior engineer insight
The session that surprised me most wasn’t on a flashy fintech build — it was an CoverNZ injury-claims portal that had been running for eight years. The scripted regression was watertight, dev confidence was high, and I had one hour before sign-off. I wrote a charter targeting the claim-amendment flow using a claimant who had two overlapping injuries filed the same week, because that felt like a corner nobody had scripted. Within twenty minutes I found that amending one claim silently reset the lodgement date on the other — a bug that would have delayed weekly compensation payments for real people. No automated test caught it because nobody imagined the sequence; my charter forced me to imagine it.
The most common mistake I see from graduates is writing a charter that sounds focused but isn’t — “Explore the payment flow using edge cases to find validation bugs” is still a blank cheque; a charter that actually constrains your session names the specific data, the specific sequence, and the specific risk you’re hunting.
From the field
A team building the TransitNZ permit system assumed testers had covered the “special vehicle” permit path because it existed in the scripted regression suite. When we ran a focused charter targeting multi-leg journeys — a single oversize load that crosses three regional boundaries and requires permits from two different councils — we found the system correctly generated permit one, silently dropped permit two, and displayed no error. The operations team would have sent a truck onto a state highway with incomplete paperwork. The fix took four hours; finding it without the charter would have taken four weeks of production complaints. Lesson: anything involving cross-boundary or multi-party workflows in NZ government systems is almost always under-scripted, because the person writing the test cases imagined one region at a time.
5 When to Use It — and When Not To
✓ Use it when
- Specs are incomplete or still evolving — ET lets you learn the system before writing formal test cases
- A major refactor just landed and you want to find emergent bugs that regression scripts won't reveal
- You're onboarding to a new codebase or feature area and need to build mental models fast
- Time is very short and you need maximum bug yield per hour — a focused 60-minute charter often beats three hours of scripted execution
- The domain is complex or unpredictable (e.g. NZ government portals, financial workflows, healthcare forms) where edge-case sequences are hard to script in advance
✗ Skip it when
- You need a compliance audit trail — regulators and accreditation bodies require traceable scripted test cases, not session notes
- The test is purely data-driven with thousands of permutations — equivalence partitioning or property-based testing is more systematic
- You're doing final regression on a stable, well-understood system — scripted tests are faster and more consistent
- The tester has no domain knowledge — ET relies on tester intuition; a brand-new grad on a complex domain will just click without learning much
- Stakeholders need a percentage-complete metric — ET doesn't map cleanly to "X of Y test cases passed" reporting
Context guide
How the right level of Exploratory Testing effort changes based on project context.
| Context | Priority | Why |
|---|---|---|
| Agile sprint — new feature with ambiguous or evolving requirements | Essential | Specs shift during the sprint; ET lets you learn the system and find edge cases before formal test cases can even be written. |
| Small NZ startup — rapid release cadence, thin QA team | High | Maximum bug yield per hour when scripting capacity is limited; a focused 60-minute charter often outperforms a full scripted regression on new code. |
| Post-production bug investigation — defect reported by a real user | Essential | Scripted tests didn’t catch it; the bug lives in a real-world sequence nobody scripted. ET lets you follow the clue and map the blast radius before the fix lands. |
| Large enterprise — stable, mature product with a full regression suite | Medium | Scripted regression covers the known paths efficiently; reserve ET for new feature areas and high-risk change points rather than running it across the whole suite. |
| Regulated system (HealthNZ, Revenue NZ, CoverNZ, RealMe identity) — compliance sign-off required | Supplement only | Auditors require traceable scripted test cases with documented pass/fail evidence; run ET alongside the formal suite to catch edge cases, but it cannot replace it for sign-off. |
| Legacy migration — replacing or re-platforming an existing NZ system | High | Legacy behaviour is often undocumented; ET sessions with a domain expert surface the real rules the old system enforced before they get accidentally dropped in the new build. |
| Usability & accessibility review — real users on NZ government or public-facing services | High | Persona-based charters (rural user, senior, non-standard name) find accessibility and localisation issues that no scripted test thinks to cover; critical before public launches. |
Trade-offs
What you gain and what you give up when you choose Exploratory Testing.
| Advantage | Disadvantage | Use instead when… |
|---|---|---|
| Finds bugs in real-world sequences no script anticipated — the paths users actually take, not the paths testers planned for | Coverage is hard to quantify; stakeholders cannot easily see “what percentage is done” from a session sheet the way they can from a test case pass-rate | You need to report a percentage-complete metric to sign off a release gate; use scripted test cases instead |
| Maximum bug yield per hour when time is short — a sharp 60-minute charter often outperforms three hours of scripted regression on a complex new feature | Quality scales directly with the tester’s domain knowledge and intuition; a grad on an unfamiliar CoverNZ claims workflow will cover far less ground than a senior who knows the edge cases | The tester is new to the domain; pair with a domain expert first or write scripted cases from a spec so the tester has a map before exploring |
| Generates deep system understanding fast — exploratory sessions are the fastest way to build a mental model of a codebase or feature you’ve never seen | Does not produce the traceable pass/fail evidence that compliance frameworks (health sector, financial regulators, Benefits NZ audit trails) require | You’re testing for regulatory sign-off on a HealthNZ or Revenue NZ system; use formal scripted test cases with documented evidence instead |
| Adapts in real time — if the system behaves unexpectedly mid-session, you can pivot the charter instead of finishing a script that no longer tells you anything useful | Reproducibility depends on note-taking discipline; a session run without live notes produces findings that are hard to hand off, reproduce, or defend post-release | The team has a history of poor note-taking; introduce SBTM session sheets before relying on exploratory testing as a primary technique |
| Thrives in complex, unpredictable domains — NZ government portals, financial calculators, and healthcare workflows have interaction effects too numerous to enumerate in advance | Poor fit for data-driven permutation testing; when you need to verify thousands of input combinations systematically, equivalence partitioning or property-based testing gives better coverage per test | You need to validate a tax rate table or Revenue NZ threshold matrix; use equivalence partitioning and boundary value analysis instead |
6 Best Practices
- ✓ Write the charter before you open the app. Even two sentences. Once you're in the UI, curiosity pulls you everywhere; the charter is the anchor that keeps you on track.
- ✓ Take live notes, not retrospective notes. Type as you go — "tried X, noticed Y, question: does Z also do this?" Notes written after the session are reconstructions, not records.
- ✓ Follow clues, then come back. When something surprising appears, note it and keep it on a "follow-up" list. Don't abandon your current thread — finish the thought, then investigate the clue.
- ✓ Keep bugs atomic. Write up each bug the moment you find it, with exact steps to reproduce. Don't batch them at the end — you'll forget how you got there.
- ✓ Use heuristics as prompts. The SFDPOT heuristic (Structure, Function, Data, Platform, Operations, Time) gives you a mental checklist of angles to investigate when inspiration dries up mid-session.
- ✓ Declare explicitly what you did NOT cover. The coverage gaps in your session note are often more valuable than the bugs — they become the next round of charters.
- ✓ Pair up for unfamiliar domains. Two testers in a paired ET session bring different mental models. A developer and a tester pairing is especially powerful — the dev knows where the bodies are buried.
- ✓ Respect the time box. When the timer goes off, stop and debrief — don't just keep going. The debrief is where you decide if a new session is warranted, which prevents scope creep from eating your whole day.
- ✓ Generate new charters, not just bugs. A good session ends with two or three follow-up charters. If you only found bugs but created no new questions, you probably went too shallow.
- ✓ Vary your persona. In NZ government or banking contexts, run sessions as different user types — a rural user with a slow connection, a senior citizen, a user with a non-standard name. Charters tied to specific personas surface accessibility and localisation bugs scripted tests miss entirely.
7 Common Misconceptions
❌ Myth: Exploratory testing is just clicking around randomly — it's what you do when you don't have time to write proper tests.
Reality: Exploratory testing is a disciplined technique with a defined structure: a charter (your brief), a time box, live notes, and a debrief. "Random clicking" is ad hoc testing — a different (and less useful) thing. ET is what you do when you want maximum learning per unit of time, not when you're cutting corners. Many senior testers consider a skilled ET session harder to run well than scripting, because it demands real-time decision-making under time pressure.
❌ Myth: Exploratory testing replaces scripted testing — if your testers are good enough, you don't need test cases.
Reality: ET and scripted testing answer different questions. Scripted tests confirm known behaviour consistently and provide an audit trail. ET finds unknown problems and builds system understanding. High-performing teams use both: scripted regression for stable, well-understood flows, and ET sessions for new features, complex domains, and anything that smells risky. Replacing one with the other leaves a gap.
❌ Myth: You can't measure exploratory testing, so managers hate it.
Reality: Session-Based Test Management (SBTM) was specifically invented to make ET measurable and visible. A session sheet records charter, duration, time split, bugs found, and coverage gaps. Management gets the same visibility they'd get from a scripted test run: what was tested, what wasn't, what was found, and what to do next. The problem is usually not ET itself — it's ET without session reporting, which really is a black box.
8 Now You Try
Three graded exercises — spot, fix, then build. Write your answer, run it for AI feedback, then compare to the model answer.
Read the two session descriptions below. Decide which one is genuine exploratory testing and which is just ad hoc clicking, and name the specific things that make the difference.
Session B: "I just clicked around the site for a while after the build dropped, tried a few things that came to mind, found something odd but I’m not totally sure how I got there, and moved on."
Show model answer
Session A is exploratory testing. Session B is ad hoc.
What makes the difference:
- Charter: A has a stated focus ("how the listing flow handles draft listings"); B has none.
- Time box: A is bounded at 90 minutes; B is "a while".
- Notes / reproducibility: A recorded what was tried, so findings are reproducible; B "isn’t sure how I got there" — a bug nobody can reproduce.
- Output: A produced logged bugs, coverage notes, and follow-up charters; B produced an unverifiable "something odd".
The point: exploratory testing is disciplined investigation — charter, time box, notes, debrief. Ad hoc has structure removed. Same freedom to follow clues; very different rigour.
The charter below, written for an Revenue NZ myIR portal, is too vague to guide a session. Rewrite it in the Explore / Using / To discover format so it has a specific target, a concrete approach, and a real information goal.
"Explore the whole portal using various inputs to find bugs."
Rewrite as a strong charter:
Show model answer
Strong charter (one good version): Explore — the GST return submission flow Using — returns with edge-case amounts (zero GST, a refund/negative position, and an amount above the field’s apparent limit), plus saving a draft and resuming it later To discover — whether the portal calculates and displays the GST position correctly, preserves a saved draft accurately, and gives error messages specific enough for a user to self-correct. Why it is stronger: - Specific target: "the GST return submission flow", not "the whole portal". - Concrete approach: named edge-case data and a draft-resume sequence, not "various inputs". - Real information goal: named quality risks (calculation correctness, draft preservation, error-message quality), not "find bugs". A charter another tester could pick up and run without asking what you meant.
You have just finished a 60-minute exploratory session on a TransitNZ TransitNZ vehicle-licence (rego) renewal flow. Write the SBTM session sheet for it: charter, tester, duration, a rough time split (setup vs exploration vs bug investigation), bugs found, coverage notes, and any follow-up charters. Make it realistic.
Show model answer
SBTM session sheet (example): Charter: Explore the rego renewal flow using vehicles with different statuses (current, lapsed, on hold) and overseas/no-email accounts to discover whether the flow correctly prices the renewal term and handles accounts that can’t receive an email confirmation. Tester / Duration: A. Tester / 60 minutes. Time split: setup ~15% (test account, sample vehicles), test design & execution ~65%, bug investigation ~20%. Bugs found: 1. Lapsed-vehicle renewal showed the current-vehicle price for one frame before correcting — confusing flash of wrong total. 2. Account with no email reached the confirmation page with no way to get a receipt; no warning earlier in the flow. Coverage notes: covered 3-, 6-, and 12-month terms for current and lapsed vehicles, and the no-email path. Did NOT cover payment decline handling or the "on hold" vehicle status — out of time. Follow-up charters: - Explore the renewal payment step using declined and partially-authorised cards to discover how failures are surfaced. - Explore "on hold" vehicle renewals to discover whether the flow blocks or misprices them. A good sheet makes the session auditable: a manager can see what was covered, what wasn’t, and what to do next — exploratory work is no longer a black box.
Why teams fail here
- They write a charter, feel organised, then ignore it mid-session — curiosity pulls them to a flashy bug in an unrelated feature, the time box expires, and the original information goal was never investigated.
- No live notes means no reproducibility. On Benefits NZ benefit-calculator sessions it is not uncommon to find a miscalculation, continue exploring, then be completely unable to reconstruct the exact input sequence that triggered it — the bug dies with the session.
- Teams use ET as a substitute for thinking about what is actually risky. On council rates portals they run a generic “explore payments” charter every sprint instead of targetting the specific changes that landed — so they keep re-covering the same happy paths and never touch the new code.
- The session ends with bugs but no follow-up charters, so coverage gaps become invisible. The next tester starts a fresh session with no map of what was already explored and unknowingly duplicates two hours of prior work.
Key takeaway
A charter without a specific information goal is just permission to click around — the discipline is naming the exact risk you are hunting before you open the app.
Self-Check
Click each question to reveal the answer.
How this has changed
The field moved. Here is how Exploratory Testing evolved from its origins to current practice.
James Bach and Cem Kaner develop the intellectual foundations at Apple and elsewhere. Testing without a complete script is common in practice but has no recognised name or methodology. Critics call it "ad hoc testing" dismissively.
Cem Kaner coins the term "exploratory testing" and begins publishing. Session-Based Test Management (SBTM) is developed by James and Jonathan Bach to make exploratory sessions accountable and measurable.
Agile adoption rehabilitates exploratory testing — its flexibility suits sprints better than heavyweight scripted approaches. Ministry of Testing and the broader community publish charters, heuristics (SFDPOT, HICCUPPS), and session templates.
Exploratory testing gains formal recognition in ISTQB syllabi. Companies begin combining scripted regression automation with exploratory sessions for new features — each doing what it does best.
AI testing tools can explore systematically, but lack the contextual curiosity and model-based reasoning that makes human exploratory testing irreplaceable. Exploratory sessions increasingly focus on AI-generated systems where rules are implicit.
Q1: What is the core idea that separates exploratory testing from scripted testing?
Test design and execution happen at the same time. You learn about the system by testing it and use what you learn to choose the next test — instead of following a script written before you understood the system. That lets you find paths nobody thought of in advance.
Q2: How is exploratory testing different from ad hoc testing?
Exploratory testing has a charter, a time box, notes, and a debrief — it is disciplined investigation. Ad hoc has no structure. Both follow the tester’s judgement, but only exploratory testing produces reproducible findings and a record of what was covered.
Q3: What are the three parts of a good charter, and why does each matter?
A specific target (so another tester could pick it up), a concrete approach or data type (not "various inputs"), and a named information goal or quality risk (not just "find bugs"). Together they give the session focus without scripting every step.
Q4: What does Session-Based Test Management (SBTM) add, and why do managers value it?
SBTM produces a session sheet per session — charter, duration, tester, bugs found, coverage notes, and a time split across setup, testing, and bug investigation. It makes exploratory work auditable and visible, so it is no longer a black box to management.
Q5: Name two situations where exploratory testing is the right tool to reach for.
Early in a project when specs aren’t final (you learn the system as you test), and after a major refactor where systematic regression may miss emergent bugs. It also shines when you suspect bugs scripted tests won’t find, in brand-new product areas, and under tight time constraints where value-per-hour matters.
Q: Your team is releasing a new KiwiSaver voluntary contribution flow on a banking portal. The scripted regression suite is green. You have 90 minutes before the go/no-go call. How would you use exploratory testing, and what charter would you write?
A: Write a focused charter targeting the riskiest sequences scripted tests ignore — for example: “Explore the voluntary contribution flow using a mix of valid, over-limit, and non-standard contribution amounts, including mid-flow browser back-navigation and session timeout, to discover whether amounts are calculated correctly and whether interrupted flows leave the account in a consistent state.” Time-box it firmly at 90 minutes, take live notes, and log any bug immediately. The scripted suite confirmed expected paths; your charter hunts the real-world sequences a member would actually do — the kind the school-enrolment bug (two children, one overwriting the other) came from.
Q: What is the key difference between exploratory testing and error guessing?
A: Both are experience-based and unscripted, but they differ in scope and structure. Error guessing targets specific, predicted failure points — a tester thinks “this field probably breaks on null” and tests exactly that. Exploratory testing is a broader investigative mode: you start with a charter, follow clues wherever they lead, and continuously steer the next test based on what the last one revealed. Error guessing is a point; exploratory testing is a session. In practice, error guessing is a useful heuristic inside an exploratory session, not a replacement for one.
Q: A developer says, “We don’t need to document exploratory sessions — the tester knows what they tested, and that’s enough.” What is wrong with this view, and how do you respond?
A: This is the most common real-world trap. Without a session sheet, the work is a black box: management cannot see what was covered, coverage gaps cannot become follow-up charters, and bugs found cannot be reproduced because nobody recorded the exact path. In an NZ regulated context — CoverNZ, Revenue NZ, Benefits NZ — this also means there is no audit trail if a defect surfaces post-release. Session-Based Test Management (SBTM) was invented specifically to fix this: a charter, time split, bugs found, and what was not covered takes ten minutes to write and makes exploratory work as visible and defensible as any scripted test run.
Q: When should you NOT use exploratory testing, even if time is short and bugs are expected?
A: Skip exploratory testing when you need a traceable compliance audit trail — regulators and accreditation bodies (such as those overseeing HealthNZ or RealMe identity services) require scripted test cases with documented pass/fail evidence, not session notes. It is also the wrong tool when the test is purely data-driven with thousands of permutations (equivalence partitioning or property-based testing is more systematic), when stakeholders need a percentage-complete metric that maps to “X of Y test cases passed”, or when the tester has no domain knowledge of the area being explored — without intuition built on understanding, an ET session produces random clicking rather than targeted investigation.
Enterprise reality
50+ testers, quarterly release cycles, governance-heavy programmes
- Session-Based Test Management (SBTM) becomes mandatory rather than optional — with dozens of testers running parallel sessions, session sheets are the only way test leads and programme managers can see what has been covered without sitting in on every session. Auditors on regulated NZ programmes (HealthNZ, CoverNZ, Revenue NZ) explicitly ask for session logs as evidence.
- Charters are reviewed and approved by a test lead before sessions begin, not written ad hoc. In a 50-tester programme, duplicate coverage — six people independently exploring the same checkout flow — is a real cost. Charter review boards assign ownership across teams and flag gaps in the coverage map before a sprint closes.
- Exploratory findings feed a shared defect taxonomy used to improve the automated regression suite. When ET uncovers a class of bug (e.g. multi-party workflow state corruption), the pattern is handed to automation engineers as a new test template — so the next release catches that class automatically, without relying on human intuition every cycle.
- Time-boxed sessions are scheduled across the team in coordinated waves to avoid collision. With quarterly release cycles and a stabilisation window of two to three weeks, test leads publish a session schedule: which feature areas are open for ET, which testers own which charters, and when debrief notes are due. Uncoordinated ET at scale wastes capacity and leaves gaps that nobody notices until a post-release defect surfaces.
★ Interview Questions
What NZ hiring managers ask about Exploratory Testing — and what strong answers look like at each level.
Q: What is exploratory testing, and how is it different from scripted testing?
Strong answer: Exploratory testing means designing and executing tests at the same time, steering each new test based on what the last one revealed. Unlike scripted testing — where you follow a predetermined sequence written before you understood the system — exploratory testing lets you follow clues and discover paths nobody thought to script. You still keep it disciplined: you write a charter, set a time box, take live notes, and debrief at the end.
Grad / Junior
Q: What is a charter in exploratory testing, and what makes a charter weak versus strong?
Strong answer: A charter is a short brief that sets the scope before you open the app — typically written as "Explore [target] using [approach] to discover [information goal]." A weak charter is too broad: "Explore the portal to find bugs" gives you no anchor, so you end up clicking wherever curiosity takes you. A strong charter names a specific target, a concrete data type or approach, and a real quality risk to investigate — specific enough that another tester could pick it up and run it without asking what you meant.
Junior
Q: You’re testing an Revenue NZ myIR tax return flow. The scripted regression suite is green and you have 90 minutes before the go/no-go call. How would you structure an exploratory session?
Strong answer: I’d write a focused charter before touching the app — for example: "Explore the GST return submission flow using edge-case amounts (zero GST, a refund/negative position, an amount above the apparent field limit) and the draft save-and-resume sequence, to discover whether amounts are calculated correctly and whether interrupted flows leave the account in a consistent state." I’d time-box it firmly at 90 minutes, take live notes as I go, log each bug the moment I find it, and end with a brief debrief noting what I covered and what I didn’t. The scripted suite confirmed happy-path behaviour; my charter hunts the real-world sequences a taxpayer would actually do that nobody thought to script.
Senior
Q: When would you NOT use exploratory testing, even under time pressure with bugs expected?
Strong answer: Exploratory testing is the wrong tool when you need a traceable compliance audit trail — for example, testing a HealthNZ or RealMe identity service where regulators require scripted test cases with documented pass/fail evidence. It’s also a poor fit for purely data-driven testing with thousands of permutations (equivalence partitioning is more systematic), when stakeholders need a percentage-complete metric that maps to "X of Y test cases passed," or when the tester has no domain knowledge of the area being explored. Without intuition built on system understanding, an exploratory session produces random clicking rather than targeted investigation.
Senior
Q: A developer argues, “Our testers know what they tested — we don’t need to document exploratory sessions.” How do you respond, and how would you introduce session reporting to a team that doesn’t currently use it?
Strong answer: Without session notes, exploratory work is invisible: management can’t see coverage gaps, bugs can’t be reproduced because nobody recorded the path, and on regulated NZ projects — CoverNZ, Benefits NZ, TransitNZ — there’s no audit trail if a defect surfaces post-release. I’d introduce SBTM incrementally: start with a one-page session sheet template (charter, duration, bugs found, what wasn’t covered, one or two follow-up charters) and run it in parallel for two sprints. The team quickly sees that coverage gaps become the next round of charters, so the overhead pays for itself. Once the habit is established, I’d show the reporting data to stakeholders so exploratory testing gets the same visibility as the scripted suite — at which point the developer argument evaporates.
Lead
Q: What are two or three trade-offs between exploratory and scripted testing that a senior tester should be able to articulate in a job interview?
Strong answer: First, coverage vs. discovery: scripted tests confirm known behaviour consistently but only find bugs in paths you already thought of; exploratory testing finds unknown problems but coverage is harder to measure and report. Second, audit trail: scripted tests produce traceable pass/fail evidence that regulators and accreditation bodies require; exploratory sessions produce session notes that, without SBTM, can look like a black box to management. Third, tester skill dependency: scripted tests are more reproducible across different testers, whereas exploratory testing quality scales directly with the tester’s domain knowledge and intuition — a junior tester on a complex CoverNZ claims system will cover far less ground than a senior who knows where edge cases live. High-performing NZ QA teams use both: scripted regression for stable flows, exploratory sessions for new features and anything that feels risky.
Senior
◆ What I would do
Professional judgment — when to reach for Exploratory Testing, when to skip it, and what to watch for.
The bottom line: Exploratory testing is the technique you reach for when the most dangerous bugs are the ones nobody thought to script — but it only pays off if you write a charter specific enough to force you to say no to interesting distractions, because an unfocused session is just expensive clicking with good intentions.
Write a charter from a mission
Using the Explore / Using / To discover charter format, fill in each part below. Complete all three to see the model answer.
Model charter
“Explore the payment submission flow using NZ bank account numbers from all major banks (Harbour Bank, Southern Bank, KiwiFirst Bank, TSB, Pacific Bank) and at least two invalid formats to discover whether account validation correctly accepts valid accounts and rejects invalid ones, and whether error messages are specific enough for users to correct their input.”
Your charter doesn’t need to match this exactly. A good charter has: a specific target (not “the whole site”), a specific approach (not “various inputs”), and an information goal (not “find bugs” — what specifically?).
Self-check
- ✓ Is your target specific enough that another tester could pick it up?
- ✓ Does your “using” describe a concrete approach or data type?
- ✓ Does your “to discover” name a specific quality risk, not just “find bugs”?