Advanced Exploratory Testing & SBTM
Exploratory testing isn't wandering around clicking buttons. At senior level, it's a structured, time-boxed, charter-driven investigation that produces auditable results.
1 The Hook — Why This Matters
In 2020, a major NZ insurer launched a new online quote system for earthquake cover. It had passed three months of scripted regression testing. On the first day, a Wellington customer entered their address, selected the highest risk zone, and received a blank premium. The "Get Quote" button simply did nothing. No error. No loading state. Just silence.
The bug had survived because no scripted test used a Wellington address (zone 3, high risk) combined with the specific building type dropdown. The tester who eventually found it wasn't following a script — she was exploring. She had a charter, a 90-minute time box, and a mission: discover premium calculation accuracy across different regions. She found three bugs in that one session, including one that would have violated Fair Trading Act disclosure requirements.
Scripted tests verify what you already suspect. Exploratory testing finds what you never imagined. But without structure, it becomes random clicking. That's why senior testers use Session-Based Test Management.
Senior engineer insight
The single change that altered how I run exploratory testing was realising that the charter is not a constraint — it's a compass. Before I adopted SBTM, I'd finish a session with a list of bugs and no idea what I hadn't tested. The charter's information goal forces you to ask that question before you start, and the session sheet answers it. In a regulated domain like insurance or payments, that distinction matters: "we tested it" and "we tested it with evidence of scope and coverage" are legally very different claims.
Most common mistake: writing a charter so narrow it becomes a scripted test in disguise, leaving no room for the tester's judgment to surface risks the charter author never anticipated.
From the field
On a NZ government payments platform migration, the team assumed their three-month scripted regression suite was sufficient coverage — every acceptance criterion passed. We ran a SBTM sprint in the final week using SFDIPOT as the structural lens: the Structure tour uncovered that the legacy database schema had silently truncated transaction reference numbers at 12 characters, while the new system accepted 20. No test had used a 13-character reference because no requirement mentioned a limit. The debrief surfaced the pattern across three separate testers' session sheets simultaneously: all three had seen odd truncation in different areas but hadn't yet connected them. The consolidated debrief turned three confusing observations into one definitive root cause within 20 minutes. The lesson: structured debriefs convert distributed individual observations into shared team intelligence in a way that individual Jira tickets simply cannot.
2 The Rule — The One-Sentence Version
Give every exploratory session a charter, a time box, and a debrief — or it isn't testing, it's guessing.
The charter defines the mission. The time box creates focus. The session sheet captures evidence. The debrief turns individual learning into team knowledge. Without all four, you cannot measure exploratory coverage, reproduce findings, or prove to stakeholders that rigorous testing occurred.
3 The Analogy — Think Of It Like...
A search-and-rescue operation in the bush.
You don't send volunteers to wander the forest hoping to find the missing hiker. You divide the terrain into sectors (charters), assign each team a time limit (time box), and require them to radio in coordinates and photos (session sheets). Afterward, you debrief: what did they cover, what blocked them, what needs a follow-up search? Exploratory testing is search-and-rescue for bugs. Structure makes it professional.
4 Watch Me Do It — Step by Step
Here is a real NZ example using an earthquake insurance quote flow. Follow these steps for every exploratory session you run.
-
Write a charter using the formula
"Explore [target] using [resources] to discover [information]."
Charter: "Explore the earthquake-quote flow using Wellington (zone 3, high risk), Auckland (zone 2, medium risk), and Christchurch (zone 4, high risk) addresses to discover premium calculation accuracy and UI state issues."
- Set a time box: 60–90 minutes Research shows focus degrades after 90 minutes. Short enough to maintain intensity; long enough to find deep bugs. Use a visible timer. When it rings, stop. Debrief. Start a new charter if needed.
-
Run structured tours
Tours are mental models that guide your exploration:
- Business Tour: Follow the full end-to-end quote-to-purchase flow as a customer would.
- Feature Tour: Compare the same flow across zones (Wellington vs Auckland vs Christchurch).
- Anti-Tour: Do everything "wrong" — skip required fields, enter impossible dates, navigate backward after submission.
- Money Tour: Test premium edge cases: $0, $50,000, $10,000,000 sums insured.
- Observed Failures Tour: Revisit areas where you've found bugs before; regressions cluster.
- Document on a session sheet Record: charter, start/end time, environment, bugs found (with repro steps), coverage areas tested, obstacles (blocked tests), and notes for follow-up. This is your audit trail.
-
Debrief within 24 hours using PROOF
Letter Meaning What to cover P Past Time spent, charter completed or not, interruptions R Results Bugs logged, coverage achieved, risks identified O Obstacles Blocked tests, missing data, environment issues O Outlook Follow-up charters, areas needing deeper testing F Feelings Gut reactions, perceived risk, quality impressions - Triangulate findings with data The Wellington address produced a blank premium. The Auckland address showed the wrong zone label. Christchurch worked. This pattern suggests a zone-mapping bug, not a general calculation failure. Your session sheet lets you spot patterns across charters.
5 When to Use It / When NOT to Use It
✅ Use SBTM when...
- You need to explore unfamiliar or complex features
- Requirements are unclear, changing, or incomplete
- You want to make exploratory work visible and auditable
- You're testing for user experience and usability issues
- You need rapid feedback before a release deadline
- You want to train junior testers by pairing on real missions
❌ Don't use SBTM when...
- You need to verify compliance with exact regulations (use scripted)
- The test must be reproduced identically every time (automation)
- You view it as a replacement for automation or regression
- You skip the debrief and session sheet (then it's unstructured)
- Management uses session counts as a productivity metric
- You have no time to document findings formally
6 Common Mistakes — Don't Do This
🚫 Vague charters
I used to think: "Test the quote flow" was a good enough charter.
Actually: A vague charter produces vague results. "Test the quote flow" could mean anything. A strong charter specifies the target, the resources, and the information goal: "Explore the quote flow using high-risk Wellington addresses to discover premium calculation accuracy." Specificity drives discovery.
🚫 Skipping the debrief
I used to think: Logging bugs in Jira was enough; the debrief was bureaucracy.
Actually: The debrief is where team learning happens. Without it, the tester who found the Wellington blank-premium bug is the only one who understands the pattern. The debrief surfaces coverage gaps, decides follow-up charters, and converts individual insight into team strategy.
🚫 Viewing exploratory as a replacement for automation
I used to think: If we do enough exploratory testing, we don't need automated regression.
Actually: Exploratory testing and automation are complementary. Exploratory finds new bugs. Automation catches regressions in old ones. A mature team uses exploratory to discover, then automates the critical paths found, then continues exploring new areas.
7 Now You Try — Interview Warm-Up
Question: Write a charter for an exploratory session on an online supermarket checkout. Apply the formula: "Explore [target] using [resources] to discover [information]."
Draft your charter before revealing a strong example.
Example charter:
"Explore the checkout flow using the Countdown website on mobile Safari with a basket containing alcohol items and a delivery address in a rural postcode to discover age-verification gate accuracy, delivery option availability, and payment validation behaviour."
Why this works: It names a specific target (checkout flow), specific resources (mobile Safari, alcohol items, rural postcode), and specific information goals (age verification, delivery options, payment validation). A tester can start immediately without asking clarifying questions.
Why teams fail here
- Charters are written too late — after the tester has already started exploring — so they describe what happened rather than guide what to discover, eliminating the focus benefit entirely.
- The debrief is skipped under time pressure, so individual session findings are never synthesised into coverage maps or follow-up charters, and the same gaps get re-explored in the next sprint.
- Teams count sessions rather than measure coverage depth — management reports "12 sessions completed" with no visibility into which areas were tested shallowly, which were blocked, and which were never reached.
- Testers abandon the SFDIPOT/tour structure mid-session and revert to intuition-driven clicking, producing findings that cluster around familiar areas and miss structural, data, and platform dimensions entirely.
Key takeaway
Exploratory testing without a charter, time box, and debrief is not a technique — it's a habit; SBTM is what turns that habit into professional, auditable, repeatable practice.
8 Self-Check — Can You Actually Do This?
Click each question to reveal the answer. If you got all three, you're ready to run professional exploratory sessions.
Q1. What does PROOF stand for in a session debrief?
Past (time spent, charter completed), Results (bugs, coverage), Obstacles (blocked tests), Outlook (follow-up needed), Feelings (gut risk, quality impressions). PROOF turns a solo testing session into shared team knowledge.
Q2. Name three tours and explain when to use each.
Business Tour: Follow the end-to-end happy path as a real user would. Use this first to establish baseline behaviour. Feature Tour: Compare the same feature across different configurations or data sets. Use this to find data-dependent bugs. Anti-Tour: Do everything wrong. Use this to find validation gaps, error-handling weaknesses, and security issues.
Q3. Why is a 90-minute time box recommended?
Because focus degrades significantly after 90 minutes of continuous cognitive effort. The time box creates urgency, prevents scope creep, and ensures the session is short enough to debrief while memory is fresh. If the charter isn't complete in 90 minutes, you start a new session rather than losing focus.
9 Interview Prep — Common Questions
Q. "What goes in a session debrief?"
I use PROOF. Past covers time spent and whether the charter was completed. Results covers bugs found, coverage achieved, and risks identified. Obstacles covers anything that blocked testing. Outlook covers follow-up charters or deeper testing needed. Feelings covers gut reactions and perceived quality. The debrief converts individual learning into team strategy.
Q. "How do you measure coverage in exploratory testing?"
I map coverage by area, not by code. The session sheet records which features, user flows, and data configurations were tested. After multiple sessions, I build a coverage matrix showing which areas have been explored and which haven't. I also note the depth: shallow (smoke), medium (functional), or deep (edge cases and stress).
Q. "How do you balance exploratory testing with automation?"
They serve different purposes. Exploratory finds new bugs and understands new features. Automation prevents regressions in known critical paths. My workflow is: explore to discover, automate the stable critical paths found, then continue exploring new areas. Exploratory feeds the automation backlog; automation frees time for deeper exploration.
Q. "What makes a good charter?"
A good charter follows the formula: "Explore [target] using [resources] to discover [information]." It is specific enough that a tester can start without asking questions, but broad enough to allow creative investigation within the mission. It names the target feature, the data or environment to use, and the risks or information goals to pursue.