Release Gates & Go/No-Go Decisions
A release gate is a formal checkpoint. The test summary report is QA’s contribution to the Go/No-Go decision. The quality of that report determines whether the organisation makes a good decision or a dangerous one.
1 The Hook
A NZ SaaS company has a Friday afternoon release scheduled. The test lead says “98% pass rate, recommend go.” The CEO says go. Saturday morning: a payment processing bug affects 2,400 transactions.
Post-incident review: the 2% failure rate included 14 payment-related test failures. The dev team had categorised them as “low priority.” The test summary reported 98% pass — technically accurate. It did not say “the 2% failure includes the payment flow.”
The report was correct. It was not decision-ready. The CEO made a dangerous decision because the report gave him a number, not a risk picture.
That is the problem a proper Go/No-Go test summary solves.
2 The Rule
A Go/No-Go recommendation is a risk statement, not a pass/fail count. The decision-maker needs to understand what was tested, what failed, what risk is being accepted, and what the consequence of that risk is.
Senior engineer insight
The moment that changed how I think about release gates was watching a CTO sign off a go-live based on a one-line Slack message: “97% pass, looks good.” Four hours later we were rolling back because the 3% contained the only test covering concurrent session invalidation. Since then I have never written a recommendation without explicitly stating what the failures are and why they do or do not block release — because a pass rate without context is not information, it is noise.
Most NZ government programmes running under AoG ICT Common Capabilities standards require formal entry and exit criteria sign-off before UAT can begin. Teams that skip this step and define criteria retrospectively — after they already know the results — create governance records that look compliant but are forensically worthless if a post-incident review ever asks “what were the pre-agreed quality standards?”
The most common mistake: framing the Go/No-Go recommendation as a pass/fail percentage rather than as a risk statement. A percentage tells the decision-maker how many tests ran. A risk statement tells them what they are accepting responsibility for if they press go.
From the field
A Wellington central-government agency had a release gate process that looked textbook: documented entry criteria, exit criteria, and a mandatory test lead sign-off before any deployment to production. The team assumed the process was working because every release had a signed-off summary report on file. What they discovered during an OIA-triggered audit was that the sign-off was happening in under four minutes — the report was being generated from a dashboard export and approved without anyone reading the defect detail.
Three releases had gone to production with outstanding medium defects in the audit log module. None had caused an incident yet, but the audit trail for ministerial-sensitive decisions had gaps the agency could not explain. The process existed; the judgement had been automated away. They rebuilt the gate to require the test lead to write a one-paragraph narrative — not a report export — specifically naming every open defect and its accepted risk. The extra ten minutes per release saved them from a potential Privacy Act 2020 breach notification.
The lesson that generalises: a release gate is only as strong as the quality of thinking that happens inside it. Process compliance and process effectiveness are not the same thing.
3 The Analogy
A pre-flight checklist review by the co-pilot.
“All systems green” is not enough. “Engine 2 is showing a caution light — it’s within operational parameters but worth watching” is what a responsible co-pilot says. Air traffic control clears the flight because they have the full picture.
A test summary that says “98% pass rate” without explaining what the 2% means is like a co-pilot who says “all green” when engine 2 is in caution. Accurate. Dangerous.
4 Watch Me Do It
Here is a one-page Go/No-Go test summary that gives decision-makers what they need. This is the format. Adapt it to your programme.
Three things make this report decision-ready:
- Scope is explicit. The decision-maker knows what was and was not tested. The 6 skipped tests are named, not hidden in the pass rate denominator.
- Failures are contextualised. Not just a count — severity, status, and the consequence of each outstanding defect.
- The recommendation names the reason. Not “12 failures” but “CRITICAL defect in reconciliation engine, risk of data corruption.” That is a sentence the CTO can act on.
5 When to Use It
Before every production release. No exceptions. The format scales — a two-person team releasing a patch still needs a Go/No-Go decision, even if it is a one-paragraph Slack message with a pass/fail and a risk statement.
Entry criteria (what must be complete before testing starts) and exit criteria (what must be true before releasing) must be defined at the start of each sprint or release cycle. You cannot measure quality against criteria that do not exist.
6 Common Mistakes
🚫 “I used to think: 95%+ pass rate always means go.”
Actually: 1 critical payment failure in 1,000 tests is a NO-GO. Pass rate percentage is meaningless without context of what failed. A 99% pass rate with 1 critical security defect outstanding is still a NO-GO. Always name what failed and why it matters.
🚫 “I used to think: the test lead makes the Go/No-Go decision.”
Actually: The test lead makes the recommendation. The risk owner — CTO, product owner, or release manager — makes the decision. Never conflate the two. The test lead who says “I decided to go-live” has taken on liability that is not theirs to carry.
🚫 “I used to think: deferred defects don’t need to be mentioned in the test summary.”
Actually: Every known defect accepted into production must be documented in the release notes and the test summary. Undocumented known issues create liability. If something breaks in production and you knew about it and said nothing, that is a governance failure — not just a quality one.
Why teams fail here
- Entry and exit criteria are defined after testing is complete — criteria written to match known results are not quality standards, they are post-hoc justifications. Any programme operating under AoG standards or ISANZ audit scope is exposed the moment a reviewer asks for the pre-agreed baseline.
- The test lead conflates recommendation with decision — saying “we decided to go live” rather than “I recommended go; the release manager accepted the risk.” The distinction matters for liability, incident review, and for building a culture where QA evidence is taken seriously rather than rubber-stamped.
- Known defects are deferred without documentation — accepting a medium defect into production without noting it in the test summary and release notes creates a governance gap. Under the NZ Privacy Act 2020 and the Public Records Act 2005, government agencies must be able to demonstrate that release decisions were made with full information. An undocumented known issue undermines that entirely.
- Schedule pressure converts the gate from a checkpoint into a ritual — the release gate meeting becomes a five-minute sign-off where the test lead reports “99% pass” and everyone moves on. When the gate has no teeth — when no one has ever said NO-GO — it is not a gate, it is theatre. Teams that have never blocked a release have usually never had a gate worth the name.
Key takeaway
A release gate that has never said no is not a quality checkpoint — it is a ceremony that gives dangerous decisions the appearance of governance.
7 Now You Try
Write a one-page Go/No-Go test summary for an NZ bank’s new internet banking login feature. Test results: 312 tests run, 298 pass, 8 fail (2 critical: session management; 4 medium: UI glitches; 2 low: help text errors), 6 skipped due to environment issues. The critical failures affect token rotation. Make a recommendation and justify it.
8 Self-Check
Click each question to reveal the answer.
Interview Questions
What NZ hiring managers ask about release gates and go/no-go decisions.
Q1. What is a release gate and what authority is required to override one?
Strong answer: A release gate is a defined criterion — automated or manual — that must be met before a release proceeds. Automated gates: all tests green, security scan clean, performance within threshold. Manual gates: UAT sign-off, change advisory board approval. Override authority depends on the risk framework: for NZ government systems, overrides typically require written approval from the CISO or programme director, and the override must be logged with rationale. A gate that is routinely overridden without consequence provides false confidence, not real quality assurance.
Q2. How do you define go/no-go criteria before a release?
Strong answer: Criteria must be specific and measurable — not "quality looks good." Automated: all P1/P2 defects resolved, zero security scan findings above medium, API response times under 500ms at expected load, WCAG AA accessibility pass. Manual: UAT sign-off from named stakeholders, security sign-off, operations confirmation that monitoring is in place. Define criteria before testing begins — not the day before go-live. For NZ government gateway reviews, document the evidence package alongside the go/no-go criteria from the start.
Q3. What do you do when a release is blocked by a gate failure 24 hours before go-live?
Strong answer: Escalate immediately with clear information: which gate failed, what it means in business terms, what it would take to fix (effort, risk), and what the risk is of proceeding. Present three options: fix and delay, proceed with a documented risk-accepted exception, or roll back. The tester's job is accurate information, not the go/no-go decision. If an exception is accepted, get it in writing with the accepting stakeholder named and residual risk documented. After release, conduct a retrospective on why the gate was hit at the last minute.
Q1: Why is a 98% pass rate insufficient information for a Go/No-Go decision?
Because pass rate percentage says nothing about what the 2% failure contains. One critical defect in a payment flow, a security bypass, or a data corruption risk can make the 2% a NO-GO regardless of the overall percentage. The decision-maker needs to know what failed and what the consequence is.
Q2: Who makes the Go/No-Go decision?
The risk owner — CTO, product owner, or release manager. The test lead makes the recommendation and provides the evidence. The decision and its consequences belong to the risk owner, not to QA.
Q3: What must happen to deferred defects?
Every known defect accepted into production must be documented in the test summary and the release notes. Undocumented known issues create legal and governance liability. If a known defect causes a production incident and it was not documented, the organisation cannot demonstrate it made an informed decision.
Q4: When should entry and exit criteria be defined?
At the start of each sprint or release cycle, before testing begins. Criteria defined after testing is complete are invented to match results, not to measure quality. Pre-agreed criteria give the test lead an objective measure to report against under pressure.
Q5: What are the three elements that make a test summary decision-ready?
Explicit scope (what was and was not tested), contextualised failures (severity, status, and consequence of each outstanding defect), and a recommendation with a named reason — not a number, but a sentence the decision-maker can act on.
9 ISTQB Mapping
The ISTQB Test Manager syllabus covers test progress monitoring, test reporting, and the test completion report. Key concepts: exit criteria evaluation, defect metrics in context of risk, and the distinction between the test manager’s role (provide evidence and recommendation) and the stakeholder’s role (make the decision). This lesson translates those concepts into a practical NZ-context Go/No-Go report format.
10 Next Steps
You can make a sound Go/No-Go decision when your environments are clean and your data is representative. Next: how to manage test data across environments without breaching the NZ Privacy Act 2020.