Automation Governance Policy
An automation suite without governance is a liability. Code that nobody owns, tests that nobody reviews, and coverage that nobody measures will rot. This module covers the policies, charters, and review cadences that keep an automation suite healthy.
1 The Hook
A Wellington bank’s SDET team builds a 2,000-test Playwright suite over 3 years. No governance. No coverage targets. No policy on what gets automated versus what stays manual. No ROI review of the suite.
Year 3: the suite takes 4.5 hours to run in CI. Nobody knows which tests are still relevant. 400 tests reference pages that no longer exist. 200 tests are permanently disabled in CI because they flake. The suite has become a liability, not an asset.
The new Test Automation Lead spends her first 3 months deleting more code than she writes. The organisation spent 3 years building something that now requires 3 months of demolition before it can be trusted again.
Every one of those problems was preventable with a one-page governance policy from day one.
2 The Rule
Every automated test must be owned, maintained, and reviewed. If nobody is responsible for a test’s health, the test will rot. Governance defines who is responsible for what.
3 The Analogy
A building maintenance plan.
The building was expensive to construct. Without scheduled maintenance, the roof leaks, the lifts fail, and the plumbing corrodes. Nobody set out to let it fall apart — they just never scheduled the inspections that would have caught it early.
An automation suite is the same. The governance policy is the maintenance schedule. It is not exciting. It is what keeps the building usable.
4 Watch Me Do It
Two artefacts: an Automation Policy and an Automation Charter. The policy governs the whole suite. The charter governs a specific sprint’s automation work.
Automation Policy (one page — this is the whole document):
Automation Charter (specific to one sprint):
The policy is permanent. The charter is disposable — one per sprint, one per feature, archived when the work is done. Together they answer the two questions that prevent suite rot: what are the rules? and what is this sprint’s scope?
5 When to Use It
Governance pays off at four moments:
- Starting an automation programme: Write the policy before the first test is committed. It is far easier to establish governance at the start than to retrofit it onto 2,000 tests that have already rotted.
- Inheriting an existing suite: The first task is a governance audit — how many tests are owned, how many are flaky, how many reference pages that no longer exist. The audit output becomes the backlog.
- Adopting a new tool or framework: Every new tool requires a policy update: what is it approved for, who approved it, who owns the licence?
- Team scale-up (3+ SDETs): Without a charter process, SDETs automate in parallel and create conflicts, duplication, and gaps. Charters align the team on scope before they start.
6 Common Mistakes
🚫 “I used to think: governance means bureaucracy and slows down delivery.”
Actually: Ungoverned automation debt slows delivery. 200 flaky tests in CI create noise that teams learn to ignore — including genuine failures. The Wellington bank team had to spend 3 months on remediation before they could trust their suite. A one-page policy would have prevented all of it.
🚫 “I used to think: coverage percentage is the right target metric.”
Actually: Coverage percentage without mutation score or defect escape rate tells you very little. A test that executes a line of code is not the same as a test that validates the behaviour of that code. Measure: defects found by automation before production reaches customers, not just lines covered.
🚫 “I used to think: the automation charter is optional — it’s just planning.”
Actually: Without a charter, SDETs automate whatever is easy, not whatever matters. The charter aligns the team on what success looks like for a specific scope before work starts. It also gives the Test Automation Lead visibility across multiple SDETs working in parallel on the same sprint.
7 Now You Try
Write an automation governance policy for a 6-person QA team at a NZ insurance company. The team has an existing Playwright suite of 800 tests but no formal policy. Cover: what gets automated, coverage targets, the flaky test process, and the quarterly review cadence.
8 Self-Check
Click each question to reveal the answer.
Q1: What is the difference between an Automation Policy and an Automation Charter?
The policy is permanent and governs the whole suite: what gets automated, coverage targets, tool approval, and review cadences. The charter is specific to a sprint or feature: what is in scope, who owns it, and what done looks like. The policy sets the rules; the charter applies them to a piece of work.
Q2: Why are flaky tests a governance problem, not just a technical problem?
Because flaky tests train teams to ignore CI failures. When a flaky test fires, the team learns to dismiss it as noise — and the next time a real failure fires, it gets dismissed too. Governance provides the process to catch, quarantine, and fix flaky tests before they corrupt the signal.
Q3: Why is coverage percentage a weak automation metric?
Because a test that executes a line of code is not the same as a test that validates its behaviour. A test with no assertions can achieve 100% line coverage while catching zero defects. Better metrics: mutation score (does the test catch when the code is wrong?) and defect escape rate (how many defects reached production that automation should have caught?).
Q4: When is the best time to establish automation governance?
Before the first test is committed. Governance retrofitted onto a suite of 2,000 ungoverned tests requires a remediation project before the governance actually helps. Starting with a policy from day one costs almost nothing; adding it three years later is a 3-month project.
Q5: What must an Automation Charter’s Definition of Done include?
At minimum: all in-scope scenarios have tests, tests run in CI on every relevant trigger, a quantified pass rate target (e.g., ≥98% over 5 consecutive runs), and peer review by a second SDET. The Definition of Done is what prevents “I wrote some tests” from being confused with “I delivered automation coverage.”
9 ISTQB Mapping
The ISTQB Test Automation Engineer syllabus covers the automation scope definition (what to automate, what not to automate), the automation policy as a governance document, and metrics for measuring automation suite health and quality. Key concepts: automation return on investment, maintainability, and the distinction between automation coverage and meaningful validation. This lesson translates those concepts into a practical one-page policy and sprint charter format.
10 Next Steps
With governance in place, the next concern is whether the tests you are writing actually catch defects. Mutation testing is the discipline that answers that question. Continue with the Senior SDET Mutation Testing module, or review Framework Design for the structural decisions that governance depends on.