Process Improvement
Using data to drive change. Moving the needle on organisational maturity and engineering performance.
1 The Hook
A Test Manager in an Auckland fintech was told: "Our testing is too slow." Instead of guessing, they measured their **DORA metrics**. They discovered that while testing took 2 days, the *wait time* for an environment took 5 days. By fixing the environment bottleneck, they halved the lead time without changing a single test case. Process improvement is about finding the "Wait" in the "Work."
2 Maturity Frameworks (TMMi)
The **Test Maturity Model integration (TMMi)** provides a roadmap for organisational growth.
| Level | Name | Characteristics |
|---|---|---|
| Level 1 | Initial | Testing is chaotic, ad-hoc, and purely reactive. |
| Level 2 | Managed | Basic test planning and environment control established. |
| Level 3 | Defined | Testing is a standardised part of the Dev lifecycle (Shift-Left). |
| Level 4 | Measured | Quality metrics are tracked and used to predict outcomes. |
| Level 5 | Optimizing | Continuous, data-driven improvement and defect prevention. |
3 Engineering Performance (DORA)
Modern QE functions align with DORA metrics to prove their impact on the delivery pipeline.
Deployment Frequency
How often we ship. High-performing teams ship multiple times per day with automated safety nets.
Lead Time for Changes
How long from "Code Commit" to "Production." Testing should be a highway, not a toll booth.
Change Failure Rate
What % of releases cause an issue? This is the ultimate measure of testing effectiveness.
Time to Restore
How fast we fix prod issues. Dependent on your BCP and test environment readiness.
4 Improvement Artifacts
Documents that track the "Journey to Better."
Maturity Assessment
A "Gap Analysis" showing where you are today vs. TMMi standards. Used to secure budget for improvements.
DORA Dashboard
A real-time view of engineering health. Shifts the conversation from "Bugs" to "Pipeline Health."
Improvement Backlog
A prioritised list of process fixes (e.g. "Automate environment spin-up", "Implement TDD").
Value Stream Map
A visual diagram of the test process identifying "Value" time vs. "Waste" time.
5 Self-Check
Q1. Is TMMi Level 5 the goal for every company?
No. Level 5 is expensive. Most NZ organisations find Level 3 (Defined) provides the best balance of quality and cost.
Q2. How does testing improve "Lead Time for Changes"?
By automating regression and environment setup, removing the need for 2-week "Testing Phases" before release.