Technical Vision
An automation architect doesn't write tests. They design the future in which tests are written. Learn how to build a vision that outlasts tools, teams, and market shifts.
1 The Hook — Why This Matters
In 2019, a NZ telco invested $2M in a custom automation framework built on Selenium Grid and a proprietary test language. By 2024, Playwright had made their browser abstraction layer obsolete, their custom DSL had no community support, and the one engineer who understood the compiler had left. The $2M asset became a $2M liability. They spent another $1.5M migrating to standard tools. The original architect had optimised for technical elegance instead of ecosystem longevity.
Technical vision is not about predicting the future. It's about building adaptability into the present so that when the future arrives, you can evolve without starting over.
2 The Rule — The One-Sentence Version
Build platforms, not products. Design for change. Standardise where it reduces friction; leave flexibility where it enables innovation.
An automation architect's output is not test scripts. It is the system that lets hundreds of engineers write better tests faster. That system must be treated as a product with customers, roadmaps, and support SLAs.
3 The Analogy — Think Of It Like...
Designing a city's water infrastructure instead of fixing individual taps.
A plumber fixes taps. A water architect designs pressure systems, mains distribution, and redundancy. When the city grows, the infrastructure scales. When a pipe bursts, the system reroutes. Automation architects design infrastructure. Test engineers fix taps. Both are necessary, but the architect's decisions determine whether the city has running water in ten years.
Senior engineer insight
The hardest lesson I learned building automation platforms for NZ financial services clients was that your technical vision has to survive a team turnover. We had a beautifully documented three-year roadmap for a large bank's testing infrastructure — and within 18 months, five of the seven engineers who built it had moved on. The ones who came in saw a system they didn't understand, didn't trust, and gradually stopped contributing to. What saved us wasn't the architecture; it was the decision to use plain TypeScript over a custom DSL and to keep every dependency in the standard npm ecosystem. New engineers could onboard in a week because the tools were familiar, not bespoke.
Most common mistake: writing the vision for the team you have today instead of the team you'll have in year three, who may have never met you.
From the field
A Wellington-based government agency — working in the health space — brought us in to review why their automation investment wasn't paying off after two years. The team had assumed they needed a sophisticated, custom-built framework because their patient data workflows were complex and their compliance requirements were strict. What they'd actually built was a 12,000-line internal library that wrapped Playwright, re-implemented its own retry logic, and abstracted the selector engine in ways that made debugging nearly impossible. The real problem wasn't the tooling — it was that no one had written down what "done" looked like for automation. There was no coverage target, no agreed quality gate, and no executive sponsor who understood what the framework was for. When we helped them define a simple vision document — two pages, three objectives, one success metric per year — the team immediately started making better decisions. They deleted 8,000 lines of custom code in the next sprint and replaced it with built-in Playwright features. The lesson that generalises: complexity in a framework is usually a symptom of unclear purpose at the top, not insufficient engineering at the bottom.
4 Watch Me Do It — Step by Step
Here is how to build and communicate a multi-year automation vision.
- Define the 3-year roadmap
Year Theme Capabilities Year 1: Foundation Establish Unified framework, CI integration, core coverage, metrics baseline Year 2: Scale Accelerate Parallel execution, cross-team shared libraries, advanced reporting Year 3: Optimise Transform AI-assisted generation, predictive quality gates, autonomous failure analysis Align with TMMi: Year 1 targets Level 3 (Defined), Year 2 targets Level 4 (Measured), Year 3 targets Level 5 (optimising).
- Make build-vs-buy decisions
Factor Build (Internal) Buy (Commercial) Core differentiator Yes — automation is competitive advantage No — commodity capability Customisation needs High — unique tech stack Low — standard web/mobile/API Time pressure Low — can invest 6-12 months High — need solution in 4-8 weeks IP / Data sensitivity On-prem required Vendor meets residency requirements - Treat the framework as a product
- Product Owner: Automation Architect or Lead
- Customers: Development teams, QA engineers, release managers
- Value Proposition: Faster feedback, reduced escape rate, confident releases
- Roadmap: Published quarterly, prioritised by customer impact
- Support Model: SLAs for pipeline failures, documentation, onboarding guides
- Success Metrics: Adoption rate, time-to-first-test, support ticket volume
5 When to Use It / When NOT to Use It
✅ Build custom platform when...
- Multiple teams need shared libraries
- Vendor tools don't integrate with your stack
- Automation is a competitive differentiator
- You have budget for dedicated platform team
❌ Buy off-the-shelf when...
- Single team, standard requirements
- Speed to value is critical
- No budget for ongoing platform maintenance
- Vendor meets all integration needs
6 Common Mistakes — Don't Do This
🚫 Optimising for technical elegance over adoption
I used to think: The best architecture wins.
Actually: The adopted architecture wins. A technically perfect framework that no one uses is a failure. Optimise for ease of use, documentation, and onboarding first. Elegance is a bonus, not a prerequisite.
🚫 No vendor partnership strategy
I used to think: We buy tools; the vendor delivers features.
Actually: NZ's smaller market size means less vendor attention. Counter this by partnering with regional user groups, joining APAC customer advisory boards, or aggregating demand through industry bodies. A strategic partnership gives you influence, early access, and pricing leverage.
🚫 Technology radar as decoration
I used to think: Publishing a technology radar is enough.
Actually: A radar that isn't reviewed quarterly and doesn't influence purchasing decisions is wallpaper. Define quadrants (Techniques, Tools, Platforms, Languages) and rings (Adopt, Trial, Assess, Hold). Review with the architecture board every quarter. Publish internally to align teams.
7 Now You Try — Interview Warm-Up
Scenario: You are the automation architect for a company with 8 development teams using 4 different tech stacks (React, Angular, mobile native, and API-only backend). Each team has built its own automation solution. There is no shared reporting, no standard patterns, and onboarding a new QA engineer takes 6 weeks because every team does things differently.
What is your Year 1 platform strategy?
Year 1 strategy:
- Audit: Map all existing tools, patterns, and pain points across the 8 teams.
- Standardise core: Define shared libraries for config, reporting, and test data management that work across all stacks.
- Flexible periphery: Let teams choose their UI tool (Playwright, Cypress, Appium) but mandate the core libraries for reporting and CI integration.
- Internal platform: Build a developer portal with documentation, onboarding guides, and self-service project scaffolding.
- Metrics: Track adoption rate, time-to-first-test, and cross-team contribution to shared libraries.
Why teams fail here
- Vision without governance: A roadmap document that lives in Confluence and is never referenced in sprint planning or quarterly reviews is not a vision — it is a wish. Without a named owner, a review cadence, and a mechanism to say no to off-roadmap requests, the vision dissolves within six months as tactical pressure accumulates.
- Treating Year 1 as optional: Teams skip the foundation year (unified reporting, CI integration, coverage baseline) because it feels unglamorous, then spend Years 2 and 3 unable to scale because every team's data is in a different format and no one can aggregate quality metrics across the portfolio. This is especially common in NZ public sector organisations where procurement cycles and team rotations mean Year 1 work gets deferred indefinitely.
- Build-vs-buy decided by preference, not evidence: Architects who love engineering default to build; those from consulting backgrounds default to buy. Neither is a strategy. The decision should follow from data: what does the team's existing skill profile support, what is the total cost of ownership over three years, and does this capability differentiate the business or merely enable it? CoverNZ, Revenue NZ, and TransitNZ all have different answers to these questions even though they face similar compliance pressures.
- Confusing tool adoption with cultural adoption: Mandating a framework and mandating that engineers actually believe in it are different things. Teams that feel the platform was imposed on them find creative ways around it — local test runs that bypass CI gates, parallel shadow frameworks, or simply not writing tests at all. A technical vision that doesn't include a change management strategy for the humans using it will be technically correct and practically ignored.
Key takeaway
A technical vision is not a document you write once to get approval — it is the standing answer to the question every engineer on your team asks before making a decision: "Does this move us closer to or further from where we said we were going?"
8 Self-Check — Can You Actually Do This?
Click each question to reveal the answer. If you got all three, you're ready to practice.
Q1. What is the difference between treating automation as a project vs. a product?
A project has an end date and a fixed scope. A product has ongoing ownership, a roadmap, customers, and SLAs. Automation frameworks should be treated as products because they evolve continuously with the applications they test.
Q2. What are the three tiers of vendor engagement?
Transactional (licence only), Strategic (dedicated CSM, roadmap input), and Partnership (joint case studies, beta programs, co-marketing). For NZ teams, strategic or partnership tiers counter the smaller market size disadvantage.
Q3. When should you build vs. buy an automation solution?
Build when automation is a core differentiator, customisation needs are high, and you have budget for ongoing maintenance. Buy when requirements are standard, time to value matters, and vendor integration meets your needs.
9 Interview Prep — Architect Q&A
Strategic and forward-looking questions test your ability to think long-term. These conversations define architect-level leadership.
Q. "What's your vision for the future of test automation?"
The trend is away from manual test maintenance toward autonomous systems. We're moving from "write tests once, maintain forever" to "define intent once, let AI generate and maintain tests." But this doesn't mean humans disappear; it means humans shift to defining requirements and validating AI output. The competitive advantage moves upstream: architects who can clearly articulate test intent, define quality metrics, and design systems that adapt as applications evolve will lead. The low-value work (writing boilerplate test code) disappears. The high-value work (setting strategy, measuring quality, preventing escape) becomes even more critical.
Q. "How do emerging technologies (AI, cloud, edge) change testing?"
AI: test generation and autonomous failure analysis. Cloud: enables elastic scaling, ephemeral environments, and global distribution. Edge: new testing scenarios (latency, offline-first, bandwidth constraints). Together, they create a new testing paradigm: tests are distributed globally, generated dynamically, and validated continuously. For NZ teams, this means: your test infrastructure must be cloud-agnostic (use Kubernetes so you can move between AWS/Azure/GCP), your tests must account for latency and failure modes unique to distributed systems, and your test generation must be adaptable (fine-tune models for your codebase rather than relying on one-size-fits-all tools).
Q. "How do you balance innovation with stability in test tooling?"
Innovation without stability creates thrashing; stability without innovation creates stagnation. The balance: standardise the core (shared libraries, CI/CD gates), allow experimentation at the edges (teams can pilot new tools on non-critical paths). Establish a technology radar: Adopt (use everywhere), Trial (pilot on one team), Assess (track but don't invest), Hold (stop using). Review quarterly. Make it easy to move up the radar (from Assess to Trial) but hard to move down (if you adopt and later regret, the migration cost is high). This governance prevents churn while enabling progress.
Q. "What would make testing a true competitive advantage?"
Most teams see testing as cost—something that slows down delivery. Competitive advantage comes from flipping that mindset: testing as enabler. High confidence in quality means you can release faster, take bigger risks, and iterate with less fear. Teams that master this gain: 1) Faster feedback (hours instead of days), 2) Lower escape rate (fewer production incidents), 3) Happier engineers (less debugging production issues), 4) Happier customers (fewer bugs). The architects who build systems that make this possible are strategic assets. For NZ teams competing globally, superior testing quality can offset smaller team size. You can't outcode Silicon Valley, but you can out-test them with smart automation and quality discipline.