Test Automation Frameworks Compared
Playwright, Cypress, Selenium WebDriver, WebdriverIO, Robot Framework — five tools that each dominated their era or context. This page helps you choose the right one for your project, team, and career goals.
1 How the Landscape Shifted 2020–2025
In 2020, Selenium WebDriver was still the default answer in most job descriptions. By 2025 that had changed materially — not because Selenium broke, but because two new tools closed the gap on its pain points and the industry noticed.
Selenium Dominance
Selenium WebDriver had a near-monopoly on browser automation. ISTQB CTAL-TA was written around it. Every Java and .NET QA team used it. Its weaknesses — flaky waits, slow feedback, complex configuration — were considered the cost of doing business.
Cypress Goes Mainstream
Cypress 4–8 gained massive traction in JavaScript front-end teams. Its in-browser execution model, automatic waiting, and time-travel debugger made test writing dramatically faster for React and Vue teams. Trade-off: single-tab, same-origin only, and limited non-Chrome support at launch.
Playwright Arrives at Scale
Microsoft open-sourced Playwright in 2020 and by 2021 it was production-ready. It offered true cross-browser testing (Chromium, Firefox, WebKit), multi-tab support, network interception, the Trace Viewer, and full component testing — addressing every significant Selenium and Cypress limitation simultaneously. State of JS 2022 showed Playwright overtaking Cypress in satisfaction scores.
AI Tooling & Codegen Acceleration
Playwright codegen and Cypress Studio began auto-generating test code from recorded interactions. LLM-assisted test writing (GitHub Copilot, Claude) improved dramatically. The practical result: framework syntax matters less than it used to, because AI scaffolds the boilerplate. Framework choice now centres on architecture fit, not syntax difficulty.
A Fragmented Market with Clear Pockets
Playwright leads on greenfield TypeScript/JavaScript projects. Cypress holds ground in JavaScript teams already using it (switching cost too high). Selenium is still the correct choice for Java/.NET codebases, Appium mobile, and ISTQB-aligned shops. WebdriverIO fills the Node.js niche needing Selenium protocol compatibility. Robot Framework thrives in legacy enterprise and non-programmer tester contexts.
2 Feature Comparison Table
Scroll right on small screens. ✓ = native support | ✗ = not supported | Partial = limited or via plugin
| Feature | Playwright | Cypress | Selenium WebDriver | WebdriverIO | Robot Framework |
|---|---|---|---|---|---|
| Language support | JS/TS, Python, Java, C# | JS/TS only | Java, C#, Python, JS, Ruby | JS/TS only | Python (keywords); any via library |
| Browser support | Chromium, Firefox, WebKit (Safari) | Chrome, Firefox, Edge (no Safari) | All major browsers via drivers | All via WebDriver / DevTools | Via SeleniumLibrary / Browser Library |
| Component testing | ✓ Native (v1.22+) | ✓ Native (React, Vue, Angular) | ✗ | ✓ Via @wdio/browser-runner | ✗ |
| API testing | ✓ Built-in fetch + fixtures | ✓ cy.request() | ✗ (use RestAssured separately) | ✓ @wdio/api-service | ✓ RequestsLibrary |
| Mobile testing | Partial — emulation only | Partial — viewport emulation | ✓ Full (Appium) | ✓ Full (Appium protocol) | ✓ AppiumLibrary |
| Parallel execution | ✓ Built-in sharding + workers | ✓ Cloud (paid); limited local | Partial — Selenium Grid | ✓ Built-in runner | Partial — pabot external tool |
| CI/CD integration | ✓ GitHub Actions, Docker image | ✓ GitHub Actions, CircleCI | ✓ Any via Maven/Gradle | ✓ Any via npm scripts | ✓ Any via robot CLI |
| NZ adoption (2025) | Rising fast — #1 greenfield choice | Strong in JS/SaaS teams | Dominant in banks & govt | Niche Node.js teams | Legacy enterprise, some govt |
| Learning curve | Moderate (TypeScript helps) | Low (great docs, sandbox) | Moderate to High (boilerplate) | Moderate (familiar if you know Selenium) | Low for testers; robotics syntax feels foreign to devs |
| License | Apache 2.0 (MIT for tooling) | MIT (core); Cypress Cloud paid | Apache 2.0 | MIT | Apache 2.0 |
| Speed (typical e2e suite) | Fast — parallel by default | Fast for isolated component tests | Slower — WebDriver protocol round trips | Moderate — WebDriver + DevTools | Moderate — depends on library |
3 Playwright
Playwright
Microsoft Open Source Greenfield default (2025)Playwright drives browsers at the DevTools protocol level, bypassing WebDriver entirely. This gives it near-instant selector resolution, reliable auto-waits, and the ability to intercept network requests and mock APIs without third-party tools. Its Trace Viewer records a full timeline of every test action, screenshot, DOM snapshot, and console log — making it the best debugging experience of any framework on this list.
✓ Use Playwright when…
- Greenfield project with no existing automation
- Team writes TypeScript or JavaScript
- Cross-browser coverage (Chrome + Firefox + Safari/WebKit) is required
- You need the Trace Viewer for debugging flaky tests
- API + UI tests should live in the same framework
- CI is GitHub Actions or a Docker-based pipeline
- You want built-in sharding and parallel workers
- Component tests alongside e2e tests in one tool
✗ Skip Playwright when…
- Codebase is Java or C# and the team is not willing to add a second language
- Real-device mobile testing is required (Appium is the better fit)
- Team is component-only — Cypress has a more mature component testing DX
- Existing Selenium Grid investment is large and the CI pipeline is already stable
- ISTQB CTAL-TA exam labs require Selenium (exam is framework-specific in exercises)
- Non-programmer testers need to write tests — Robot Framework is more accessible
Trace Viewer workflow: When a test fails in CI, run npx playwright show-report. The HTML report embeds a full trace — every network request, console error, and DOM state at each step. This alone eliminates half the “it passes locally, fails in CI” debugging sessions.
4 Cypress
Cypress
JavaScript SPA teams React / Next.js sweet spotCypress runs inside the browser, sharing the JavaScript runtime with the application under test. This means no out-of-process waits, no WebDriver handshakes, and direct access to the application’s state, DOM, and network calls. The developer experience — the interactive test runner, real-time reruns on save, time-travel debugging — is still the best in class for front-end teams in 2025.
✓ Use Cypress when…
- Front-end team already uses JavaScript/TypeScript
- Application is a React, Next.js, Vue, or Angular SPA
- Fast feedback during development is the top priority
- Component-level isolation tests are needed alongside e2e
- Team is new to automation — Cypress docs and sandbox are the best onboarding experience
- You need to stub and intercept network calls easily (
cy.intercept())
✗ Limitations to know
- No multi-tab testing — Cypress cannot control two browser tabs; flows that open new windows must be refactored
- No cross-origin navigation in a single test without
chromeWebSecurity: falseworkarounds - No Safari/WebKit support — if iOS Safari compatibility is required, Playwright is the tool
- Parallelism requires Cypress Cloud (paid) or self-hosted orchestration
- Not suitable for backend-only testing — use
cy.request()for smoke-level API checks, not full API suites - Non-JS teams — Cypress is JavaScript only, so Java or Python shops need Playwright or Selenium
5 Selenium WebDriver
Selenium WebDriver
Java / .NET Enterprise ISTQB CTAL-TASelenium WebDriver is the foundation on which everything else was built. It is a W3C-standardised protocol, supported by every major browser vendor, and the underlying implementation that WebdriverIO, Robot’s SeleniumLibrary, and many commercial tools use under the hood. Do not write it off — its market position in NZ banking, government, and insurance is durable because the codebases those organisations maintain are Java and .NET, and the testers in those roles often hold ISTQB CTAL-TA.
✓ Selenium is still right for…
- Java or C# codebases where automation lives alongside application code (Maven, Gradle, NuGet)
- Appium mobile testing — Appium extends Selenium WebDriver; your skill transfers directly
- ISTQB CTAL-TA exam prep — the advanced certification uses Selenium in its worked examples
- Existing Selenium Grid infrastructure — re-platforming has a real migration cost
- Enterprise procurement environments where open-source governance requires long-track records
- Sauce Labs, BrowserStack, LambdaTest — all cloud grids natively support Selenium protocol
✗ Consider migrating when…
- Test suite has >30% flakiness and most of it is timing-related
- Team writes TypeScript and finds Java/Maven overhead draining
- You need trace-level debugging that Selenium cannot provide
- New project, no existing codebase — no reason to start with Selenium in 2025
Selenium 4 is better than you remember: Selenium 4 (2021) added the DevTools Protocol bridge, relative locators, improved grid, and removed the need for WebDriverManager boilerplate in most cases. Many of the complaints about Selenium are valid for Selenium 3 in 2018, not the current toolchain.
6 WebdriverIO
WebdriverIO
Node.js TeamsWebdriverIO (WDIO) sits between Selenium and Playwright on the maturity/modernity axis. It runs on Node.js, uses the WebDriver and DevTools protocols interchangeably, and has a rich plugin ecosystem. If your team writes JavaScript/TypeScript but your organisation has infrastructure or cross-browser requirements that Cypress cannot meet, WebdriverIO is often the pragmatic compromise — familiar to Selenium-trained testers, modern enough for a JavaScript shop.
✓ Use WebdriverIO when…
- Node.js team that needs Selenium protocol compatibility (Sauce Labs, BrowserStack already set up)
- Need to test both web and native mobile (via Appium) from the same runner
- Organisation has browser compatibility requirements that go beyond Chromium only
- Team wants Jest/Mocha/Jasmine-style assertions in a Selenium-protocol framework
- Existing Cucumber/Gherkin BDD stories that need a Node.js executor
✗ When to choose something else
- Greenfield with no mobile or legacy grid requirements — Playwright is simpler
- Java/C# team — native Selenium bindings are a better fit than Node.js tooling
- Non-programmer testers — Robot Framework is more learnable
7 Robot Framework
Robot Framework
Keyword-Driven Non-Programmer TestersRobot Framework is a keyword-driven test automation framework. Test cases are written in natural-language keywords arranged in tabular format — not code in the traditional sense. This design decision makes it genuinely accessible to business analysts, manual testers, and domain experts who need to write and read tests without being Python programmers. Selenium, requests, database access, and more are available as pluggable libraries.
✓ Use Robot Framework when…
- Non-programmer testers need to write and maintain test cases themselves
- Organisation uses a keyword-driven or data-driven methodology
- Legacy enterprise environment where plain-English test specifications are required for audit or compliance
- Team needs shared browser automation + API testing + database validation in one readable format
- Python is the scripting language of choice and developers are happy to write custom keyword libraries
✗ Trade-offs to understand
- Developers often dislike it — the keyword syntax feels unnatural to programmers; dev buy-in is harder to get
- Verbose test files — what takes 3 lines in Playwright takes 10 in Robot
- Parallel execution requires the external
pabottool, not built-in - Ecosystem moves slowly — BrowserLibrary (Playwright-backed) is the modern browser option, but adoption is patchy
- Debugging is harder — stack traces are keyword-layer deep; error messages can be cryptic for beginners
BrowserLibrary vs SeleniumLibrary: Robot Framework historically used SeleniumLibrary (WebDriver-backed). The modern alternative is BrowserLibrary, which wraps Playwright. If you are starting a new Robot project in 2025, use BrowserLibrary — you get Playwright’s speed and reliability with the Robot Framework keyword syntax your non-programmer testers already know.
8 Decision Flowchart
Work through these questions in order. Stop at the first answer that applies — that is your framework.
9 NZ Market Reality
Based on job listings, community discussions (NZ Tech Rally, Ministry of Testing Auckland chapter), and hiring patterns observed through 2024–2025.
Banks & Financial Services (ANZ, BNZ, ASB, Kiwibank, TSB)
- Selenium WebDriver + Java is the dominant choice — large codebases, ISTQB-certified teams, and enterprise procurement cycles favour known quantities
- Playwright is entering through “innovation pod” projects and new digital product teams that are JavaScript-first
- Robot Framework appears in compliance-driven automation suites where audit-readable keywords are a requirement
- Job postings: Selenium still outnumbers Playwright 3:1 in NZ banking as of 2025
Government Agencies (IRD, MSD, NZTA, Ministry of Education)
- Selenium WebDriver and Robot Framework are common — procurement stability and vendor support matter
- Some agencies are migrating to Playwright on new builds (e.g. digital service rebuilds) but the core enterprise systems remain Selenium
- Cross-browser requirements (IE compatibility was until recently a hard requirement for some agencies) historically made Selenium the only option
SaaS Startups & Scale-ups (Xero, Pushpay, Deputy, Vend)
- Playwright and Cypress dominate — JavaScript / TypeScript codebases and small agile teams prefer the DX
- Playwright is the most common new-project choice since 2023
- Cypress retains market share in companies that adopted it 2019–2022 and have not had a strong enough reason to migrate
- Test automation is often written by developers, not dedicated QA — which favours code-first tools over keyword-driven ones
Insurance, Healthcare, Utilities (Southern Cross, Meridian, Spark)
- Mixed market — Selenium in legacy system automation, Playwright or Cypress in customer-facing web products
- Mobile testing via Appium is increasing as apps become primary channels
- WebdriverIO appears in organisations where Node.js front-end teams took ownership of automation from a historically Selenium-based QA function
- Selenium is still the most requested skill in NZ QA job ads, but the gap with Playwright is closing rapidly (est. 40% of new automation roles mention Playwright by name in 2025).
- Cypress + React testing experience is a strong differentiator for SaaS roles in Auckland and Wellington.
- Knowing Playwright opens both camps — its Java bindings mean you can demonstrate cross-browser automation to Java shops while its TypeScript bindings are what SaaS teams want. It is the closest thing to a universal modern skill.
- Robot Framework is niche but sticky — there is consistent demand from mid-large enterprise that runs keyword-driven test suites. If you already know it, keep the skill sharp; it is not in decline, it is just specialised.
- No framework replaces Appium for mobile — if you want to work on banking or health apps at the native layer, Appium (which runs on the Selenium/WebdriverIO protocol) remains essential.
- AI-assisted test generation is changing onboarding — teams increasingly use LLM tooling to scaffold Playwright and Cypress tests, which lowers the syntax barrier but raises expectations around architecture, maintainability, and CI design. Understanding the framework deeply matters more, not less.