Mobile Automation
Native mobile automation is harder than web. Different tools, different architectures, and device fragmentation that makes consistent CI challenging. This module covers Appium, Detox for React Native, and how to build a mobile test pipeline that actually works.
1 The Hook — Why This Matters
An Auckland retail bank ships a new mobile banking app. The SDET team builds automated tests using the web-based Playwright suite — the same tooling they use for every other project. Tests pass. They go live on Monday.
By Wednesday, the support line receives 340 calls. Users on iPhones can't complete the FaceID authentication flow after updating to iOS 17.4. The Playwright tests were running against the web view layer, not the native iOS layer. FaceID is a native biometric interaction — it is invisible to web-based automation. The test suite had never touched it.
A proper Appium suite using the XCUITest driver would have flagged the failure during regression. Instead, the issue reached 340 real customers before anyone knew. The cost of retrofitting native automation after a live incident is ten times the cost of building it correctly before go-live. The tool choice made the difference.
2 The Rule — The One-Sentence Version
Mobile automation requires matching the tool to the layer. Web views use Selenium or Playwright. Native UI uses Appium or platform-native tools (Espresso for Android, XCUITest for iOS). Never automate native interactions with web tools.
3 The Analogy — Think Of It Like...
A TV remote trying to control a microwave.
Both are remote controls. Both send signals. But they speak different protocols. You can press every button on the TV remote and the microwave will never respond, because the signals don't match the receiver. Using Playwright to automate a native mobile app is the same problem. Playwright talks to the browser protocol. Native iOS talks to the XCUITest protocol. You need the right tool for the right layer — and a senior SDET knows which protocol each layer speaks before they write a single line of test code.
Senior engineer insight
The moment you stop treating mobile as "just another Appium project" and start treating it as three distinct layers — native UI, web view, and network — your test strategy becomes dramatically sharper. On a KiwiSaver onboarding project, we discovered that our entire Appium suite was only exercising the web view layer embedded inside a native shell. The biometric step, the camera-based ID scan, and the native push permission dialog were all invisible to every test we had written. Switching to a layered strategy — Detox for the native shell, Playwright for the embedded web content, and contract tests for the API — reduced our escaped defect rate by 60% in two sprints.
The most common mistake is treating Appium's cross-platform API as a guarantee of cross-platform coverage — it unifies syntax, not behaviour, and teams often discover the gaps only after a live incident.
From the field
A Wellington health startup was building a patient-facing app for booking GP appointments through an iwi health provider. The team assumed that because the happy-path login worked perfectly on the Samsung Galaxy S23 emulator in CI, they were covered. They weren't — three different failure modes emerged within two weeks of launch: an Oppo A57 running Android 11 (a hugely common sub-$400 device in Aotearoa) rendered the date picker natively instead of via the custom component, breaking the booking flow; iOS 17.3 introduced a change to the way accessibility permissions are requested that caused the Appium session to time out silently; and the back-navigation behaviour on Android differed between the physical devices patients actually owned versus the emulator. The fix was a tiered device strategy: emulators for fast developer feedback in every PR, plus a real-device matrix on Firebase Test Lab (Pixel 6, Samsung A-series, iPhone 14, older iPad) triggered nightly. The lesson generalises — in NZ, the device spread is wide and the long tail of older Android devices is longer than most teams expect.
4 Watch Me Do It — Two Patterns
Pattern 1: Appium with WebdriverIO (cross-platform)
Use this for cross-platform automation or when the team works across Android and iOS from a shared codebase. Appium sits as a server between your test scripts and the platform driver (UiAutomator2 for Android, XCUITest for iOS).
// wdio.conf.js (Android config)
exports.config = {
runner: 'local',
hostname: 'localhost',
port: 4723,
capabilities: [{
platformName: 'Android',
deviceName: 'emulator-5554',
app: './apps/resync-bank.apk',
automationName: 'UiAutomator2',
newCommandTimeout: 240
}],
specs: ['./tests/mobile/**/*.spec.js']
};// tests/mobile/login.spec.js
describe('Login flow', () => {
it('should login with valid credentials', async () => {
const emailField = await $('~email-input'); // accessibility ID selector
await emailField.setValue('tane@resync.nz');
const passwordField = await $('~password-input');
await passwordField.setValue('SecurePass123!');
const loginBtn = await $('~login-button');
await loginBtn.click();
const dashboard = await $('~dashboard-header');
await expect(dashboard).toBeDisplayed();
});
});~id-name) wherever possible. XPath and class-name selectors break on every OS update. Accessibility IDs are set by developers and survive UI refactors. Make it a team agreement that every interactive element ships with a testID or accessibilityLabel before code review passes.Pattern 2: Detox for React Native (recommended for RN apps)
Detox is purpose-built for React Native. It runs in-process with the app rather than over a network socket, which makes it significantly more stable and faster than Appium for RN projects. If the app is React Native, Detox should be your default choice.
// e2e/kiwiSaverEnrolment.e2e.js
describe('KiwiSaver Enrolment', () => {
beforeAll(async () => {
await device.launchApp();
});
beforeEach(async () => {
await device.reloadReactNative();
});
it('should complete enrolment flow', async () => {
await element(by.id('ird-input')).typeText('123456789');
await element(by.id('contribution-rate')).tap();
await element(by.text('3%')).tap();
await element(by.id('enrol-button')).tap();
await expect(element(by.text('Enrolment complete'))).toBeVisible();
});
});CI: Firebase Test Lab (real devices without a device farm)
Most NZ engineering teams can't maintain a physical device lab. Firebase Test Lab gives you access to real Android and iOS devices in Google's data centres, triggered from CI via the gcloud CLI. This is the pragmatic choice for teams that need real-device coverage without the overhead.
# GitHub Actions step — run Espresso tests on Firebase Test Lab
- name: Run tests on Firebase Test Lab
run: |
gcloud firebase test android run \
--type instrumentation \
--app app/build/outputs/apk/debug/app-debug.apk \
--test app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk \
--device model=Pixel6,version=33,locale=en_NZ,orientation=portrait \
--timeout 5m5 When to Use It / When NOT to Use It
✅ Use Appium or Detox when...
- The app has native UI: biometrics, notifications, camera, native date pickers, hardware sensors
- The app is React Native — use Detox
- The app is Flutter — use the Flutter integration test driver
- You need OS-version regression (iOS 17.x vs 18.x behaviour differences)
❌ Don't use native tools when...
- The app is purely web-view based — Playwright mobile emulation is fine
- You only need responsive layout testing — use browser viewport flags
- You're testing an API the mobile app calls — test the API directly
6 Common Mistakes — Don't Do This
🚫 Using Playwright viewport emulation to test a native app
I used to think: Playwright's mobile viewport emulation tests the mobile app.
Actually: Viewport emulation tests the responsive web layout in a browser. It does not test native components, device sensors, push notifications, biometric flows, or platform-specific behaviour. A bank's mobile app and its web site are different things, even if they look similar. This is the Auckland bank mistake — don't repeat it.
🚫 Assuming one Appium test covers both iOS and Android
I used to think: Appium's cross-platform API means one test script works on both platforms.
Actually: Appium provides a common API, but iOS and Android have different element selectors, different accessibility ID conventions, different timing, and different native dialog behaviour. Tests usually need platform-specific tweaks. Write shared business logic, but keep platform-specific selectors and waits in separate capability files or page object variants.
🚫 Insisting on a physical device lab
I used to think: Mobile automation needs a physical device lab on-premise to be reliable.
Actually: Cloud device farms — Firebase Test Lab, AWS Device Farm, BrowserStack App Automate — give you real device coverage across dozens of models and OS versions without the maintenance overhead of physical devices. For most NZ teams this is the right call. Physical labs are expensive, require dedicated maintenance, and go stale fast when new OS versions release.
Why teams fail here
- Emulator-only CI: Emulators don't reproduce real-device timing, sensor behaviour, or manufacturer-specific Android skins. Teams ship with green emulator runs and discover failures on real Samsungs or Oppos, which dominate the NZ mid-market. Firebase Test Lab or BrowserStack real devices must be part of the pipeline before release, not after.
- No testID discipline: Developers ship components without
testIDoraccessibilityLabelattributes, forcing testers to use XPath or class-name selectors. These break on every minor OS update or layout change. Without a team agreement that every interactive element requires a stable selector before code review passes, the test suite becomes a maintenance burden within months. - Treating Appium flakiness as inevitable: Unstable Appium suites are almost always a symptom of missing explicit waits, bad selectors, or the wrong driver — not an inherent property of mobile automation. Teams that accept flakiness as normal stop trusting the suite and eventually abandon it. Investigate every flaky test; fix or delete it within one sprint of it appearing.
- Automating the wrong layer: Native interactions — biometrics, camera, NFC, push notifications, permission dialogs — cannot be tested by driving the UI the way a user would. They require either mocking at the OS level (Detox's
device.sendUserNotification(), Appium'smobile: sendPushNotification) or separating the integration concern from the UI concern. Trying to automate these through the standard UI driver produces flaky, unmaintainable tests that give false confidence.
Key takeaway
Mobile automation is not hard to write — it is hard to keep working, and the difference between a suite that lasts and one that gets abandoned is whether the team matched the tool to the layer, enforced selector discipline from day one, and ran against real devices before every release.
7 Now You Try — Prompt Lab
A React Native banking app for Harbour Bank has a login flow: email input, password input, then biometric confirmation (FaceID on iOS, fingerprint on Android) on supported devices, with a fallback to a 6-digit PIN.
8 Self-Check — Can You Actually Do This?
Click each question to reveal the answer. Three for three means you're ready to practice.
Q1. What is the difference between the Appium UiAutomator2 and XCUITest drivers?
UiAutomator2 is the Appium driver for Android. It communicates with the Android UI framework via Google's UiAutomator2 library and works with any Android app. XCUITest is the Appium driver for iOS. It sits on top of Apple's XCUITest framework, which is how Xcode's native test tooling interacts with iOS UI. You configure which driver to use via the automationName capability in your Appium session. They share the WebDriver API surface but have different selector strategies and timing characteristics underneath.
Q2. When would you use Detox instead of Appium for a mobile project?
When the app is built with React Native. Detox runs inside the same process as the React Native bridge, giving it direct access to the JS runtime. This means it can synchronise with React Native's async rendering and avoid flaky waits that plague Appium on RN apps. Appium communicates over a network socket to a separate server, which introduces timing uncertainty. For non-React-Native apps (native Swift, Kotlin, Flutter), use Appium or the platform-native tooling instead.
Q3. How do you test push notification delivery in a mobile automation suite?
You don't automate the push delivery itself — that's the responsibility of the push notification service (APNs for iOS, FCM for Android). What you automate is the app's response to receiving a notification. In Detox, use device.sendUserNotification() to simulate a received notification and then assert on how the app handles it (navigates to the right screen, displays the content correctly, updates badge counts). For Appium, use the mobile: sendPushNotification command on iOS simulator or a custom intent on Android emulator. Never rely on real network delivery in automated tests.
9 ISTQB Mapping
ISTQB CT-TAE (Certified Tester — Test Automation Engineer)
Section 3.5 — Mobile test automation, covering tool selection criteria, driver architecture, and device management strategies. The CTAL-TAE syllabus expects candidates to distinguish between web-based and native automation approaches and to justify tool selection based on the application under test's architecture.
ISTQB CTAL-TA v3.1.2
Section 3.2.6 — Non-functional testing on mobile platforms. Exam questions focus on understanding device fragmentation, OS-version coverage strategy, and the distinction between emulator/simulator testing and real-device testing. Cloud device farms are explicitly recognised as an industry-standard approach for real-device coverage without on-premise infrastructure.