Testing AI Systems — ISO/IEC 42119 in Practice
The first NZ-localised guide to testing AI systems against the international standard.
ISO/IEC TS 42119-2:2025 is the new technical specification for testing AI. It extends the ISO/IEC/IEEE 29119 software testing series into the AI domain, with a risk-based approach and AI-specific test types that traditional testing never covered — data representativeness, model drift, fairness, and explainability. This module shows you how to apply it on real NZ systems.
Aligned to ISO/IEC TS 42119-2:2025
Why AI Testing Is Different
Deterministic versus probabilistic systems. The five AI-specific failure modes traditional testing misses. The AI lifecycle, how 42119 extends 29119, and the risk register that drives test scope.
~30 min read · ~70 min with exercises · ISO/IEC 42119
Lesson 2Data Quality Testing
Representativeness, provenance, and label correctness testing. The data quality dimensions that map to AI failure modes. What a data test case looks like, and the audit-ready evidence 42119 requires.
~30 min read · ~70 min with exercises · ISO/IEC 42119
Lesson 3Model Testing
Model performance, adversarial, and explainability testing. Accuracy, precision, recall, and F1 in plain English. Drift testing and continuous validation — the model that fails silently after go-live.
~35 min read · ~80 min with exercises · ISO/IEC 42119
Lesson 4Bias and Fairness Testing
Where AI bias comes from. Counterfactual fairness and demographic parity testing. Protected characteristics under the NZ Human Rights Act 1993. Fairness as a testable quality characteristic, not an opinion.
~30 min read · ~75 min with exercises · ISO/IEC 42119
Lesson 5Audit-Ready Test Artefacts
The mandatory fields a 42119 test case carries. Risk-based traceability. What a test summary report looks like to an FMA or RBNZ auditor. Evidence requirements for every AI test type.
~30 min read · ~70 min with exercises · ISO/IEC 42119
Lesson 6Applying 42119 in a Real NZ Project
An end-to-end walkthrough: risk register in sprint 1, data tests during development, fairness testing before go-live, drift detection post-deployment. What good looks like — and the 42119 roadmap ahead.
~35 min read · ~80 min with exercises · ISO/IEC 42119
A standard for a new kind of system
Traditional software testing was built for systems that do the same thing every time. AI systems do not. The same input can produce a different output. The system is shaped by its training data, not just its code. It can keep learning after go-live, and its behaviour can drift without anyone touching a line of code. It can be biased. It can fail silently.
ISO/IEC TS 42119-2:2025 is the international response to that gap. It does not replace the testing you already know — it builds on ISO/IEC/IEEE 29119 and adds the AI-specific test types those standards never had. It is risk-based throughout, it ties to the ISO/IEC 25059 AI quality model, and it requires test artefacts a regulator can actually read.
NZ testers need this now. The FMA, the RBNZ, the Government Algorithm Charter, and public sector procurement are all moving in the same direction: if an AI system makes or shapes a decision about a person, you will be asked to show how it was tested. This module teaches you to answer that question — with NZ examples throughout, on the systems you actually work on.
The Test with AI modules
GenAI Foundations
How large language models work, and what they can and cannot do for testers.
Ch 3Managing AI Risks
Hallucination, bias, data privacy under the NZ Privacy Act 2020, and non-determinism.
Ch 5Adopting GenAI
Shadow AI, GenAI test strategy, model selection, and building AI capability in a test team.