Te Tiriti in Digital Testing
Te Tiriti o Waitangi is the founding agreement of Aotearoa, and the Crown’s commitments reach into the digital services it builds. This lesson teaches you to bring partnership, Māori Data Sovereignty, and equity into a test plan — accurately, and with the respect they are owed.
1 The Hook
A government agency built a digital service to let people apply online for a support payment. The team was experienced and the build was solid — the forms worked, the calculations were right, the page loaded fast. Testing passed. It went live.
Within weeks a pattern showed up in the data. Applications from Māori applicants were being abandoned partway through at a much higher rate than applications from other groups. The same form, a very different experience. When the agency looked closely, several things had quietly stacked up. The form demanded a single permanent residential address and rejected anything else — which failed people living across multiple whānau households. It asked for relationships using categories that did not fit whānau structures, so people could not describe their actual situation truthfully. And an automated identity step leaned on data sources that had thinner coverage for Māori applicants, so they were kicked into a slow manual queue more often.
None of this was a bug in the usual sense. Every field validated. Every calculation was correct. The service did exactly what it was specified to do — and it still delivered a worse outcome to Māori than to everyone else. The specification itself carried assumptions that did not hold for the people the service was meant to serve.
Here is the lesson hidden in that story: the team tested whether the service worked, and never tested whether it worked equitably. They had no test that asked “does this produce a fair outcome across the populations of Aotearoa?” — and so a service that met every functional requirement still fell short of the Crown’s obligations under Te Tiriti o Waitangi.
Te Tiriti is not a policy footnote that lives outside the test plan. Its commitments — partnership, participation, and protection — can be translated into checks a tester runs. That is what this lesson teaches.
2 The Rule
A digital service can meet every functional requirement and still fail the people it serves, because a specification can carry assumptions that do not hold across all populations of Aotearoa. Te Tiriti o Waitangi places duties of partnership, participation, and protection on the services the Crown builds — and those duties are testable. If equity, Māori Data Sovereignty, and culturally safe design are never turned into checks with evidence, they are hopes, not assurances.
3 The Analogy
A building that passed inspection but has no ramp.
A new building can pass every structural check — the beams are sound, the wiring is safe, the consent is signed — and still have only a flight of steps at the front door. For anyone using a wheelchair, the building does not work, no matter how perfect the engineering. The inspection asked “is it built correctly?” and never asked “can everyone actually get in?”
A digital service is the same. Functional testing asks “does it work as specified?” Equity testing asks “does it work for everyone it is meant to serve?” The two are different questions, and a service can pass the first while failing the second. Te Tiriti obligations are the reason the second question is not optional for services built in Aotearoa — and like a ramp, an equitable outcome is something you design in and then verify, not something you hope was there.
4 Partnership, Participation, and Protection
Te Tiriti o Waitangi is the agreement signed in 1840 between the Crown and Māori. The principles that have come to express the Crown’s ongoing obligations are often summarised as partnership, participation, and protection. They are commonly known as the three Ps, and each translates into something a tester can act on.
Partnership
The obligation: the Crown and Māori work together, in good faith, as partners. For a digital service this means Māori are involved in shaping the thing being built, not consulted at the end. For a tester, partnership shows up as a question about process: were Māori part of defining what “working correctly” means here? If the acceptance criteria were written without Māori input on a service that affects Māori, the test basis itself may be incomplete. A tester who notices that the requirements assume one worldview can raise it as a risk — that is partnership applied at the level a tester can reach.
Participation
The obligation: Māori can take part fully and on fair terms. For a digital service this means Māori can actually use it, and use it as effectively as anyone else. This is the most directly testable of the three. It becomes equity testing — measuring whether outcomes (completion rates, approval rates, error rates, time-to-decision) are comparable across populations, and treating a gap as a defect to investigate rather than a fact to accept.
Protection
The obligation: the Crown actively protects Māori interests, including taonga. Data about Māori — and data Māori hold — is widely understood as taonga, and protection extends to it. For a tester this means Māori data is not treated as ordinary data: how it is collected, stored, used, and governed carries obligations that ordinary fields do not. Protection is where Te Tiriti meets Māori Data Sovereignty, which the next section covers.
5 Māori Data Sovereignty and Te Mana Raraunga
Māori Data Sovereignty is the principle that Māori data is subject to Māori governance. Data describing Māori people, language, resources, and knowledge is taonga, and Māori have rights and interests in how it is collected, held, used, and shared. Te Mana Raraunga, the Māori Data Sovereignty Network, has articulated principles that put this into practice. A tester does not need to recite them all, but should recognise the ideas behind them and what they imply for a system.
Whakapapa (relationships) — data has context and lineage; it relates to people and place. Implication: data stripped of its context can be misused; provenance and meaning matter, not just values.
Whanaungatanga (collective interest) — Māori data carries collective as well as individual interests. Implication: privacy is not only about the individual record; aggregate uses can affect a whānau, hapū, or iwi.
Kotahitanga (collective benefit) — data use should benefit Māori. Implication: ask who benefits from a data use, and whether that was agreed.
Manaakitanga (respect) — data is used in ways that uphold mana and are culturally respectful. Implication: how data is labelled, categorised, and displayed can cause harm even when technically correct.
Kaitiakitanga (guardianship) — Māori data is guarded for current and future generations. Implication: retention, residency, and security duties carry a long-term responsibility, not just compliance for today.
For a tester the practical upshot is concrete. A system holding Māori data should let you verify that data residency is correct (where is it stored and processed?), that access and governance controls match what was agreed with the relevant Māori partners, that categories and labels applied to people are accurate and respectful, and that consent covers the actual use. These are checks, with evidence — not abstractions.
6 Equity Testing Across Populations
Equity testing measures whether a service produces comparable outcomes across the populations it serves. It is the most directly testable expression of participation, and it is where the opening story would have been caught. The method is straightforward: identify the outcomes that matter, break them down by population, and treat a meaningful gap as a defect to investigate.
What to measure
- Completion and abandonment: for the support-payment service, what share of applicants complete the form, broken down by population? A higher abandonment rate for one group is a signal, not noise.
- Approval and routing outcomes: are people in one group routed to slow manual review, or declined, at a higher rate for comparable cases?
- Error and rejection rates: does one group hit validation errors more often — often a sign the form assumes circumstances that do not fit everyone?
- Time-to-outcome: does one group wait longer for a decision on equivalent applications?
How to test it well
Equity testing has a few rules that keep it honest. Compare like with like — a gap only matters once you have controlled for the things that legitimately differ between cases. Measure against a recent, real picture of who uses the service, not an assumption. Treat a gap as the start of an investigation, not a verdict — the cause might be the form, the data sources behind an automated step, or an upstream process. And design the service so the data needed to test equity can be measured at all; you cannot check a gap you have no way to see.
7 The Algorithm Charter and Its Tiriti Commitments
The Government Algorithm Charter for Aotearoa New Zealand is a public commitment that signatory agencies make about how they use algorithms that affect people. It is directly relevant to a tester because it turns several of the ideas in this lesson into things an agency has formally signed up to — which means they become things you can test against.
The Charter’s commitments most relevant to Tiriti-aligned testing include:
- A Te Ao Māori perspective: agencies commit to embedding a Te Ao Māori perspective in the development and use of algorithms. For a tester, this is a check on process and design — was that perspective actually part of how the algorithm was built and validated?
- Identifying and managing bias: agencies commit to actively working to identify and address unintended bias. This is equity testing made into an obligation — the agency has said it will look for bias, so the evidence that it did becomes a legitimate test artefact.
- Transparency: agencies commit to being transparent about how and when algorithms are used. A tester can verify that the documented behaviour matches the actual behaviour, and that the explanation given to people is accurate.
- Data that is fit for purpose: agencies commit to using data that is fit for purpose, understanding its limitations. This links straight to the data quality and representativeness ideas a tester already knows.
The Charter does not replace Te Tiriti — it is one expression of the Crown’s broader obligations in the specific area of algorithms. But for a tester it is useful precisely because it is concrete: where an agency has signed it, “did you embed a Te Ao Māori perspective and look for bias?” is a fair test question with a documented commitment behind it.
8 Building Tiriti-Aligned Test Cases
A Tiriti-aligned test case looks like any rigorous test case — it just targets equity, data sovereignty, or culturally safe design instead of a function. The system under test might be a whole service rather than a single screen, the acceptance criterion is often a measured comparison across populations, and the evidence is a measurement plus a record of who reviewed it.
Here is an equity test case for the support-payment service from the opening story:
Obligation: Te Tiriti — participation (equity of outcome)
Type: Equity / outcome-parity test
Description: Verify that application completion rates for the online support-payment
form are comparable across populations, with specific attention to Māori
applicants, controlling for application type.
Acceptance criteria: For comparable application types, the completion rate for Māori
applicants is within 5 percentage points of the all-applicant rate. Any
gap beyond tolerance is raised for investigation of cause.
Evidence required: Completion-rate comparison table by population and application type;
the query used; the date and source of the data snapshot; reviewer sign-off.
Traceability: Risk R-02 (service produces inequitable outcomes for Māori) in the
project risk register; Algorithm Charter bias commitment.
Result: [Pass / Investigate] — populations outside tolerance listed.
Notice the shape: the acceptance criterion is a measured tolerance across populations, not “the form works”; the result is “investigate” rather than a blunt fail, because an equity gap is a signal whose cause you must find; the evidence is a reproducible measurement; and the case traces to a named risk and a Charter commitment. Those properties are what make it a real test case rather than a good intention.
9 Common Mistakes
🚫 Treating Te Tiriti as a policy concern that lives outside the test plan
Why it happens: It is framed as legal or strategic, so it feels like someone else’s responsibility.
The fix: The obligations translate into checks — equity comparisons, data-governance verification, culturally safe design review. A tester who turns them into test cases with evidence is doing exactly the work that proves the obligation was met.
🚫 Assuming “the service works” means “the service works for everyone”
Why it happens: Functional testing passes, so the service looks finished.
The fix: Functional correctness and equity are different questions. Always add the second — does it produce comparable outcomes across the populations of Aotearoa? — and measure it, do not assume it.
🚫 Treating Māori data as ordinary data
Why it happens: In a database every field looks the same, so data sovereignty is invisible to a purely technical view.
The fix: Māori data is taonga, governed by Māori Data Sovereignty. Verify who governs it, where it lives, who can access it, and whether consent covers the use — with the same rigour you would apply to any other control.
🚫 Designing categories that force people to answer wrongly, then trusting the data
Why it happens: Fixed dropdowns and rigid relationship categories are easy to build and validate.
The fix: If a form cannot represent whānau structures or living arrangements truthfully, people enter wrong answers and the data is corrupted at source. Culturally safe design is a testable quality — check that people can describe their real situation, and treat forced-wrong answers as a defect.
10 Now You Try
Three graded exercises — spot the risk, fix the practice, then design the checks. Write your answer, run it for AI feedback, then compare to the model answer.
Read the description of a test setup for a fictional public-sector housing-register service below. Identify 3 Tiriti or data-sovereignty risks in the way this is being built and tested, and name the obligation or principle each one touches.
People apply online to join the housing register. The form requires one residential address and a single “head of household,” and offers a fixed list of relationship types. Acceptance criteria were written by the delivery team from the policy document; no Māori partners were involved in defining what a correct outcome looks like. Test data is a copy of last year’s real applicant records, including ethnicity. The team tests that the form validates, calculates priority correctly, and loads quickly. There is no test of completion or approval rates by population. The applicant data, including iwi affiliation where given, is stored in an overseas cloud region because it was the cheapest option, and there is no record of who governs access to it.
List 3 risks and the obligation or principle each touches:
Show model answer
There are at least five real risks here; any three well-explained earns full marks. 1. Participation / equity — There is no test of completion or approval rates by population, so an inequitable outcome for Maori (the opening-story failure) would go undetected. The form's single-address, single-head-of-household, fixed-relationship design is exactly the kind that forces wrong answers and drives an equity gap. Obligation: participation, tested as equity. 2. Partnership — Acceptance criteria were written with no Maori involvement on a service that affects Maori. The test basis may encode assumptions that do not hold, and "correct" was defined without the partner. Obligation: partnership. 3. Protection / Maori Data Sovereignty — Applicant data including iwi affiliation is stored offshore with no record of who governs access. Residency and governance of Maori data as taonga are unverified, and the choice was made on cost alone. Obligation: protection, expressed through Maori Data Sovereignty (rangatiratanga, kaitiakitanga). Bonus risks: Culturally safe design — the fixed categories cannot represent whanau truthfully, corrupting data at source. Privacy/provenance — real applicant records including ethnicity used as test data with no mention of consent or minimisation (this links to Lesson 3). The trap: a team testing only validation, calculation, and load would pass everything and still ship a service that fails Maori and mishandles taonga.
A team is using the test-data practice described below for a fictional iwi-facing grants portal that holds data governed under Māori Data Sovereignty. Rewrite the practice into a compliant approach, addressing data governance, residency, minimisation, and respect. Explain what you changed and why.
“For testing we took a full copy of the production database — all applicant records including iwi affiliation, whānau details, and contact info — and loaded it into a shared test environment hosted in whatever cloud region was cheapest. Anyone on the project can access the test environment. We keep the data indefinitely so we can re-run tests.”
Rewrite as a compliant test-data approach:
Show model answer
A compliant approach addresses every dimension the original ignored. - Governance: Confirm who governs this data under Maori Data Sovereignty and that the agreed governance covers any use in testing. Do not assume the project may copy taonga into a test environment just because it is technically possible. - Residency: Store and process all test data within approved NZ jurisdictions as agreed with the data's kaitiaki. "Cheapest region" is not an acceptable basis for taonga; residency is a kaitiakitanga obligation, not a cost decision. - Minimisation / de-identification: Do not copy the full production set. Use the minimum data the test actually needs, and de-identify or synthesise wherever the real values are not required. Iwi affiliation and whanau detail should not appear in test unless a specific test genuinely needs them and that use is governed. - Access control: Restrict the test environment to people who need it, with named, auditable access — not "anyone on the project." - Retention: Hold test data only as long as the testing requires, then dispose of it securely. Indefinite retention of taonga in a test system is a long-term kaitiakitanga failure. What changed and why: the original treated Maori data as ordinary, cheap, and freely copyable. The compliant version treats it as taonga — governed, kept onshore, minimised, access-controlled, and time-limited — which is what protection and Maori Data Sovereignty require, and is also good privacy practice for any sensitive non-prod data (Lesson 3).
Design a set of 4 equity / Tiriti-aligned test cases for a fictional automated student-allowance eligibility service. Each case should have at least an ID, the obligation it addresses, what it verifies, a measurable acceptance criterion, and the evidence required. Cover outcome parity, an automated-decision step, culturally safe design, and data governance.
Show model answer
EQ-01 | Obligation: participation (equity) | Verifies: approval rates for comparable applications are parable across populations | Acceptance criteria: for comparable application types, the approval rate for Maori applicants is within 5 percentage points of the all-applicant rate; any gap is raised for investigation | Evidence required: approval-rate comparison table by population and type; query; data snapshot date; reviewer sign-off EQ-02 | Obligation: participation + Algorithm Charter (bias) | Verifies: the automated eligibility step does not route or decline one population disproportionately for comparable cases | Acceptance criteria: routing-to-manual and auto-decline rates for Maori applicants are within tolerance of the all-applicant rate for matched case profiles; otherwise investigate the data sources behind the step | Evidence required: routing/decline breakdown by population; matched-case method; reviewer sign-off EQ-03 | Obligation: protection / culturally safe design | Verifies: applicants can describe their living and whanau situation truthfully without being forced into a wrong answer | Acceptance criteria: a defined set of real-world whanau/living scenarios can each be entered accurately; 0 scenarios require a knowingly false answer to proceed | Evidence required: scenario walkthrough results with the scenarios listed; defects raised for any forced-wrong path EQ-04 | Obligation: protection / Maori Data Sovereignty | Verifies: applicant data is stored, governed, and accessed as agreed | Acceptance criteria: all applicant data resides in approved NZ jurisdictions; access matches the agreed governance; consent covers the use; 0 exceptions | Evidence required: residency attestation; access-control review; governance/consent reference; reviewer sign-off Strong plans: each case is specific, has a measurable criterion, names concrete evidence, and together they cover outcome parity (EQ-01), the automated step (EQ-02), culturally safe design (EQ-03), and data governance (EQ-04). Weak plans say "check it is fair for Maori" four times without a measurable criterion — that is the difference being marked.
11 Self-Check
Click each question to reveal the answer.
Q1: How can a digital service pass all functional testing and still fail its Te Tiriti obligations?
Because functional testing asks “does it work as specified?” while Te Tiriti obligations require it to also work equitably across the populations of Aotearoa. A specification can carry assumptions — a single address, fixed relationship categories, a data source with thin coverage for some groups — that meet every requirement and still deliver a worse outcome to Māori. The fix is to test equity explicitly, not assume it.
Q2: Name the three Ps and what each means for a tester.
Partnership — were Māori part of defining what “correct” means here? Participation — can Māori use the service as effectively as anyone else, and can we measure it (equity testing)? Protection — is Māori data protected as the taonga it is (Māori Data Sovereignty)?
Q3: What is Māori Data Sovereignty, and what does it mean for a system holding Māori data?
It is the principle that Māori data is taonga and subject to Māori governance — Māori have rights in how it is collected, held, used, and shared. For a system it means verifying who governs the data, that residency is correct, that access and sharing controls reflect what was agreed, and that consent covers the actual use. Te Mana Raraunga expresses this through principles such as rangatiratanga and kaitiakitanga.
Q4: What does equity testing measure, and why is a gap treated as “investigate” rather than an automatic fail?
It measures whether outcomes — completion, approval, error, and decision-time rates — are comparable across populations for like-for-like cases. A gap is a strong signal that something is wrong, but its cause could be the form, the data behind an automated step, or an upstream process, so it triggers an investigation to find and fix the cause rather than a blunt pass/fail.
Q5: Why is the Government Algorithm Charter useful to a tester?
Because it turns several Tiriti-aligned ideas into commitments a signatory agency has formally made — embedding a Te Ao Māori perspective, actively identifying and managing bias, being transparent, and using data fit for purpose. Those commitments become legitimate test targets: you can ask for the evidence that the agency did what it signed up to do.
12 Interview Prep
Real questions asked in NZ QA interviews for government and public-sector roles. Read the model answers, then practise your own version.
“How would you bring Te Tiriti o Waitangi into a test plan for a public-facing service?”
I’d translate the obligations into checks rather than treat them as policy that sits outside testing. Partnership becomes a question about the test basis — were Māori involved in defining what a correct outcome looks like, and if not, that is a risk I raise. Participation becomes equity testing — I measure completion, approval, and error rates across populations for comparable cases and treat a gap as a defect to investigate. Protection becomes data-governance verification — I check that Māori data is held, governed, and accessed as agreed under Māori Data Sovereignty. Each of those produces evidence, which is what shows the obligation was actually met rather than assumed.
“What is Māori Data Sovereignty, and how does it change how you handle data in testing?”
It is the principle that Māori data is taonga and subject to Māori governance — Māori have rights and interests in how it is collected, stored, used, and shared. In testing it changes how I treat that data in non-prod: I do not copy it freely. I confirm who governs it and whether testing use is covered, keep it within approved NZ jurisdictions, minimise or de-identify so the real values appear only where a test genuinely needs them, restrict access to named people, and dispose of it when the testing is done. Concepts like kaitiakitanga make clear that this is a long-term guardianship duty, not a box-tick for today.
“A service passed all its functional tests but a regulator is asking whether it’s fair to Māori. How do you answer?”
Functional pass and equity are different questions, so I’d answer with evidence, not assurance. I’d show an outcome-parity comparison — completion, approval, routing, and error rates broken down by population for like-for-like cases, measured against a recent real snapshot of who uses the service. If the gaps are within tolerance, that is the evidence. If a gap shows up, I’d show the investigation into its cause — often a form that forces wrong answers or an automated step relying on data with thin coverage for some groups — and the fix. If the agency has signed the Algorithm Charter, I’d also point to the bias-identification work the Charter commits them to. “Every function passed” is not an answer to “is it fair?” — a measured equity result is.