Test Data Management
Test data is where privacy law and testing craft collide. Get it wrong and you either break tests for want of realistic data, or you break the law by using real customer records in an environment that was never built to protect them.
1 The Hook
Your UAT environment has real customer data: 47,000 names, IRD numbers, and bank accounts. Refreshed every Friday. Is this normal? Yes. Is it legal under the NZ Privacy Act 2020? Almost certainly not.
This is the situation at dozens of NZ organisations right now. The database refresh is a Friday afternoon job-run: five lines of SQL, prod backup to UAT, done. Developers, contractors, offshore support staff, and occasionally demo visitors all have access. Nobody applied for consent from those 47,000 people. Nobody classified the data. Nobody masked a column.
Under IPP 10 (use limitation) of the Privacy Act 2020, that customer data was collected for one purpose — running their accounts — and using it in a test environment is a different purpose the customers never agreed to. Under IPP 5 (storage security), a UAT environment with lighter access controls than production cannot provide the same security safeguards that production data requires. Both failures are notifiable-breach territory if the data is later disclosed.
Nobody is malicious. Nobody thought about it. That is exactly the problem test data management solves.
Training a new bank teller with live customer accounts.
You would never sit a new teller in front of open live accounts to practise on — real names, real balances, real credit histories. You would give them a practice environment that behaves exactly like the real thing, but is populated with fictional people who cannot be harmed. The teller learns everything they need; no customer is exposed. Test data management builds those practice accounts so that testers never have to bring real ones into the back room.
2 The Four Test Data Problems
Teams suffer from four distinct test data failure modes. Each needs a different fix.
✗ Problem 1: No Data
The test environment is empty. Testers spend the first half of every sprint creating accounts by hand before they can test anything. Sprint velocity collapses. Edge cases are never reached because building them is too time-consuming.
Fix: Idempotent seed scripts that populate a known baseline in one command. Every new environment starts with the same dataset, already in place.
✗ Problem 2: Wrong Data
The environment has data, but it does not exercise the right scenarios. The team has hundreds of happy-path accounts and zero accounts in arrears, at a threshold boundary, or with a missing mandatory field. Real bugs hide in the gaps.
Fix: Purpose-built synthetic datasets that deliberately include boundary values, error states, and rare-but-important configurations. Design your data to your test plan, not the other way around.
✗ Problem 3: Stale Data
The environment was seeded three months ago. Since then, the schema has changed, test runs have mutated records, and the data no longer matches any realistic production state. Tests fail for data reasons, not code reasons.
Fix: Idempotent seeds (re-running them resets state to baseline) plus version-controlled seed scripts that evolve with the schema. Treat test data like code: it lives in source control and has a migration history.
✗ Problem 4: Unsafe Data
The environment contains real customer personal information. IRD numbers are real. Names are real. Bank accounts belong to living people. A laptop export, a misconfigured access control, or a careless email turns this into a notifiable privacy breach.
Fix: Never use real personal data in test environments. Use synthetic data generated from scratch, or irreversibly masked derivatives of production data, applied before the data touches the lower environment.
3 NZ Privacy Act 2020
Two Information Privacy Principles land directly on test data. You should be able to cite them by number in any meeting about a database refresh.
| IPP | Name | What it means for test data |
|---|---|---|
| IPP 5 | Storage and security | Personal information must be protected by security safeguards that are reasonable given the sensitivity of the information. A UAT environment with weaker access controls, no audit logging, and contractor access does not meet this standard for real customer data. |
| IPP 10 | Use limitation | Personal information must not be used for a purpose other than the purpose for which it was collected, unless an exception applies. Customers consented to having their data used to run their accounts, not to populate a test environment for UAT. |
Production data in a test environment is a notifiable breach risk. Under section 113 of the Privacy Act 2020, an agency must notify the Office of the Privacy Commissioner (and affected individuals) of a breach that has caused, or is likely to cause, serious harm. Exposure of IRD numbers, bank account details, or financial position to unauthorised parties in a test environment can meet that threshold. “It was only UAT” is not a defence.
The practical implication: the test data decision is a legal decision, not just an engineering one. When a developer proposes a Friday database refresh from production, the correct QA response is to name IPP 5 and IPP 10, ask whether a PIA covers this use case, and decline until a safe alternative is in place.
What counts as personal information?
Under the Privacy Act 2020, personal information is any information about an identifiable individual. In a typical NZ application this includes:
- Name, address, email, phone number
- IRD number, NHI number, passport number
- Bank account numbers and financial position
- Date of birth, gender, ethnicity
- IP address and device identifiers (these are personal information in NZ law)
- Support ticket text, chat transcripts, complaint notes — even when the name is removed, if the text contains enough context to identify the person
The last point is critical: free-text fields are where masking silently fails. You can mask every structured column perfectly and still leave a person identifiable because a notes field reads: “spoke to Aroha about her overdue account, ph 021 …”.
4 Data Masking Techniques
When you must derive test data from production records (for realistic volume or distribution), masking replaces real values with realistic-but-fake ones. Three core techniques:
| Technique | How it works | Use when | NZ example |
|---|---|---|---|
| Substitution | Replace each value with a different plausible value from a lookup dictionary (e.g., real first names replaced with other real first names) | Name, address, email fields where format realism matters | Replace “James Smith” with a randomly drawn name like “Wiremu Tane” from a NZ name dictionary |
| Shuffling | Reassign values between rows. Customer A gets Customer B’s IRD number; Customer B gets Customer C’s. Each value is still realistic; it just belongs to the wrong person. | Fields where value distribution must match production exactly (e.g., income ranges, postcode spread) | Shuffle IRD numbers across rows so the set retains realistic checksum distribution without any IRD mapping to the original person |
| Format-preserving encryption (FPE) | Deterministically encrypt a value so the output has the same format and length as the input. The same input always produces the same output (referential consistency across tables). | Keys that appear in multiple tables (customer_id in customers, orders, and support_tickets) where foreign keys must still join | FPE an IRD number: 049-688-321 becomes 083-241-097 — still 9 digits, no leading zero, same checksum pattern |
NZ-specific masking rules
IRD Number Checksum (must be preserved)
IRD numbers use a weighted checksum algorithm. If you generate or mask IRD numbers by substituting random digits, the result fails the checksum and will be rejected by any system that validates them — breaking your tests. Your masking must either: (a) use FPE that maps valid IRD numbers to other valid IRD numbers, (b) generate synthetic IRD numbers using the algorithm, or (c) use a lookup table of pre-validated fictional IRD numbers.
Checksum weights (8-digit base): 3, 2, 7, 6, 5, 4, 3, 2
Checksum weights (9-digit, when first digit is non-zero): 7, 4, 3, 2, 5, 2, 7, 6, (check digit)
Remainder 0 → check digit = 0. Remainder 1–9 → use secondary weights.
NZ Bank Account BECS Format
NZ bank accounts follow the BECS format: BB-bbbb-AAAAAAA-SS — 2-digit bank, 4-digit branch, 7-digit account, 2–3 digit suffix. Bank codes 01–38 are assigned to actual NZ banks. For test data, use bank code 00 (unassigned) with branch 0000 to produce accounts that are structurally valid but cannot be mistaken for real: 00-0000-0123456-00.
NZ Phone Numbers
NZ mobile numbers begin with 02x. For test data, use the reserved test range 021 000 0000–021 000 9999 which cannot belong to a real subscriber. NZ landlines begin with a regional code (04 = Wellington, 09 = Auckland). Use 04 000 0000 or 09 000 0000 for test landlines.
Masking tools
The @faker-js/faker package with en_NZ locale generates NZ names, addresses, and phone numbers. Combine with custom IRD/bank account generators for full NZ coverage. Ideal for Node.js test seeds and Playwright globalSetup.
The Faker package with en_NZ locale. Best for SQL seed scripts, data pipeline masking, and pytest fixtures. Write a custom provider for IRD numbers and BECS bank accounts.
Web-based data generator with custom field types, referential integrity across tables, and CSV/JSON/SQL export. Good for one-off seed files. Free tier: 1,000 rows per download. Add a custom formula for IRD checksum.
For in-database masking of production clones, a parameterised UPDATE run before the clone leaves the production network ensures unmasked data never reaches the test environment. Run in transit, not after-the-fact.
5 Synthetic Data Generation
The safest test data was never anyone’s in the first place. Synthetic data is generated from rules, not copied from real records. There is no real person behind it to re-identify, so Privacy Act obligations do not arise at all.
When to use synthetic data
- First choice in all cases: if you can satisfy the test with synthetic data, do so. It carries no privacy risk and gives you full control over the dataset.
- New features and greenfield systems: no production data exists yet; all data must be synthetic.
- Specific edge cases: scenarios that are rare in production (a customer at exactly the income threshold, a KiwiSaver member who has made no contributions for 7 years) are designed into synthetic data rather than hunted for in production.
- CI/CD pipelines: automated test runs need a deterministic seed that resets on every run. Synthetic data scripts run fast, produce the same output every time, and do not depend on a production copy.
NZ-specific generators
Standard Faker libraries do not include NZ-specific identifiers. Write your own generators for these:
Valid NZ IRD number generator (JavaScript)
Python Faker with NZ providers
6 Environment Seeding Strategies
A seed script populates a known, deterministic dataset into a test environment. Good seeds are idempotent: running them twice produces the same result as running them once. They are version-controlled alongside the application code and run as part of CI/CD setup.
SQL seeds
The simplest approach for database-backed applications. Use INSERT OR REPLACE / UPSERT semantics so the script is idempotent:
Factory patterns
In JavaScript/TypeScript test suites, object factories build test entities with sensible defaults that can be overridden:
API seeding
When there is no direct database access (microservices, third-party SaaS), seed data through the application’s own API. This validates the creation path as well as the test scenario:
Playwright globalSetup
For end-to-end test suites, run seeding once before all tests and teardown after:
UPSERT in SQL, check-before-insert in API seeding, and counter-resets in factories. A seed that fails on second run will silently corrupt your CI environment.
7 Edge Case Test Data
Bugs cluster at boundaries. The following edge cases are endemic to NZ applications and are regularly missed in test datasets. Design them in explicitly.
Timezone transitions: NZST/NZDT
New Zealand observes daylight saving. The clocks go forward 1 hour at 2:00 am on the last Sunday of September (moving to NZDT, UTC+13) and back at 3:00 am on the first Sunday of April (returning to NZST, UTC+12). Dates and times exactly at these transitions expose off-by-one and missing-hour bugs:
| Scenario | Test value | Expected behaviour |
|---|---|---|
| Spring forward (September) | 2025-09-28T02:30:00+12:00(this local time does not exist) |
System rejects, or maps to 03:30 NZDT without losing data |
| Fall back (April) | 2025-04-06T02:30:00(this local time occurs twice) |
System disambiguates correctly; no duplicate records created |
| Midnight crossing | Event timestamped 2025-09-27T23:59:59NZST |
Reported on correct calendar date (Sunday, not Monday) |
Financial year boundary (1 April)
The NZ financial year runs 1 April to 31 March. Any system that aggregates by financial year must handle the boundary correctly:
- Transaction on
2025-03-31→ FY2025 - Transaction on
2025-04-01→ FY2026 - Transaction at
2025-03-31T23:59:59NZDT(which is2025-03-31T10:59:59Z) — is it FY2025 or FY2026? Depends on whether the system uses local or UTC for the cut-off. - KiwiSaver contribution caps, tax returns, and benefit assessments all reset on 1 April.
Leap years
- Birthday:
1992-02-29— testers born on this date. Systems that calculate age or send birthday emails must handle this without crashing on non-leap years. - Financial calculations: a 30-day billing cycle starting
2024-01-31ends2024-02-29(valid in 2024) but2023-02-28in a non-leap year. Add both to your dataset. - The year 2000 is a leap year (divisible by 400); 1900 is not. Any system with a Y1900 date might have this lurking.
Unicode and macrons (ā ē ī ō ū)
Te reo Māori names containing macrons (tohutō) are common in NZ systems. They are a reliable source of encoding bugs, truncation failures, and sort-order issues:
Special characters in email addresses
Email fields connected to name fields are a common source of encoding bugs. When a customer’s name contains a macron and the system auto-generates an email, the result may contain invalid characters:
NZ edge case reference table
Keep this table as your test data checklist for any NZ application:
| Data type | Edge case value | Why it matters |
|---|---|---|
| IRD number | 000-000-000 | Zero value — must be rejected; also tests null-handling |
| IRD number | 049-688-321 | Valid checksum — structurally correct test IRD |
| NZ postcode | 0110 (Whangarei), 9999 (Invercargill) | Min/max of valid range; both extremes must resolve to a city |
| NZ postcode | 1000 | Unassigned — valid format but maps to no real location |
| Bank account | 02-0500-0890754-00 (ANZ) | Real bank/branch code — test that system doesn’t validate against live routing tables |
| Bank account | 00-0000-0000001-00 | Reserved test bank code; structurally valid, cannot be a real account |
| Phone | 021 000 0000 | Reserved mobile range; use for all test records |
| Phone | 0800 123 456 | Freephone number — edge case if system requires a mobile |
| Name | Māia Ngāti | Macrons; tests UTF-8 storage, display, and sort order |
| Name | Tūhoe-o-te-Rangi-Tūwharetoa-Arikirangi | Very long Māori name; tests column length limits (100 chars) |
| Name | O’Brien vs O‘Brien | ASCII apostrophe vs Unicode curly quote; different codepoints, same intent |
| Date | 1992-02-29 | Leap day birthday; must not crash on non-leap years |
| Date | 2025-04-01 | NZ financial year start; FY attribution boundary |
| DateTime | 2025-09-28T02:30:00+12:00 | Non-existent time during NZDT spring-forward |
8 Self-Check
Click each question to reveal the answer.
Q1. A developer wants to copy the production database into UAT on Friday afternoon to get realistic data for next week’s testing. What do you say, and which IPPs do you cite?
No. Under IPP 10 (use limitation), that customer data was collected to run customer accounts, not to populate a test environment — a different purpose the customers never agreed to. Under IPP 5 (storage security), UAT with lighter access controls and contractor access cannot provide the required safeguards for real personal information. The correct path is synthetic data or irreversibly masked derivatives applied before the data touches UAT. Cite both IPPs by name; this is a legal question, not just an engineering preference.
Q2. Your masking script updates the first_name, last_name, email, and ird_number columns. Is this sufficient? What have you missed?
Almost certainly not. The common gaps are: (1) free-text fields such as notes, complaint_text, or support_history that contain names and phone numbers in unstructured text; (2) quasi-identifier combinations — date of birth + postcode + a rare attribute can re-identify a person even with name and IRD removed; (3) phone numbers, addresses, and device identifiers which are personal information and must also be masked; (4) audit/log tables that duplicate PII from the main tables. Mask every column that could identify a person, not just the obvious structured ones.
Q3. What makes a seed script idempotent, and why does it matter?
An idempotent seed script produces the same result regardless of how many times it is run. In SQL, this means using INSERT ... ON CONFLICT DO UPDATE (UPSERT) rather than plain INSERT. In API seeding, it means checking whether the record already exists before creating it. It matters because CI/CD pipelines often run setup more than once (failed jobs, re-runs, parallel environments), and a non-idempotent seed will either fail with duplicate-key errors or create duplicate records that break test assumptions.
Q4. You are generating synthetic IRD numbers with random 9-digit numbers. Tests are failing with “invalid IRD number” validation errors. What is wrong?
IRD numbers use a weighted checksum. A random 9-digit number almost never passes it. You must use the IRD checksum algorithm to generate structurally valid test IRD numbers — either by implementing it yourself (as shown in the lesson code examples) or by using a pre-validated list of fictional IRD numbers. Never substitute random digits directly for an IRD field in any NZ system that validates the format.
Q5. Name three NZ-specific edge cases that should be in every test dataset for a customer-facing application.
Any three from: macrons in name fields (ā ē ī ō ū, as in Māori names); NZST/NZDT transition datetimes (the non-existent 2:30 am during spring-forward, or the ambiguous 2:30 am during fall-back); 1 April financial year boundary (transaction FY attribution); 29 February birthdate (leap day); IRD number with valid checksum (random digits will not pass validation); NZ bank account in BECS format using reserved test bank code 00; very long Māori compound name (up to 100 characters under the Legal Name Change Act 1995); apostrophe variants in names (ASCII vs Unicode curly quote).
9 Knowledge Graph
Test data management connects to adjacent specialisations. Explore the related lessons: