Senior · Complex System Integration

Advanced API Testing Patterns

Junior testers check that endpoints exist. Senior testers check that systems talk reliably. Master contract testing, stateful flows, security vulnerabilities, and async patterns.

Senior ISTQB CTAL-TA Ch. 5 — K3 Apply ~15 min read + exercise

1 The Hook — Why This Matters

In 2023, a major NZ fintech company released a payment API that passed all junior-level tests: endpoints responded, status codes were correct, data was persisted. But in production, the system began losing money silently. Customers would create a payment order, pay successfully, but the webhook that notified the merchant never fired. Because of eventual consistency issues in the async notification system, some notifications were dropped.

Three days later, the company had processed $480,000 in payments but only notified merchants of $312,000. The missing notifications meant unsent invoices, unreconciled accounts, and furious customers. The root cause? No one tested the webhook delivery pattern. No one validated idempotency keys. No one checked what happened when a notification failed and was retried. The company lost $168,000 in reconciliation costs and compensation.

A senior tester would have designed the test strategy differently. Not just "does the API return 200?" but "does the entire flow — from payment to webhook to merchant reconciliation — work under real-world conditions?" That is what this page teaches.

2 The Rule — The One-Sentence Version

Test the contract between systems, not just the response format. Verify that systems remain consistent, idempotent, and observable under failure conditions.

At the junior level, you test individual endpoints in isolation. At the senior level, you test integration patterns: how does the frontend react when the API is slow? What happens to the user's data if a webhook fails? Can two clients safely make concurrent requests? Does the system recover gracefully from network timeouts? These patterns determine whether your API works reliably in production.

3 The Analogy — Think Of It Like...

Analogy

Testing a postal service.

Junior testing: "I send a letter to 123 Waitemata Road. It arrives." Senior testing: "I send 100 letters to the same address. Do they all arrive? Do they arrive in order? What if the postman crashes halfway through the route? What if the recipient doesn't acknowledge receipt — does the postman try again? What if I send the same letter twice by accident — does the recipient open two copies or notice they're identical? What if the recipient's address changes mid-delivery?" Integration testing is not about individual pieces; it's about whether the whole system functions as a coherent whole.

4 Watch Me Do It — Advanced Patterns

Pattern 1: Stateful API Testing (Sequence of Calls)

Real users don't make isolated API calls. They follow workflows: create an account, verify email, set up billing, then make a purchase. Each step depends on the previous step.

# Step 1: Create user account
POST /api/v1/users
{ "email": "alice@test.co.nz", "password": "Secure123" }
Response: 201 Created, returns user_id = "usr_123"

# Step 2: Verify email (using token from email, simulated)
POST /api/v1/users/usr_123/verify-email
{ "token": "verify_abc123" }
Response: 200 OK, email_verified = true

# Step 3: Set up payment method
POST /api/v1/users/usr_123/payment-methods
{ "card": "4111...", "expiry": "12/26" }
Response: 201 Created, returns payment_id = "pm_456"

# Step 4: Create order using the payment method
POST /api/v1/orders
{ "user_id": "usr_123", "payment_id": "pm_456", "items": [...] }
Response: 201 Created, order_id = "ord_789"

# Step 5: Verify the order was stored
GET /api/v1/orders/ord_789
Response: 200 OK, includes user_id=usr_123, payment_id=pm_456

The critical test here: does Step 5 see the user_id and payment_id from earlier steps? If not, data was lost between calls. Postman Collections or API test suites (Newman, pytest) let you chain calls and validate each response before moving to the next step.

Pattern 2: Contract Testing (Frontend ↔ Backend Agreement)

The frontend and backend must agree on the API contract: what fields are required, what types they are, what errors are possible. Contract tests verify this agreement without needing both teams to run at the same time.

Test Layer What It Checks When to Use
Response ContractDoes the API return the promised fields? Are they the right types? Is the structure valid JSON Schema?Every API endpoint, before integration testing
Error ContractWhen validation fails, does the API return the promised error status and format?Every error path (400, 401, 403, 409, 500)
Backwards CompatibilityCan v1.5 clients still work with v2.0 API? Are deprecated fields still supported with warnings?Before breaking changes, during versioning

Pattern 3: Security Testing at the API Layer

  1. SQL Injection Try inserting SQL fragments into string fields: "name": "Robert'; DROP TABLE users;--". The API should escape the input or reject it with a 400 error.
  2. Authentication Bypass Try requests without an API key, with an expired token, or with another user's token. The API must return 401 Unauthorized.
  3. Authorization (horizontal privilege escalation) Can user A access user B's data? Try: GET /api/v1/users/user_b_id when authenticated as user A. The API should return 403 Forbidden.
  4. Rate Limiting Send 100 requests in 1 second. The API should return 429 Too Many Requests. Check that rate limits are enforced per API key and reset correctly.
  5. Data Exposure in Errors Trigger a 500 error. Does the error message expose internal paths, database names, or stack traces? It shouldn't.

Pattern 4: Async and Webhook Testing

Many APIs don't return results immediately. Instead, they accept the request, queue the work, and call your webhook when done. This requires a different test approach.

# Client initiates a long-running operation
POST /api/v1/reports/generate
{ "type": "monthly-transactions", "month": "2026-04" }
Response: 202 Accepted, returns request_id = "req_xyz123"

# Client polls for status (or waits for webhook)
GET /api/v1/reports/req_xyz123
Response: 200 OK
{ "status": "processing", "progress": 45 }

# Later, webhook fires when complete
POST https://client-webhook.example.com/on-report-ready
{ "request_id": "req_xyz123", "report_url": "https://..." }

# Test checklist:
# 1. Does 202 response include a request_id to track progress?
# 2. Can the client poll the status endpoint?
# 3. Does the webhook fire with the correct data?
# 4. Is the webhook idempotent (if it fires twice, client handles it)?
# 5. What happens if the webhook endpoint is down? Does the API retry?

Pattern 5: Idempotency (Handling Retries Safely)

When a network fails after you send a POST request, how do you know if it was received? Did the server crash before responding? You retry — but now you risk creating duplicate records. The solution is idempotency.

# Client sends request with idempotency-key header
POST /api/v1/payments
Headers: idempotency-key: "ik_9f8a7b6c5d4e3f"
{ "amount": 99.50, "currency": "NZD" }
Response: 201 Created, payment_id = "pay_123"

# Network fails. Client retries with the SAME idempotency-key
POST /api/v1/payments
Headers: idempotency-key: "ik_9f8a7b6c5d4e3f"
{ "amount": 99.50, "currency": "NZD" }
Response: 200 OK, payment_id = "pay_123"  # SAME payment, not a duplicate!

# Test: Verify idempotency by:
# 1. Send the same request twice with the same idempotency-key
# 2. Confirm you get back the same resource ID both times
# 3. Verify only one payment was actually charged

Pattern 6: Privacy Act 2020 Compliance (NZ Context)

If your API handles personal data (names, emails, IRD numbers, health info), you must test compliance with the NZ Privacy Act 2020.

Principle What to Test
Collection (Principle 1)Can users see why their data is collected? Is consent recorded? Can you delete data on request?
Use Limitation (Principle 2)Does the API prevent using personal data outside the stated purpose? E.g., if data was collected for "send newsletter," can it be used for "sell to third party"?
Data Security (Principle 9)Is personal data encrypted in transit (HTTPS) and at rest? Are API keys rotated? Are audit logs kept for access?
Right to Access (Principle 7)Can users request a copy of all their personal data in machine-readable format? (Right to Data Portability)
Right to Deletion (Principle 9)Can users request deletion of their personal data? Test: POST /api/v1/users/me/delete-all-data returns 200 OK and data is actually deleted (not just marked as deleted).
Pro tip: Use contract testing libraries like Pact to define the API contract once and verify it from both the client and server sides. This prevents "I tested the endpoint" and "I built the endpoint differently" from going out of sync.

5 Testing Patterns — Which to Use When

✅ Advanced patterns to test...

  • Stateful flows (multi-step workflows)
  • APIs handling payments or sensitive data
  • Async operations (queues, webhooks)
  • Systems with external dependencies
  • Performance-critical endpoints
  • APIs with multiple versions in production

❌ Don't overengineer...

  • Junior CRUD APIs with no state
  • Internal-only APIs not used by clients
  • Endpoints that rarely change
  • Systems with no legal/compliance requirements
  • Early prototypes before the spec is stable

6 Common Mistakes — Don't Do This

🚫 Testing each endpoint in isolation, forgetting workflows

I used to think: If GET /users/{id} works and POST /users works, the API is fine.
Actually: Real users follow workflows: create → verify → update → delete. Test the sequence. Does Step 3 see data from Step 1? What happens if Step 2 fails — is Step 3 still valid?

🚫 Ignoring async patterns and webhooks

I used to think: If the API returns 202 Accepted, the job will complete.
Actually: Async APIs are where silent failures hide. Test the entire flow: request → queue → process → webhook. What happens if the webhook endpoint goes down? Does the API retry? How many times? Is it idempotent?

🚫 Not testing idempotency and retry logic

I used to think: If I send a request and get a response, it worked.
Actually: Network failures happen. A client might retry a payment three times by accident. If the API isn't idempotent, you charge the customer three times. Always use idempotency keys for state-changing operations.

When advanced API testing fails

You designed tests for a stateful flow but they fail intermittently. Usually means: (1) state is being shared between tests (use setup/teardown), (2) timing assumptions (async operations need polling or callbacks, not sleep), (3) missing data validation (each step checks the previous step's output). Isolate the failing step, verify its inputs and outputs, then rebuild the sequence.

7 Now You Try — Design a Webhook Test

🎯 Interactive Exercise

Scenario: An invoice API accepts POST /api/v1/invoices and generates a PDF asynchronously. When ready, it calls your webhook at https://your-app.example.com/webhooks/invoice-ready with the invoice ID and PDF URL.

Task: List five test cases for this webhook integration. Consider: success, retry, idempotency, and failure scenarios.

Sample webhook test cases:

  1. Happy path: Invoice is created, PDF is generated, webhook fires with correct invoice ID and URL. Your app receives it and stores the PDF URL.
  2. Webhook timeout: Invoice API calls your webhook but your endpoint takes 30 seconds to respond. Verify the API retries and doesn't time out too early.
  3. Webhook down: Your webhook endpoint returns 500 error. Does the API retry? How many times? At what intervals?
  4. Idempotency: Webhook fires twice with the same invoice ID (e.g., auto-retry triggered twice). Your app receives two webhook calls. Verify you only store the PDF once, not twice.
  5. Duplicate requests: Two concurrent invoice requests for the same invoice ID. Verify only one PDF is generated, not two. The second request should return the same URL (idempotent).

8 Self-Check — Can You Actually Do This?

Click each question to reveal the answer. If you got all three, you're ready to architect API test strategies.

Q1. What is the difference between contract testing and integration testing?

Contract testing: Verifies the API contract (request/response format) without needing other systems to be deployed. Frontend and backend teams can run tests independently. Integration testing: Tests the entire workflow with all systems connected and running. Found later, is more expensive to set up, but catches real-world issues.

Q2. Why is idempotency crucial for payment APIs?

Networks fail. A client might retry a payment request after a timeout, even if the first request was processed. Without idempotency, the same payment is charged twice. With idempotency keys, the second request with the same key returns the first payment's ID and isn't re-processed. This prevents duplicate charges.

Q3. What three things must you test when integrating webhooks?

1. Delivery: Does the webhook actually fire when the event happens? 2. Retry: If the webhook endpoint is down, does the API retry? How many times? 3. Idempotency: If the webhook fires twice (accidental retry), can your client handle it without processing the event twice?

9 Interview Prep — Senior-Level Questions

Q. "How do you test a stateful workflow across multiple API endpoints?"

I use Postman Collections or automated test suites (Newman, pytest) to chain requests. Each step validates the previous step's output before moving to the next. I parameterise values (store user_id from Step 1, use it in Step 2) and assert that data persists correctly. I also test failure modes: what if Step 2 fails? Can Step 3 recover, or is the workflow broken?

Q. "How do you test an async API with webhooks?"

I mock the webhook endpoint or use a service like RequestBin to capture live webhook calls. I verify: (1) the webhook fires with the correct data, (2) it fires within the documented time window, (3) it retries if the endpoint is down, (4) my client idempotently handles duplicate webhooks. I also test negative scenarios: what if the webhook never fires? Can my client poll the status endpoint as a fallback?

Q. "What is contract testing and why is it important?"

Contract testing verifies the API contract (request/response structure) between clients and servers. Frontend and backend teams write tests against the same contract, then run them independently. This lets teams deploy without coordinating, which is critical for continuous deployment. Tools like Pact define contracts once and verify them from both sides.

Q. "How do you balance performance testing with security testing at the API layer?"

They're often inseparable. Performance tests reveal whether rate limiting works correctly (send 1000 requests/sec, verify 429 kicks in). Security tests verify you can't bypass authentication (no auth key → 401) or escalate privileges (user A can't access user B's data). I run security scans first (check for obvious vulnerabilities), then performance tests under realistic load to ensure the system degrades gracefully, not catastrophically.