Microservices Testing

1 The Hook — Why This Matters

In 2023, a major NZ fintech company deployed a payment processing system built on microservices: a Payment Service, Fraud Detection Service, Notification Service, and Ledger Service. Each team owned their service, deployed independently, and trusted that interfaces were stable. Everything worked in production for three months. Then, without warning, 40% of payments were failing.

Investigation revealed: the Fraud Detection Service had added a new required field, riskScore, but hadn't informed the Payment Service. The Payment Service was calling Fraud Detection without the field. Fraud Detection rejected the request with a cryptic error. The Payment Service retried, then timed out, then crashed. Customers couldn't pay. The ledger got out of sync. Recovery took six days.

Microservices testing finds these breaks before they hit production. You must test service boundaries, contract violations, failure modes, and data consistency across multiple services. A single failing service can bring down the entire system.

2 The Rule — The One-Sentence Version

Microservices testing spans three layers: (1) Contract Testing — does each service provide what consumers expect? (2) Integration Testing — do multiple services work together under normal and failure conditions? (3) Observability Testing — can you trace, debug, and understand what happened when things fail? Test layers in isolation: contract tests are fast and run per-service, integration tests are slower and run across services, observability tests validate that you can detect and debug problems.

3 The Analogy — Think Of It Like...

4 Watch Me Do It — Step by Step

Here is a real NZ e-commerce example: an Order Service calls an Inventory Service and a Shipping Service. Follow these steps to test the boundaries.

1. Define and test service contracts Use PACT (consumer-driven contracts). The Order Service (consumer) defines: "I expect Inventory Service to accept a POST /check-stock with {sku, qty} and return {available: boolean, quantity: number}." Write a PACT test that mocks Inventory Service. Your test passes locally. Then, in CI, Inventory Service runs the same PACT against its real implementation. If the contract is broken, the build fails before anyone merges.

   // Order Service (consumer) PACT test
   const pact = new Pact({consumer: 'OrderService', provider: 'InventoryService'});
   pact.addInteraction({
     state: 'product SKU-001 has 50 units',
     uponReceiving: 'a stock check request',
     withRequest: {method: 'POST', path: '/check-stock', body: {sku: 'SKU-001', qty: 5}},
     willRespondWith: {status: 200, body: {available: true, quantity: 50}}
   });
   expect(inventoryClient.checkStock('SKU-001', 5)).resolves.toEqual({available: true, quantity: 50});

2. Test backward compatibility When Inventory Service adds a new field (e.g., warehouseId), Order Service must still work if it ignores the field. Test: deploy Inventory Service with the new field, run Order Service tests. They should pass. Then, add warehouseId to the response and verify Order Service ignores unknown fields without breaking.

   Found: Order Service was deserializing the response using strict schema validation. When Inventory Service added warehouseId, Order Service crashed trying to deserialize an unexpected field.

3. Test service boundaries with network mocks Use WireMock or Mountebank to simulate service behaviour. Test happy path: Order Service calls Inventory, gets a response, calls Shipping. Test failures: Inventory Service returns 500, Shipping Service times out, Inventory responds with a malformed JSON. For each failure, verify Order Service handles it gracefully (retries, circuit break, fallback, or explicit error).

   // WireMock setup for Inventory failure
   stubFor(post(urlEqualTo('/check-stock'))
     .willReturn(aResponse()
       .withStatus(503)
       .withBody("Service Unavailable")));
   
   // Order Service should retry or fallback
   expect(orderService.createOrder({...})).rejects.toThrow('InventoryServiceUnavailable');

4. Test timeout and resilience patterns If Inventory Service is slow (responds after 10 seconds), Order Service should timeout after 3 seconds, not wait forever. Test: configure WireMock to delay responses by 10 seconds, verify Order Service times out and handles it. Test circuit breaker: if Inventory Service fails 5 times, stop calling it for 60 seconds (circuit open). After 60 seconds, try once (circuit half-open). If it succeeds, circuit closes.

   Pattern: Use timeout + retry + circuit breaker. Timeout alone risks cascading failures. Retry alone risks overwhelming a failing service. Circuit breaker alone risks silent failures. Combined, they prevent cascades and allow graceful recovery.

5. Test eventual consistency and data synchronization When Order Service places an order, it writes to its own database, then publishes an "OrderCreated" event. Inventory Service listens and decrements stock asynchronously. If the event broker is down, or Inventory Service is slow, the data falls out of sync. Test: place an order, verify it's written to Order database, then verify (after a delay) that Inventory stock is decremented. Test the failure case: Order succeeds, but event broker is down. Inventory stock is not decremented. When the event broker recovers, orders should be reprocessed.
6. Test with Docker Compose locally Spin up all three services (Order, Inventory, Shipping) in Docker Compose on localhost. Run integration tests against real service instances, not mocks. This catches environment-specific issues: database connection strings, service discovery, startup order, and timing.

   # docker-compose.yml
   services:
     order-service:
       build: ./order-service
       ports: ["8001:8080"]
       environment:
         INVENTORY_URL: "http://inventory-service:8080"
     inventory-service:
       build: ./inventory-service
       ports: ["8002:8080"]
     shipping-service:
       build: ./shipping-service
       ports: ["8003:8080"]
       depends_on: [order-service, inventory-service]

7. Test distributed tracing and correlation IDs Assign a unique correlation ID to each order request. Pass the ID through all service calls. When debugging a failed order, search logs across all services for the same correlation ID. Verify that each service logs the ID and passes it downstream. Test: create an order with correlation ID abc123, verify it appears in Order Service logs, Inventory Service logs, and Shipping Service logs.

   Pattern: Use OpenTelemetry or similar to auto-inject correlation IDs. Without this, debugging is impossible: you see a failed order in logs but can't trace what happened in downstream services.

5 When to Use It / When NOT to Use It

6 Common Mistakes — Don't Do This

7 Now You Try — Interview Warm-Up

8 Self-Check — Can You Actually Do This?

Click each question to reveal the answer. If you got all three, you're ready to test microservices.

9 Interview Prep — Common Questions