Multi-Vendor ERP Testing
Modern NZ ERP programmes rarely involve just one vendor. Oracle Fusion talks to Salesforce. Salesforce talks to a legacy HR system via middleware. The SI configured everything. Nobody owns the joins. You do.
1 The Hook
A Crown entity runs Oracle Fusion Financials and Salesforce CRM. Sales orders created in Salesforce should automatically generate customer invoices in Oracle Fusion via a MuleSoft integration layer. Integration testing confirms orders flow correctly. Go-live happens.
Three weeks later: 140 orders sit in Salesforce as “Invoiced” but are absent entirely from Oracle Financials. Revenue is not recognised. The AR team cannot see the receivables. Investigation takes 4 days. The cause: the integration handles standard orders correctly but fails silently for orders with the “Government Pricing” flag enabled — a Salesforce-specific field with no mapping in the MuleSoft transformation layer.
Neither Oracle nor Salesforce owns MuleSoft. The SI configured it and left. The integration test matrix covered the happy path. It did not cover the “Government Pricing” flag because no tester asked: “What flags exist in Salesforce that the middleware might not know about?”
2 The Rule
In a multi-vendor ERP programme, test every integration with every flag, status, and exception that each source system can produce. The happy path is rarely where the bugs live. Silent failures are the worst failures — design for visibility.
3 The Analogy
Multi-vendor ERP integration is like a translation chain — English to French to Spanish.
Each translator is competent. The English-to-French translator handles standard sentences perfectly. But if the French translator encounters a word that has no Spanish equivalent, the message is silently dropped. No error. No warning. The Spanish reader never receives the sentence and never knows they were supposed to.
Your job as the integration tester is to find the untranslatable words before they reach production. That means mapping every field in the source system and asking: “What happens in the middleware when this field has this value?”
4 Watch Me Do It
An integration test matrix for an Oracle Fusion + Salesforce + MuleSoft order-to-invoice flow. Each row is a distinct test scenario.
| Scenario | Salesforce order type | Expected in Oracle Fusion | Silent failure risk |
|---|---|---|---|
| Happy path: standard NZD order | Standard order, NZD, no flags | Customer invoice created, correct amount | Low — this is what integration testing covers first |
| Government pricing order | Standard order + “Government Pricing” flag | Customer invoice created, discounted rate applied | HIGH — unmapped field silently drops or creates at wrong price |
| Cancelled order | Order status = “Cancelled” after invoice exists | Void or credit memo in Oracle Financials | HIGH — cancellation may not propagate; invoice remains outstanding |
| Part-fulfilled order | Order partially shipped (60% delivered) | Partial invoice for delivered quantity only | Medium — middleware may send full order amount regardless |
| New customer (not in Oracle) | Order for customer ID not yet in Oracle | Error surfaced and routed to AR team for resolution | HIGH — silent failure likely; order lost with no notification |
What correct error handling looks like: when an order cannot be processed (missing customer, unmapped field, transformation failure), the middleware raises an error that is visible to the integration team — logged, alerted, and routable to someone who can fix it. What silent failure looks like: the order disappears into the middleware and nothing appears anywhere. Design tests that force the error condition and verify the error is visible.
5 When to Use It
Any project where two or more vendor systems exchange data through a middleware or integration layer. In NZ enterprise and government this typically means:
- Oracle Fusion + Salesforce via MuleSoft — the combination that produced the scenario above, and one of the most common in NZ Crown entity and insurance sector
- D365 Finance + a legacy payroll system — typically via Azure Integration Services or a SFTP file drop
- Salesforce CRM + a policy administration system — common in NZ insurance
- Any ERP + a NZ-specific regulatory reporting system — IRD, ACC, MSD integrations
The testing approach applies to any middleware: MuleSoft, Azure Integration Services, AWS EventBridge, Boomi, or a custom API gateway. The platform changes; the principle does not.
6 Common Mistakes
🚫 “Integration testing is the SI’s responsibility. I just verify the end result.”
Actually: the SI tests that their configuration works for the scenarios they tested. They do not know your business edge cases — the Government Pricing flag, the part-fulfilled order with a backdated shipping date, the customer who exists in Salesforce but was merged to a different account in Oracle. That knowledge lives with your business users and your test team. You must own the integration test matrix.
🚫 “If there’s no error message, the integration worked.”
Actually: silent failures produce no error messages. The integration worked for the standard path, and the non-standard record simply vanished. Always verify the target system shows the expected record — do not rely on the absence of an error as confirmation of success.
🚫 “We tested the integration in the development environment. Production will be the same.”
Actually: production data has variations that development test data does not. The Government Pricing flag example — it existed in production Salesforce but no development test record had it set. Always derive your integration test scenarios by auditing the actual production data for every combination of flags, statuses, and field values that exist. Do not rely on what the development team thought to create.
7 Now You Try
Paste or edit the prompt below, then click Run to get a real AI response.
8 Self-Check
Click each question to reveal the answer.
Q1: What is a “silent failure” in an integration context and why is it more dangerous than a visible error?
A silent failure is when a transaction or record fails to propagate through an integration layer without producing any error message, log entry, or alert. It is more dangerous than a visible error because a visible error tells you something went wrong and where. A silent failure means the source system thinks the transaction succeeded, the target system has no record of it, and nobody is notified. In the Oracle + Salesforce example, 140 orders showed “Invoiced” in Salesforce while Oracle had no corresponding invoice — and it took 3 weeks before a human noticed the discrepancy. Design your integration tests to force the failure condition and verify that something, anywhere, becomes visible when it occurs.
Q2: How do you build an integration test matrix when the SI has left the project?
Start with the data. Export the production source system data and catalogue every unique combination of flags, statuses, and field values that exist. Each unique combination is a potential integration scenario. Then map those against the middleware transformation logic (if you can access it) or against the API documentation. Scenarios where a source field has no documented target mapping are your highest-risk items. Supplement with business knowledge — ask the business users what unusual things they do with the system. They will tell you about the Government Pricing flag, the backdated orders, the merged customers. That is where the gaps are.
Q3: Who is accountable for integration defects in a multi-vendor ERP programme?
Contractually, it depends on the programme structure — typically the SI is responsible for the middleware they configured. But in practice, when a defect surfaces post-go-live, accountability is academic: the business is losing revenue, and the test team that signed off integration testing without covering the failing scenario will be asked why. The practical answer is: own the integration test scope regardless of contract. Define what you are testing, test it, and document what is explicitly out of scope and why. That documentation is your protection if a post-go-live defect falls outside your tested scope.
9 ISTQB Mapping
CTAL-TA v3.1.2 — Section 3.2.2: Integration testing — testing the interfaces between components, including the ETL and API integration layers between vendor systems in a multi-vendor architecture.
CTAL-TM v3.0 — Section 3.5: Risk-based testing — applying risk analysis to integration scenarios to identify which flag and status combinations carry the highest probability and impact of silent failure, and prioritising test coverage accordingly.