The first integration is usually simple: send a website lead to the CRM. The second adds an email platform. The third connects accounting. A year later, nobody knows which system owns customer status, why totals disagree, or what will break when a field changes.
This is automation sprawl: a collection of useful connections that no longer behaves like a coherent system.
Why point-to-point connections become expensive
A direct connection is not automatically wrong. For one stable source and one destination, it may be the smallest sensible design. The problem begins when business logic is copied into several connectors.
For example, the definition of a “qualified customer” might exist in a form workflow, the CRM, an email automation, and a reporting query. Updating the rule now requires coordinated changes in four places. If one update is missed, the business operates with conflicting truth.
The visible cost is maintenance. The less visible cost is hesitation: teams stop improving a process because nobody understands the consequences of changing it.
Map the integration before choosing tools
Create an integration inventory with one row per data flow. Record:
- source and destination;
- direction of the flow;
- trigger and expected frequency;
- system of record for each field;
- authentication method;
- expected volume and peak load;
- acceptable delay;
- retry behavior;
- owner and escalation route;
- data classification and retention requirement.
Microsoft’s Azure Architecture Center recommends considering each integration point independently because direction, volume, identity, networking, and compliance requirements can differ even between the same systems. Its guidance on integration and data-access approaches also warns against unnecessary tight coupling.
This map reveals where a simple request is enough and where the workflow needs queues, events, batches, or orchestration.
Choose the communication pattern deliberately
Use a synchronous API call when the caller needs an immediate answer and the dependency is expected to be available. Use asynchronous messaging when work can happen later, traffic arrives in bursts, or one system should continue operating while another is unavailable.
Use events when several consumers need to react to the same business change. “OrderConfirmed” is more reusable than embedding three separate calls inside the checkout process.
Use scheduled batches for high-volume reconciliation or when the source cannot produce real-time events. Real time is not inherently better if the business only makes the decision once each morning.
Put contracts between systems
An integration contract should define required fields, types, identifiers, status values, versioning, and error responses. Without a contract, every connector guesses.
Prefer stable internal identifiers over names or email addresses. Normalize dates, currencies, country codes, and enumerated values at a clear boundary. Version breaking changes rather than silently changing a payload used by several consumers.
An API gateway can centralize concerns such as authentication, rate limits, routing, and observability. AWS documents hostname, path, and header approaches in its API routing patterns. The right choice matters less than having a consistent entry point and ownership model.
Design failure as part of the normal flow
Every external system will eventually time out, reject a record, change a limit, or return incomplete data. “Retry three times” is not a complete failure strategy.
For each integration, decide:
- which errors are safe to retry;
- how duplicate processing is prevented;
- where failed messages are stored;
- who receives an alert;
- how a person can replay or correct the item;
- how the source and destination are reconciled later.
Idempotency is essential. Replaying an order event should not create a second invoice. Replaying a lead should update or safely ignore the existing record instead of creating another contact.
Make the system observable
A successful HTTP response does not prove the business outcome happened. Monitor both technical delivery and business reconciliation.
Useful signals include request volume, latency, error rate, queue age, retry count, duplicate prevention, schema failures, and the difference between source and destination totals. Attach a correlation identifier to each transaction so a team can trace it across services.
A maintainable integration roadmap
Begin by identifying the system of record for customer, order, inventory, and financial data. Move shared business rules into one owned service or workflow. Add contracts and logging before replacing connectors. Then migrate one data flow at a time, running reconciliation beside the old path until results agree.
A well-designed custom software platform does not eliminate integrations. It gives them structure: clear boundaries, controlled access, explicit failure paths, and enough visibility to change the business without fearing the wiring underneath it.