Integration Technologies: Patterns, Tradeoffs, Controls

Review note: Rewritten September 4, 2026. Vendor products, editions, connectors, and support status change; confirm them in current first-party documentation.

Integration connects systems while preserving business meaning, security, and operational control. A broker, event log, API gateway, ETL/ELT engine, change-data-capture tool, federation engine, or integration platform addresses a different boundary. Product count is not architecture.

Start with the contract

Define producer, consumer, owner, purpose, source of authority, schema, identifiers, units, event and availability time, ordering, volume, latency, freshness, retention, privacy, delivery semantics, and change policy. State how corrections, duplicates, deletions, replay, and reconciliation work.

Match patterns to requirements

Pattern	Best suited to	Key controls
Synchronous API	Bounded request/response	Timeouts, idempotency, authentication, rate limits
Queue	Work distribution/load leveling	Acknowledgment, retry, poison handling, backlog
Event log/pub-sub	Independent consumers/replay	Ordering scope, retention, schema evolution, offsets
Batch ETL/ELT	Scheduled bulk movement	Atomic publish, manifests, control totals, reruns
CDC	Source change propagation	Snapshots, log position, DDL, deletes, ordering
Federated query	Read-time access without copying	Pushdown, source load, access, consistency, cost

Do not overstate guarantees

“Exactly once” is scoped to defined components and failure assumptions. External side effects can remain ambiguous after timeout. Use stable identifiers, idempotency keys, transactional outbox/inbox patterns where suitable, deduplication, bounded retries, status checks, and business reconciliation.

Secure every data flow

Use service/workload identities, least privilege, TLS, secret and key management, network policy, schema/payload validation, rate and size limits, protected logs, and data minimization. Separate administrative planes and test credential revocation. A provider certification does not make an integration secure.

Evaluate products with evidence

1. Freeze versions, connectors, configuration, and representative workloads.
2. Test throughput and latency distributions, ordering, duplicates, loss, backpressure, schema change, and recovery.
3. Exercise source/sink outage, partial batch, poison record, replay, failover, upgrade, and rollback.
4. Measure operator effort, observability, accessibility, support, portability, lifecycle cost, and exit.
5. Record the owner and re-evaluation triggers for product, contract, source, scale, or risk changes.

Build the architecture with enterprise application integration, monitor dependencies via data pipeline monitoring tools, and compare deployment choices using cloud architecture patterns.