Common Integration Pattern Pitfalls and Design Best Practices:
Note the following best practices and integration pattern pitfalls to avoid when designing an integration.
- Avoid Common Integration Pattern Pitfalls
- Avoid Creating Too Many Scheduled Integrations
- Synchronous Integration Best Practices
- Design Long-Running or Time-Consuming Integrations as Asynchronous Flows
- Time Outs in Service Calls During Synchronous Invocations
- Parallel Processing in Outbound Integrations
Avoid Common Integration Pattern Pitfalls
- Chatty Integrations
- Scheduled Job that Never Stops Trying to Process
- lmport an Externally Updated IAR File
- Synchronous Integration Doing Too Much
- Too Many Connections in an Integration
- Read Files with Many Records'
- Integrations Running Unchanged Despite Changing Business Needs
1. Chatty Integrations:
Use case:
Synchronise records in a file or large data set with an external system.
Pitfall:
Use an invoke activity within a looping constuct to call external APIs for every record.
Why pitfall:
Downstream apps are receiving a large number of atomic requests. This puts the entire system under duress.
Best practices:
- Leverage application capabilities to accept multiple records in a single request.
- Leverage adapter capabilities to send a large data set as attachements or files.
- Use a stage file action and use apend file option to send the file to the destination.
2. Scheduled job that never stops trying to process:
Use case:
Process records within a set of files with a tight SLA.
Pitfall:
The scheduled integration looks for all files to process and loops over all to sequentially process until no files remain.
Why pitfall:
If a large number of files exist, one run of a sheduled job executes for a long time and starves other jobs and may get terminated by the framework.
Best Practice:
- Limit the number of files to process in a single scheduled run.
- Use scheduled parameters to remember the last processed file foe the next run.
- Invoke the run now command to trigger processing of the next file if waiting for the next scheduled run is not feasible.
3. Import an externally updated IAR file:
Use case:
Need to leverage advanced XSL constructs that may not be avaialble in the mapper.
Pitfall:
Updating the IAR file externally and then importing it into Oracle integration.
Why pitfall:
Activation failures may occure.
This can lead to metadata inconsistency and validation failures.
Best Practice:
Use import map feature in oracle integration.
4. Synchronous Integration doing too much:
Use case:
A request triggers complex processing involving enrichment and updates across multiple systems.
Pitfall:
Huge synchronous integration modeling a large number of invokes/conditional logic.
Why pitfall:
Susceptible to timeouts.
Blocking call - holds resources and starves other integrationa.
Best Practice:
- Explore moving completely to an asynchronous integration - fire and forget , async response. Thus it will alao support resubmission of failures.
- Optimise sync processing with a coarse grained external API to replace multiple chatty calls.
- Split into a sync integration containing mandatory processing before sending out a response and triggering separate asyn fire and flrget integrations for other processibg logic.
5. Too many connections in an integration
Use case:
As developers create integrations, they define their own connectiona pointing to the same application. This leads to many duplicate connections.
Pitfall:
Every developer creates their own connection using different set of configurarions/credentials.
Why pitfall:
High number of connections make manageability painful, specially when you need to update the endpoint, credentials etc.
Best practice:
Have a custodian create needed connections and ensure duplicate connections of the same type are not created.
Build a best practice for naming conventions and maintaining a set of configurarions.
6. Read files with many records
Use case:
Read a file with a large number of records and process individual records.
Pitfall:
Reading the entire file in memory using the read file option and processing record by record.
Why pitfall:
Consumes large amounts of momory and impacts other system procesing.
Best practice:
Download the file to the stage location using the download file option.
Use the read file with segments options. Platform automatically processes segments in parallel. Platform brigs in only the portions of the file to the memory, as needed.
7. Integrarions running unchanged despite changing business needs
Use case:
Integrations/schedules created during the initial implementation continue to run even though your business requirements have changed over time.
Pitfall:
Integrations and scheduled jobs created during the initial product implementation are never re-evaluated against changing business needs.
Why pitfall:
Unnecessary runs of jobs that handle no work.
Clutter with dead integrations, life cycle management overheads and developer confusion.
Best practice:
Periodically analyze existing integrarions or schedules against current business needs.
Deactivate integrations that are no longer needed.
TBD.
Reference: Common Integration Pattern Pitfalls and Design Best Practices
