OIC - Common Integration Pattern Pitfalls and Design Best Practices

Common Integration Pattern Pitfalls and Design Best Practices:

Note the following best practices and integration pattern pitfalls to avoid when designing an integration.

• Avoid Common Integration Pattern Pitfalls
• Avoid Creating Too Many Scheduled Integrations
• Synchronous Integration Best Practices
• Design Long-Running or Time-Consuming Integrations as Asynchronous Flows
• Time Outs in Service Calls During Synchronous Invocations
• Parallel Processing in Outbound Integrations

Avoid Common Integration Pattern Pitfalls

• Chatty Integrations
• Scheduled Job that Never Stops Trying to Process
• lmport an Externally Updated IAR File
• Synchronous Integration Doing Too Much
• Too Many Connections in an Integration
• Read Files with Many Records'
• Integrations Running Unchanged Despite Changing Business Needs

1. Chatty Integrations:

Use case: 

Synchronise records in a file or large data set with an external system.

Pitfall:

Use an invoke activity within a looping constuct to call external APIs for every record.

Why pitfall:

Downstream apps are receiving a large number of atomic requests. This puts the entire system under duress.

Best practices:

• Leverage application capabilities to accept multiple records in a single request.
• Leverage adapter capabilities to send a large data set as attachements or files.
• Use a stage file action and use apend file option to send the file to the destination.

2. Scheduled job that never stops trying to process:

Use case:

Process records within a set of files with a tight SLA.

Pitfall:

The scheduled integration looks for all files to process and loops over all to sequentially process until no files remain.

Why pitfall:

If a large number of files exist, one run of a sheduled job executes for a long time and starves other jobs and may get terminated by the framework.

Best Practice:

• Limit the number of files to process in a single scheduled run.
• Use scheduled parameters to remember the last processed file foe the next run.
• Invoke the run now command to trigger processing of the next file if waiting for the next scheduled run is not feasible. 

3. Import an externally updated IAR file:

Use case:

Need to leverage advanced XSL constructs that may not be avaialble in the mapper.

Pitfall:

Updating the IAR file externally and then importing it into Oracle integration.

Why pitfall:

Activation failures may occure.

This can lead to metadata inconsistency and validation failures.

Best Practice:

Use import map feature in oracle integration.

4. Synchronous Integration doing too much:

Use case: 

A request triggers complex processing involving enrichment and updates across multiple systems.

Pitfall:

Huge synchronous integration modeling a large number of invokes/conditional logic.

Why pitfall:

Susceptible to timeouts.

Blocking call - holds resources and starves other integrationa.

Best Practice:

• Explore moving completely to an asynchronous integration - fire and forget , async response. Thus it will alao support  resubmission of failures.
• Optimise sync processing with a coarse grained external API to replace multiple chatty calls.
• Split into a sync integration containing mandatory processing before sending out a response and triggering separate asyn fire and flrget integrations for other processibg logic.

5. Too many connections in an integration

Use case:

As developers create integrations, they define their own connectiona pointing to the same application. This leads to many duplicate connections.

Pitfall:

Every developer creates their own connection using different set of configurarions/credentials.

Why pitfall:

High number of connections make manageability painful, specially when you need to update the endpoint, credentials etc. 

Best practice:

Have a custodian create needed connections and ensure duplicate connections of the same type are not created. 

Build a best practice for naming conventions and maintaining a set of configurarions.

6. Read files with many records

Use case:

Read a file with a large number of records and process individual records.

Pitfall:

Reading the entire file in memory using the read file option and processing record by record.

Why pitfall:

Consumes large amounts of momory and impacts other system procesing.

Best practice:

Download the file to the stage location using the download file option. 

Use the read file with segments options. Platform automatically processes segments in parallel. Platform brigs in only the portions of the file to the memory, as needed.

7. Integrarions running unchanged despite changing business needs

Use case:

Integrations/schedules created during the initial implementation continue to run even though your business requirements have changed over time.

Pitfall:

Integrations and scheduled jobs created during the initial product implementation are never re-evaluated against changing business needs.

Why pitfall:

Unnecessary runs of jobs that handle no work.

Clutter with dead integrations, life cycle management overheads and developer confusion.

Best practice:

Periodically analyze existing integrarions or schedules against current business needs.

Deactivate integrations that are no longer needed.

TBD.

Reference: Common Integration Pattern Pitfalls and Design Best Practices

django - webpage to input text and remove punctuation from it.

 urls.py:

from django.contrib import admin
from django.urls import path
from . import views
urlpatterns = [
    path('admin/', admin.site.urls),
    path('',views.index, name="index"),

    path('index',views.index,name="index"),
    path('removePunctuation',views.removePunc,name="removePunctuation")
]

views.py

def index(request):
#    return HttpResponse("Hello")
    return render(request,'index.html')

from django.http import HttpResponse
from django.shortcuts import render

def removePunc(request):
    text=request.GET.get('text','default')
    check=request.GET.get('removepunc','off')
    if check == "on":
        puncList= '''!()-[]{};:'"\,<>./?@#$%^&*_~'''
        newText=''
        for char in text:
            if char not in puncList:
                newText=newText + char

        return HttpResponse(newText)
    else:
        return HttpResponse(text)

index.html:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>template is working</title>
</head>
<body>
<h1>Welcome to text analyzer. please enter your text.</h1>
<form action="/removePunctuation" methlod="get">
    <textarea name="text" style="margin: 0px; width: 1307px; height: 111px;"></textarea><br/>
    <input type="checkbox" name="removepunc">Remove Punctuation<br/>
    <button type="submit">Analyze Text</button>
</form>
</body>
</html>

Web pages:

You can create another template and show the removed punctuation text there.

template:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Analyzing Your Text...</title>
</head>
<body>
<h1>Your Analyzed Text - {{ purpose }}</h1>
<p>
    {{ analyzed_text }}

</p>
</body>
</html>

params = {'purpose': 'Removed Punctuations', 'analyzed_text': newText}
        return render(request, 'analyze.html', params)

django - webpage to have personal navigations

 urls.py:

from django.contrib import admin
from django.urls import path
from . import views
urlpatterns = [
    path('admin/', admin.site.urls),
    path('',views.index, name="index"),
    path('navigation',views.navigationBar,name="navigationBar")
]

views.py:

from django.http import HttpResponse
from django.shortcuts import render

def navigationBar(request):
    return render(request, 'navigationUrls.html')

templates/navigationUrls.html:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Personal Navigation</title>
</head>
<body>
<h1>Personal Navigations:<br/></h1>
<a href="https://soalicious.blogspot.com/" target="_blank">My Blog Page</a><br/>
<a href="https://www.facebook.com/" target="_blank">Facebook Login Page</a><br/>
<a href="https://twitter.com/?lang=en" target="_blank">Twitter Page</a><br/>
<a href="https://www.instagram.com" target="_blank">Instagram Page</a>
</body>
</html>

WebPage:

django - homepage - fetch form textarea value from template to views

 template - index.html:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>template is working</title>
</head>
<body>
<h1>Welcome to text analyzer. please enter your text.</h1>
<form action="/textAnalyze" methlod="get">
    <textarea name="text" style="margin: 0px; width: 1307px; height: 111px;"></textarea>
    <button type="submit">Analyze Text</button>
</form>
</body>
</html>

urls.py:

from . import views
urlpatterns = [
    path('admin/', admin.site.urls),
    path('',views.index, name="index"),
    path('about/',views.about, name="about"),
    path('textAnalyze',views.textAnalyze, name="textAnalyze")
]

views.py:

from django.http import HttpResponse
from django.shortcuts import render

def index(request):
    params = {"name":"Sri","role":"develoepr"}
#    return HttpResponse("Hello")
    return render(request,'index.html',params)

def about(request):
    return HttpResponse("<h1>about</h1>")
def textAnalyze(request):
    ttext = request.GET.get('text','default')
    print(ttext)
    return HttpResponse(ttext)

web page:

suppose you entered "abcd" in the textarea, then it will go to the views.py and print the inputted data.

django - how to use templates

 Step1: Open settings.py and write templates in the DIR of template section as below:

Step2: Create a templates folder and create a index.html file in it as below:

Contents of index.html:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>template is working</title>
</head>
<body>
<h1>from index page, Hello</h1>
</body>
</html>

Step3: in the views.py, import the render module from shortcuts and call the template as below:

from django.http import HttpResponse
from django.shortcuts import render

def index(request):
#    return HttpResponse("Hello")
    return render(request,'index.html')

def about(request):
    return HttpResponse("<h1>about</h1>")

The server will run automatically:

You can also create a dictionary and send the same to the template while rendering from views.py and use them in the tempate.

views.py:

from django.http import HttpResponse
from django.shortcuts import render

def index(request):
    params = {"name":"Sri","role":"develoepr"}
#    return HttpResponse("Hello")
    return render(request,'index.html',params)

template - index.html:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>template is working</title>
</head>
<body>
<h1>from index page, Hello</h1>
{{name}}
</body>
</html>

web page:

OIC - Configre rest trigger to get CSV file

Here I will show you how to configure a rest trigger to binary mode to get csv file. The media type will be text/plain.

You can also get binary file as 

• Octet-stream
• Pdf
• Msword
• Zip
• Jpeg,png,bmp,gif
• Other media type like text/plain

Implementation Steps:

Step1: Create an app driven integration and choose verb as POST, Configure a request payload for this endpoint and /csvData as relative resource URI.

Step2: In the request section, select the request payload format as Binary and media type as Other media Type and media type as text/plain.

Step3: Drag and drop a stage action and read the file 

Choose stage file operarion as Read Entire File.

Configure the file reference as Yes

Specify the Filr reference: /nssrcmpr:execute/nsmpr2:streamReference

Step4: Provide a sample csv file, record and recordset names.

Test.csv

Name,Id

"Sri", 1

Step5: Take a logger action and select Log as always, logger message as readfile/readresponse/empRecordSet.

Step6: Add the tracking as streamReference and save and activate.

Step7: Once the integrarion activated, do the following:

• click on run and Test, 
• go to body and select File button and select your file and test. 
• Go to the monitoring section and open activity stream, you can see the logger contains the csv file data.

Test from Postman:

You can also test this rest from postman following the below steps:

Step1: From the integration take the endpoint uri and open in postman with post verb,

Step2:  Go to the authorization tab and select basic auth and provide OIC instance access credentials.

Step3: Go to the header tab and add Content-Type as text/plain.

Step4: Go to the Body tab and select binary and choose the test.csv file and hit the send button.

OIC - dynamic email send with html table content.

Here, i will explain the steps how we can send the dynamic email with html table content.

Steps:

Step1: create an app driven integration and drag and drop a rest trigger connection and configure the request json as below:

{

"EmployeeDetails":{

"Employee": [{

"Name": "asd",

"Id": "as123"

}]

}

}

Step2: take an assign and create a variable like emailContent with

"<table><tr><th>Name</th><th>Id</th></tr>"

Step3: take a for each action and map with fkr each employee

Repeating element: /nssrccmpr:execute/nsmpr3:request-wrapper/nsmpr3:EmployeeDetails/nsmpr3:Employee

Step4: in the for each action, in the emailContent variable add the follow:

concat($emailContent, "<tr><td>", Name_xpath,"</td><td>",Id_xpath,"</td></tr>")

Step5: outside of the loop, take assign and add the close table tag in the emailContent variable.

Concat($emailContent, "</table>">

Step6: take notification action and provide details for From, To and Subject. And then create a paramter as body and assign the variable $emailContent. And write the beloe code in the Body section.

<html>

<head>

You can place any style script here

</head>

<body>

<p>Below are the emp details:</p>

{body}

</body>

</html>

Step7: add the tracking, save and activate and test it.