OIC gen3 - Integration patterns and Define schedules

Integration patterns:

1. Application : Real time call or data feed from the subscriber.
2. Schedule: Runs at specific dates and times defined in a schedule. Mainly used for bulk or batch integration or file processing.
3. Event: starts when an event is published. It is like EDN or even driven Architecture or publish subscribe model.

Creating a schedule: 

There are two ways, we can create or define a schedule of a schuduled integration.

1. On the design canvas , click the ellipsis on schedule node and edit schedule definition.
2. Add a schedule on the integrations page before or after the integration has been activated.

Schedule types:

1. Simple schedule

• We can define only once - hour minutes, days months, weeks
• We can also define recurring schedule - hourly, minutely, daily, monthly, weekly
• Frequency can not be less then 10 mins.
• We can set specific timezone, add a time window to run or never expire mode by default.

1. Advanced schedule or iCal expression

• We use calender expression 
• Frequncy can be set minutely(<10 mins).
• We can also define multiple schedule frequencies together using &.

Examples:

Example 1: This example runs on the 1st, 10th and 15th days of the month at 5:15am, 10:15am , 3:15pm and 8:15 pm.

FREQ=MONTHLY;

BYMONTHDAY=1,10,15;

BYHOUR=5,10,15,20;BYMINUTE=15;

Example 2: multiple schedule frequencies together: The example provided runs every day between 5:30 PM and 7:30 PM, executing every 10 minutes during these hours.

FREQ=DAILY;BYHOUR=17;BYMINUTE=30,40,50;BYSECOND=0;

&FREQ=DAILY;BYHOUR=18;BYMINUTE=0,10,20,30,40,50;BYSECOND=0;

&FREQ=DAILY;BYHOUR=19;BYMINUTE=0,10,20,30;BYSECOND=0;

Note: we can only start the schedule if the integration is activated. Once started, we can see the future runs of the schedule. 

OIC Gen3 - About Tracing Levels while activating integration

OIC has following 3 tracing levels:

1. Production
2. Audit
3. Debug

Production:

• Data is retained for 32 days.
• Log all flow actions
• Log only invoke /logger actions.
• Wihin loops( upto 1000 iterations)

Audit:

• Data is retained for 8 days.
• Logs same as production option
• Also logs external payloads

Debug:

• Data is retained only for 24 hrs.
• Logs same as audit option
• Also logs all actions within the loops.

Can we modify the tracing dynamically?

Yes, We can also dynamically or runtime modify the tracing level without deactivating the integration. 

Steps:

Integration >> "..." actions >> configure activation >> modify the tracing level.

What occurs or observe if we deactivate any integration:

• Activated integrations cannot be edited.
• Stops processing any new messages.
• Pending requests remain unprocessed and are lost.
• Existing history, monitoring, and runtime data are lost.
• In-flight instances will fail, which can be observed.
• You must stop the schedule before deactivation.

OIC Gen3 - Integration Lifecycle

 

OIC Gen3 - Working with Projects | Packages vs Projects

Differences between Packages and Projects:

Area	packages	projects
Overall Usage & Goals	Packages provide for organization of integrations to allow for import and export of related resources.	In addition to resource organization, projects provide for improved release management and a single unified workspace.
Access Control	Resources within packages are visible to all OIC users the same as all other global resources.	Projects provide for fine-grained access control.
Deployment	Creating a CI/CD pipeline to deliver updated integrations to other environments requires work outside of OIC	Projects provide built-in deployment capabilities with release management and controlled deployment capabilities.
Observability	Packages do not provide separate monitoring capabilities.	Projects provide internal Observe pages for monitoring activated project integrations.

Convert a package to project:

You heard it right. We can convert a package to project. Accelerator and recipe packages are not supports for conversion.

Projects notes to remember:

1. A project allows to view and work on  integrations, connections, lookups and libraries in one page. 
2. A project has 3 sections - Design, Deploy and Observe.
3. There is a "share" option from which we can give access to others to this project. By default a project and its resources is private and only the owner can access.
4. From design >> we can design our integrations, create lookups, connections etc. And run or test the integrations.
5. From observe >> we can view the integrations, instances, future runs and audit info
6. From Deploy >> we can create a deployment of the project, we can select the integrations of the project to be part of the deployment. After creating the deployment, we can export it and use or import it to higher instance.
7. For packages, export creates a .par file and for projects, export creates a .car file.

Oic Gen 3 links

OIC generation 3 links:

1. Oracle Integration Generation 3 New Features
2. Integration patterns and how to define schedules
   
3. About RBAC - Resource based access control
4. How to publish and subscribe events | What is oracle integration events | what is publish event action
5. Read in segment - set your own chunk or sement size
6. About different Actions available in OIC Generation 3
   
7. Parallel action
8. Stage file action
   
9. File Server Action
   
10. File adapter vs FTP adapter
    
11. FTP does not support polling as trigger then what is the alternate solution to achieve FTP Polling?
    
12. Max file size supported by Rest, soap, file and ftp and database adapters | Service limit
    
13. Data Stitch action
    
14. OIC Error Handling
15. OIC gen3 - Working with XSLT constructors
16. Working with Projects | Packages vs Projects
17. About Tracing levels while activating integration
    
18. TBD

OIC Gen3 - Stage file action

Stage or vfs or virtual file system is a temporary location in the oic local file system which stores temporary files required for processing in an oic instance. It provides a file reference to the file which we can use to access the data or file content.

Stage file action operations:

1. List Files
2. Unzip file
3. Zip file
4. Decrypt file
5. Encrypt file
6. Write file
7. Read entire file
8. Read file in segments

Points to remember:

1. To decrypt use private and to encrypt use public PGP keys. First we need to upload pgp keys in oic certificates sction, so that we can use them later in stage action.
2. To read or write file schema configuration options, EDI document is also avaiable along with csv, json, xsd, xml document (single or no namespace)
3. To write file with schema option,it supports max 10mb file size.
4. To write file reference or opaque file. We have to use a opaque xsd as sample file. And in the mapper, use encodeReferenceToBase64() function to convert from reference to base64 content.
5. OIC automatically handles the creation, deletion and cleanup of the temprary files in the vfs or local file system. When the instance gets completed, the files in vfs will be flashed out.
6.  To "read file in segments", it supports min segment size: 200 and max segment size: 2000.
7. To "read file in segements", it does not support json sample for schema options.
8. To "read file in segments", by deafult, the rows are processing in parallel. To process in sequentially, select the check box for sequentialprocessing.
9. To "read file in segments", remove trailer only supports for csv file. We can do remove "last n rows"  or "last row" of the file.
10. In stage, it can store file size max upto 1 gb file.
11. Csv, json, or xml format file or data what ever we are trying to write or read, internally it is translated to xml structure, thats why if we open mapper, will see it as xml in both source or target points.

OIC Gen3 - File Server action

FTP adapter vs File server action:

• Using file server action, we can access to OIC embeded file server, Though using ftp adapter we can also connect to OIC embeded file server. 
• Where as Ftp adapter use sFTP protocol and user credential to connect to oic file server. For file server action, it uses internal apis http/TLS and no user credential required.
• If File server is enabled from OCI console, then only file server action will be displayed in the action pallet.

When to use FTP adapter:

• Read a file into the vfs for further processing.
• Encrypt or decrypt a file
• Sign or verify a file.

file service action supports following Five operarions:

1. List Directory
2. Get file reference
3. Write file
4. Move file
5. Delete file

Notes: 

• For list directory, max files supports 1000 only.
• Using file server action, if we write, move or delete files, in the response, success field boolean value will indicate where the operation completed successfully or not.