12c OSB - split large xml files

Requirement:

In current process, Source application sends New hire and change employee data into a apps DB. OSB service pulls the large payload (>2K employee elements) using apps adapter and sends the data as single instance to SOA service which again push the data in a JMS queue from where soa service pulls the data and sends to the EDN as events to a default JMS queue from where all the subscribing event composites processing the data.

Cons: This huge payload is processing as a single instance payload through soa service and this is taking huge time to process and the instance is also not loaded/not opening.

Solution:

Now we are splitting the large payload into smaller chunks of batches (each batch size =500 employee elements) and write them in a directory using JCA based file adapter. Then using another proxy service we are reading all the files in 10 mins interval and then sends to SOA service.

Pros: This has resolved “the instance in not loading/opening issue”. Now all instances are getting loaded/opened with success.   

Pipeline flow :

Activity wise Split logic explained:

1.Assign: Take a batch Load

Value: number('500')

Variable: batchLoad

2.Assign: Count total employees

Value: fn:count($body/emp:WF_EVENT_T/EVENT_DATA/hrms_event_data/employee_data/employee)

Variable: varTotalEmpCounts

3.Assign : Taking the number of batches or iteration

fn:ceiling(xs:int($varTotalEmpCounts) div number($batchLoad))

varIterationCount

4.Assign: Taking the body payload into a tempBody variable

5.Delete: Emptying the employee elements from the tempBody. This way it is keeping the header and other elements.Only employee elements are deleted.

6.Assign: Now assigning the emptied employee elements into 2nd tempBody2 variable.

7.Assign: Taking an initial batch count

8.For Each: For each employee we are now adding the the employee into the tempBody variable until the batch size(500) meets.

9.Insert: employees into the tempBody variable

10. IF: Below is the condition to stop the for each loop once the batch size meets:
$varIndex = (number($batchLoad) * number($batchCount)) or

$varIndex = $varTotalLoopCount

11.Publish: Publish the batch in a file directory.

12.Replace:

13.Assign: then again assign tempbody2 payload into the tempBody variable

14.Assign: Then increasing the batch count

File Write JCA:

<adapter-config name="fileWrite" adapter="file" wsdlLocation="../WSDL/fileWrite.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">

  <connection-factory location="eis/FileAdapter"/>

  <endpoint-interaction portType="Write_ptt" operation="Write">

    <interaction-spec className="oracle.tip.adapter.file.outbound.FileInteractionSpec">

      <property name="PhysicalDirectory" value="/soashare/abc/fileIn"/>    

      <property name="Append" value="false"/>

      <property name="FileNamingConvention" value="Employee%yyMMddHHmmssSS%.xml"/>

      <!--<property name="SingleThreadModel" value="true"/>-->

    </interaction-spec>

  </endpoint-interaction>

</adapter-config>

File Read JCA:

<adapter-config name="fileReadService" adapter="file" wsdlLocation="../WSDL/fileReadService.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">

  <connection-factory UIincludeWildcard="*.xml" location="eis/FileAdapter"/>

  <endpoint-activation portType="Read_ptt" operation="Read">

    <activation-spec className="oracle.tip.adapter.file.inbound.FileActivationSpec">

      <property name="PhysicalDirectory" value="/soashare/abc/fileIn"/>

      <property name="PhysicalArchiveDirectory" value="/soashare/abc/archive"/>

      <property name="UseHeaders" value="false"/>

      <property name="MinimumAge" value="0"/>

      <property name="Recursive" value="false"/>

      <property name="PollingFrequency" value="600"/>

      <property name="MaxRaiseSize" value="1"/>

      <property name="SingleThreadModel" value="true"/>

      <property name="DeleteFile" value="true"/>

      <property name="IncludeFiles" value=".*\.xml"/>

    </activation-spec>

  </endpoint-activation>

</adapter-config>

Function of <property name="SingleThreadModel" value="true"/>
You want to configure File Adapters that it should pick one file at time from the given location with given polling interval.

Solution :
You set the "SingleThreadModel" and "MaxRaiseSize" properties for your file adapter.
Edit the adapter's jca file and add the following properties:
property name="SingleThreadModel" value="true"
property name="MaxRaiseSize" value="1"