Processing CSV Files with a Header and Unbounded Detail Records in Oracle Integration Cloud (OIC)
Use Case
Organizations often receive CSV files from external sources, such as banks, vendors, or internal systems, that contain structured data. These files usually have a single header record followed by unbounded detail records (i.e., a dynamic number of transaction rows). The challenge is to efficiently ingest, parse, and process these files in Oracle Integration Cloud (OIC) before integrating them with Oracle ERP, HCM, or other enterprise applications.
For example, a company receives a daily transaction report in CSV format, and the data needs to be validated and loaded into Oracle Fusion ERP for reconciliation.
Solution Approach in OIC
Step 1: Receive the CSV File
- The file is received via SFTP, Object Storage, REST API, or a manual upload.
- OIC’s Stage File action is used to read the CSV file.
Step 2: Define the NXSD Schema for Parsing
- Create an NXSD schema in OIC to define the structure of the CSV file.
- The schema consists of:
- A header record (e.g., file metadata such as date, batch number).
- Unbounded detail records (i.e., transactions with dynamic row count).
Step 3: Parse and Extract Data
- Use Stage File → Read File action to parse the CSV file based on the NXSD definition.
- The header is extracted and stored separately for reference.
- The detail records are processed in a loop, allowing for dynamic handling of unbounded records.
Step 4: Transform Data for Downstream Systems
- The extracted detail records are mapped to the required format (e.g., XML, JSON, or another CSV structure).
- Validation rules are applied to filter and clean data before sending it to Oracle Cloud applications.
Step 5: Send Data to the Target System
- The transformed data is sent to Oracle ERP, HCM, or other applications via REST, SOAP, or database integration.
- The processed CSV file can also be archived for future reference in Object Storage or SFTP.
Nxsd code: take help of Jdeveloper tool to create the NXSD.
<?xml version="1.0" encoding="UTF-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:nxsd="http://xmlns.oracle.com/pcbpel/nxsd" xmlns:tns="http://BOCCDM.com/payments" targetNamespace="http://BOCCDM.com/payments" elementFormDefault="qualified" attributeFormDefault="unqualified" nxsd:version="NXSD" nxsd:stream="chars" nxsd:encoding="US-ASCII" >
<xsd:element name="Payments">
<xsd:complexType>
<xsd:sequence maxOccurs="unbounded">
<xsd:element name="Payment">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="Header" minOccurs="1" maxOccurs="1">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="C1" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C2" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="${eol}" nxsd:quotedBy=""" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="Detail" minOccurs="1" maxOccurs="unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="C1" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C2" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C3" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C4" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C5" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C6" type="xsd:decimal" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C7" type="xsd:decimal" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C8" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C9" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C10" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C11" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C12" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," nxsd:quotedBy=""" />
<xsd:element name="C13" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="${eol}" nxsd:quotedBy=""" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>