OIC – Unified File Reprocessing Framework Using Scheduler Flag, Dual SFTP Sources, Child Integration, and Centralized Error Handling

📌 Use Case

Business processes often require file reprocessing and fresh processing from two different locations:

1. OIC SFTP → for reprocessing previously failed files
2. Vendor SFTP → for processing newly delivered source files

To manage both scenarios within a single automated pipeline, we use a Scheduler flag, a central Main Integration, and a dedicated Child Integration. A unified Global Fault Handler sends errors to CCS, logs them into Datadog, and moves failed files to an error directory.

This ensures stable, recoverable, and auditable file processing within OIC.

---

🛠️ Solution Steps

---

1️⃣ Scheduler Integration – Control the Processing Mode

The scheduler integration triggers the main service and passes a flag:

filePresentInOICSFTP = Y

• System should process files from OIC SFTP
• This is used for reprocessing failed files

filePresentInOICSFTP = N

• System should process files from Vendor SFTP
• This handles fresh daily load

This flag enables dynamic routing and avoids maintaining separate integrations for reprocessing.

---

2️⃣ Main Integration – File Discovery & Preparation

The main service behaves differently based on the flag.

---

✔️ A. If filePresentInOICSFTP = Y (Reprocessing Mode)

• List files from OIC SFTP
• For each file found:
  • Call the Child Integration
  • Pass file name + path + flag

No need to download or create new files—direct reprocessing.

---

✔️ B. If filePresentInOICSFTP = N (Fresh Run Mode)

• List files from Vendor SFTP
• For each file:
  • Download file from vendor SFTP
  • Write the file to OIC SFTP for internal processing
  • Call the Child Integration, passing the newly stored file name

This ensures all files—new or old—are processed uniformly through OIC SFTP.

---

3️⃣ Child Integration – Validation & File Handling

This integration performs the actual business logic.

---

✔️ Step A: Retrieve File Reference

Since files are stored in OIC SFTP in both modes:

• Use File Server Action → Read File
• Generate file reference for reading & parsing

---

✔️ Step B: Read File Content

• Read the contents of the file using file reference
• Parse the data based on flat file layout

---

✔️ Step C: Business Validation

Apply the core rule:

The required field must contain “00”.

If the condition fails:

• Use Throw New Fault
  • This forces execution of the Global Fault Handler
  • Ensures controlled, trackable exceptions

---

✔️ Step D: Delete Source File

After reading:

• Delete file from OIC SFTP

If file was originally from vendor SFTP:

• The vendor SFTP file is already cleaned earlier by the main service
• (or can be cleaned here based on your design)

This avoids duplicate processing.

---

4️⃣ Global Fault Handler – Centralized Error Management

When the child flow throws an exception:

---

✔️ A. Create Case in CCS (TO Module)

• Send error details and file metadata
• Trigger TO creation for functional follow-up

---

✔️ B. Log Errors to Datadog

Log structured fields:

• File name
• Validation failure message
• Timestamp
• Integration instance ID
• Severity

This ensures observability and alerting.

---

✔️ C. Move the File to Error Directory

• Take the original payload/file
• Move it to OIC SFTP Error Folder
• Prevents accidental deletion of faulty files
• Allows manual reprocessing if needed

---

✔️ Final Outcome

This end-to-end design enables:

• Single unified pipeline for fresh processing + reprocessing
• Smart routing based on Scheduler flag
• Guaranteed file availability in OIC SFTP for consistent logic
• Strong validation using controlled Throw New Fault
• Automatic integration with CCS for error cases
• Observability via Datadog
• Robust error-handling and file preservation
• Minimal manual intervention

A complete, enterprise-grade framework for file ingestion & reprocessing in OIC.