How to handle more than 10 mb file or large file poll???
If we use ftp or file adapter to poll files from a directory, it can only allow to poll or read file content max 10MB file.
So in order to read large files(>10 MB), we’ll have to download the file to a Virtual file system or a stage location as it supports 1 GB data poll but if you want to read the content, again it supports 10MB , so using file read in segment , best option to read contents in segments or smaller chunks and then marge it to create full target specific file.
Basically, we need to follow the following sequences:
- First list the files using ftp.
- Then for each file
- Download each file to stage location
- Using stage activity, read file in segments. In esch segment it support 200 records to process.
- Create the required file format and marge each segment to create full file.
No comments:
Post a Comment