IICS: File Processor Connector - Transferring Files between a Local File System and an FTP or SFTP Server

The file processor connector is used to securely transfer files from one location to another, regardless of size and location.

We can use File processor connectors for file processing operations, such as transferring, archiving, unarchiving, encryption, decrypting, compressing, decompressing, moving or copying files. 

The File Processor Connector is not a native connector, to add a file connector go to Administrator Add-on-Connection  and search for "File Processor Connector" and click start a free trial for 30 days trial 

Confirm the license  menu 

We must create a File Processor connection to create tasks to process files. 

Go to Administrator->Connection to create File Processor Connection use Type: FileProcessor

You can configure a connection on the Connections page or in a wizard as you configure a task.

On the 
Connections
 page, click 
New Connection
 to create a connection.
The 
New Connection
 page appears.
The following image shows the New Connection page details:

1. 
Configure the following connection properties on the New Connection page:
Connection Property	Description
Runtime Environment	The name of the runtime environment where you want to run the tasks.
Source File Directory	The location that contains files you want to transfer.
Target File Directory	The location where you want to place the transferred files.
Select File	The files that you want to transfer. You can select files based on the fields.
File Pattern	The pattern of the files that you want to transfer. For Example, if you want to select a file based on a date pattern, you can specify the date format as DD/MM/YYYY in the file pattern field. Note: The file Pattern field is not applicable when you select  all  from the  Select File  connection property.
Days Calculation	Use days calculation to select files that are created or modified before the specified date or after the specified date. Select files based on Contains Date Pattern, and specify the  day's calculation  value so that you can select files that are modified before or after the specified date. Specify the value in terms of days. You cannot specify the value in terms of month and year. For example, if you select a file based on Contains Date Pattern, use the data filters to specify LastModDate as 02/02/2016 in DD/MM/YYYY format, and specify days calculation as -1. Files that are modified till 01/02/2016 are selected.
PassKey	The credentials to connect to FTP or SFTP server. For example, you can specify the password and passphrase of the FTP or SFTP server as passkey1 and passkey2 values.
Click 
Save
 to save the connection.
You should test the connection before you save the connection details. You can click 
Test Connection
 to verify if the connection is successful. 

Selection of Specific Files

When you perform a File Processor operation, you can select a single file or multiple files. The files are selected based on the fields that you configure from the Select File connection property. The following table describes the Select File fields to select a specific file or set of specific files.

Fields	Description
All	Selects all the files from the source directory.
Equals	Selects the files that are equal to the specified name in the file pattern property. For Example, if you specify sample.docx, the sample.docx file is selected.
Notequals	Selects all the files other than the file pattern that you specify. For example, if you specify, the file pattern as sample.txt, the sample.txt file is not selected.
Startswith	Selects files that start with the pattern that you specify. For example, if you specify the StartsWith value as a sample, all the files that start with the name sample are selected.
Endswith	Selects files that end with the pattern that you specify. For example, if you specify the EndsWith pattern as .txt, all the files that end with .txt are selected.
FileExtension	Selects the files that contain the specified file name extension. For example, if you specify the values as .txt in the file pattern, all the .txt files are selected.
Contains	Selects files that contain the specified pattern. For example, if you specify the contains pattern as a sample, all the files that contain sample in the file name are selected.
FileSizeGreaterThan	Selects files greater than the size specified in the file pattern. Specify the file size in KB. For example, if you specify the file size as 30, all the files greater than 30 sizes are selected.
FileSizeLessThan	Selects files that are less than the specified size. Specify the file size in KB. For example, if you specify the file size as 30, all the files with less than 30 sizes are selected.
FileSizeEqualsto	Selects files equal to the specified size. Specify the file size in KB. For example, if you specify file size as 30, all the files with 30 sizes are selected.
FileSizeNotEqualsto	Selects files not equal to the specified size. Specify the file size in KB. For example, if you specify file size as 30, all the files with a size that is not equal to 30 sizes are selected.
Starts with DatePattern	Selects files that start with the specified date pattern. For Example, if you specify the file pattern as DD/MM/YYYY, all the files that start with DD/MM/YYYY date pattern are selected.
EndsWithDatePattern	Selects file that ends with a specified date pattern. If you specify the file pattern as DD/MM/YYYY, all the files that end with DD/MM/YYYY date pattern are selected.
Contains Date Pattern	Selects files that contain the specified date pattern. For example, if you specify the file pattern as DD/MM/YYYY, all the files that contain DD/MM/YYYY date pattern are selected.

Transferring Files between a Local File System and an FTP or SFTP Server

Use File Processor Connector to transfer files from a local system to an FTP or SFTP server or from an FTP or SFTP server to a local file system. You can use the FTP_PUT and SFTP_PUT objects to upload files from a local system to an FTP or SFTP server. Use the FTP_GET and SFTP_GET objects to download files from an FTP or SFTP location to a local file system.

You are a website administrator and want to upload files from a local file system to an SFTP server. In this example, you can specify the hostname, password, and UserID as data filters.

Perform the following steps to create a 

synchronization

 task to upload files from a local file system to an SFTP server:

In 
Data Integration
, click 
New
Tasks
.
Select 
Synchronization Tasks
, and click 
Create
 to create a synchronization task.
The 
Definition
 tab appears.
Configure the following fields on the 
Definition
 tab:
Field	Description
Task Name	Name of the  synchronization  task. For example File_Process_SFTP_PUT
Description	Description of the  synchronization  task. Maximum length is 255 characters.
Task Operation	Select Insert.
Click 
Next
.
The 
Source
 tab appears.
Configure the following fields on the 
Source
 tab:
Field	Description
Connection	Select the connection you created. For example File_Process_SFTP_PUT
Source Type	Select Single.
Source Object	Select SFTP_PUT.
Click 
Next
.
The 
Target
 tab appears.
Configure the following fields on the 
Target
 tab:
Field	Description
Connection	Select the flat file connection.
Target Object	Click Create Target.
The target file displays the status of the files that are transferred.
Click 
Next
.
The 
Data Filters
 tab appears.
Select the filter object, filter field, and filter operator to create a data filter on the 
Data Filters
 page.
The following image shows the 
Data Filters
 page:

When you perform an SFTP_GET, SFTP_PUT, FTP_GET, and FTP_PUT operation, you must specify values to the hostname, password, and userID filter fields to connect to the FTP or SFTP server.
Click 
Next
.
The 
Field Mapping
 tab appears.
Click 
Automatch
 on the 
Field Mapping
 tab to map source fields to target fields accordingly.
Click 
Validate Mapping
 to validate the mapping.
Click 
Save
 and then 
Finish
.
From the 
Explore
 page, select the task and click 
Actions
Run
.
In 
Monitor
, you can monitor the status of the logs after you run the task.

Author : Antosh Dyade

Share this

Related Posts