FTP Get

Allows to recover one or more files via the FTP or SFTP protocol. For example you want to daily import products in Product-Live. So you daily send a csv file of your products to a sFTP and you need to import them to Product-Live.

Json

Example

Mode = PARAMETERS

{
  "name": "protocol-ftp-get",
  "taskReferenceName": "ftp_get",
  "description": "The business description of the task",
  "type": "SUB_WORKFLOW",
  "optional": false,
  "inputParameters": {
    "connection": "SFTP",
    "host": "",
    "username": "",
    "password": "",
    "port": 21,
    "mode": "PARAMETERS",
    "remoteFolder": "/",
    "sort": "LAST_MODIFIED_DESC",
    "filter": "-products\\.xml$",
    "maxFiles": 1
  }
}

Mode = REQUEST

{
  "name": "protocol-ftp-get",
  "taskReferenceName": "ftp_get",
  "description": "The business description of the task",
  "type": "SUB_WORKFLOW",
  "optional": false,
  "inputParameters": {
    "connection": "SFTP",
    "host": "",
    "username": "",
    "password": "",
    "port": 21,
    "mode": "REQUEST",
    "request": "${previous_taskReferenceName.output.file}"
  }
}

Definition

Property	Type	Required	Description
			The name (or type) of the task
			The unique name of the task in your job. It is used to reference this task in the workflow.
			The functional description of this task in your job.
			The type of the task. It must be SUB_WORKFLOW.
			true: the job continues if there is an error on this task. false: the job fails.
			Input parameters of the task. See below

Inputs

Property	Description
connection	The protocol. Must be FTP or SFTP or FTPS.
authenticationMethod	PASSWORD, PUBLIC_KEY Default: PASSWORD If connection is not SFTP and authenticationMethod is PUBLIC_KEY, the task fails.
host	You can use a dns or an IP address.
privateKey	The private key It is required to use a secret variable to store the private key.
passphrase	The private key passphrase Only taken into account if authenticationMethod=PUBLIC_KEY.
username	Username
password	Password If connection is no SFTP, if connection is SFTP and password is set then an invalid input error is added
port	Port
mode	Defines how files must be recovered. You must select between PARAMETERS (get one or multiple files with the parameters defined in the json below) or REQUEST (programmatically define which file must be recovered).
remoteFolder	The complete path to the folder. If you want to search in the root folder, just set /
sort	Defines how files must be ordered when they are read. You can choose between LAST_MODIFIED_ASC
filter	optional if mode = PARAMETERS The regex allows you to filter to do something like: "retrieve the last modified file which begins with the keyword ...", it is mandatory when you have several types of files in the same directory. If the value is empty, in this case we consider that there is no regex.
maxFiles	The number of files to recover. It is an integer greater than or equal to 1.
request	A valid input. A file that defines which files must be recovered. See the definition below.

TIP

The supported private keys are the following :

• ---- BEGIN SSH2 PUBLIC KEY ----
• -----BEGIN RSA PRIVATE KEY-----
• -----BEGIN DSA PRIVATE KEY-----
• -----BEGIN EC PRIVATE KEY-----
• PuTTY-User-Key-File-2: (ssh-rsa)
• PuTTY-User-Key-File-2: (ssh-dss)
• -----BEGIN OPENSSH PRIVATE KEY-----

Input request

Example

<Ftp-Get>
    <File>
        <Path>/products-1.csv</Path>
    </File>
    <File>
        <Path>/sub-folder/products-2.csv</Path>
    </File>
</Ftp-Get>

Definition

Usually you need this after listing all files on sFTP with the FTP List task, and after having done the filtering of the file that must be retrieved in an Transform XSLT task. Use this method only if the other methods does not fit with your needs.

| XPath | Description | Occurs | | :--| :-- | | Ftp-Get | Root of the document | 1 | | ./File | For each file. | 0..* | | ./File/Path | The exact file path that must be retrieved. Must starts with / | 1 |

Outputs

Output

Property	Type	Description
file		The first file recovered.
files		All files recovered.
listing		An XML file containing the url of the generated temporary XML file.
noFile		YES | NO. YES: no files have been found. NO: at least one file has been found.
allFilesRecovered		YES | NO. YES: all expected files have been recovered. NO: at least one file has not been recovered.

Listing

Example

<Files>
  <File>
    <Url>https://app.product-live.com/files-data-factory listing /d05a74cf11788d8f3ae9bf0e0e028dde66f0c83005c5e0d1211b0069945c0c11</Url>
    <Path>/products-1.csv</Path>
    <Last-Modified>2020-04-10T13:40:23.83Z</Last-Modified>
    <Size>1954387</Size>
  </File>
  <File>
    <Url>https://app.product-live.com/files-data-factory/fb26911d77fe9a9dc44b111eef5b5db7ca2019c8038445662f29b20c54cb6f29</Url>
    <Path>/sub-folder/products-2.csv</Path>
    <Last-Modified>2020-04-10T13:40:23.83Z</Last-Modified>
    <Size>1954387</Size>
  </File>
</Files>

Definition

XPath	Description	Occurs
Files	Root	1
File		A file
Url	Full, private URL of the file	1
Path	The full path of the file on the remote server. Subfolders are separated by the / character Start with a /	1
Last-Modified`	The date of the last modification of the file, format is UTC	1
Size	The size of the file in Kb	1

Limits and additional notes

• The limitation of the input file size is 250 Mb.
• 100 files in the same folder (can be less depending on file sizes).

In addition to these limitations, the following practices are recommended:

• Always limit the number of files on a FTP folder
• Always create one folder for one type of file eg. never have a folder and a file at the same level of path.This way you can always use the Last file method to retrieve your file on the server.