File generation / Generate Archive

Allows to generate an archive with a .zip extension. For example it can be used to generate an arhive of images or files generated by another task.

TIP

This task allows generating one or more archives.

Task name:

Task maximum duration: 120 minutes

Example

{
  "name": "file-generation-archive",
  "taskReferenceName": "generate_archive",
  "description": "The business description of the task",
  "type": "SUB_WORKFLOW",
  "optional": false,
  "inputParameters": {
    "request": "${previous_taskReferenceName.output.file}",
    "allowPartial": true
  }
}

Definition

Property	Type	Required	Description
			The name (or type) of the task
			The unique name of the task in your job. It is used to reference this task in the workflow.
			The functional description of this task in your job.
			The type of the task. It must be SUB_WORKFLOW.
			true: the job continues if there is an error on this task. false: the job fails.
			Input parameters of the task. See below

Inputs Parameters

TIP

If at least one task parameter (whether mandatory or not) is invalid, task execution is stopped and the returned status is FAILED. For example:

• The proposed value for the allowPartial property is invalid.
• The file specified for the request property does not exist

Property	Type	Required	Description
			See the complete definition below. It can reference the output of a previous task, example: ${previous_taskReferenceName.output.file} , or a static file, example: file://assets/file.xml, or a user input, example: ${workflow.input.input_key}.
			true: if there is an error adding a file to an archive, the archive is still generated although it is incomplete false: If there is an error adding a file, the archive is not generated, a log is added to the execution report and the task continues to run.

false: If there is an error adding a file, the archive is not generated, a log is added to the execution report and the task continues to run

Input request

Example

<Archives>
    <Archive>
        <File-Name>test-2-1.zip</File-Name>
        <Compression-Method>7z</Compression-Method>
        <Compression-Level>NORMAL</Compression-Level>
        <Password>********</Password>
        <Files>
            <File>
                <Path>/folder-1/image-1.jpg</Path>
                <Url>https://example.com/ee0c68fabf0a2b3c76b2643bff3cc215030f0caf.jpg</Url>
            </File>
            <File>
                <Path>/image-2.jpg</Path>
                <Url>https://example.com/7a597747dbc536e53eeb47760b5145d002b5bc25.jpg</Url>
            </File>
        </Files>
    </Archive>
</Archives>

Definition

You can use a static file stored in the asset folder, or you can generate this file from a Transform XSLT task.

XPath	Description	Occurs
Archives	Root	1
Archives/Archive	For each archive to be generated.	1..*
Archives/Archive/File-Name	The name of the generated archive. The name must include the extension.	1
Archives/Archive/Compression-Method	The type of compression to use. Possible values: zip, 7z, tar, gzip Default: zip	0..1
Archives/Archive/Compression-Level	The level of compression to use. Possible values: NONE, FASTEST, FAST, NORMAL, MAX, ULTRA Default value: NORMAL Compression tar only accepts NORMAL compression level	0..1
Archives/Archive/Password	The password to use for compression. Only available for 7z and zip compression	0..1
Archives/Archive/Files	The files contained in the archive	1
Archives/Archive/Files/File	For each file to put in the archive. Creating an empty archive is allowed.	0..*
Archives/Archive/Files/File/Path	The path of the file in the archive. Notes: We use here a representation of paths of the "Unix" type (more details below)	1
Archives/Archive/Files/File/Url	The URL of the source file. The source and target extension may not match.	1

Representation of paths for files within an archive

The representation for the path of a file within an archive is expected to be a "Unix" type of representation.

Also:

• Paths are relative to the root of the archive
• The root of the archive is represented by /. Paths can start with / or not. The paths below are equivalent:
  • /folder-1/image-1.jpg
  • folder-1/image-1.jpg

Compression modes

The compression modes available are:

• 7z

7Z is a modern, open source archive format, featuring AES encryption, native volume spanning, and high compression ratio (best one in mainstream formats), in many cases - depending on the properties of the input data - better than competing RAR and ZIPX formats.

• zip : Default value

ZIP is a popular compressed file archiving format, mainstream on Microsoft Windows systems ZIP support is nearly ubiquitous (i.e. most Linux distributions and Windows since XP have basic support for ZIP standard out of the box), making ZIP format the ideal choice when distributing files i.e. downloadable packages, mail attachments etc.

• tar

TAR is a pure-archiving format popular on Unix and Unix-like systems (often used for backup and for content distribution on those platforms). It does provide only archiving (concatenate input data and metadata in a single output file), delegating functions as compression, encryption, parity/integrity check, to external software.

• gzip

gzip is a file format and a software application used for file compression and decompression. The decompression of the gzip format can be implemented as a streaming algorithm, an important feature for Web protocols, data interchange and ETL (in standard pipes) applications.

gzip compression only compresses a single file

Compression is not available for a list of files organized within directories. It is only possible to compress a single file with the gzip compression method. It is generally advisable to first assemble the target files within a tar archive which can then be compressed with the gzip compression method.

If several files are mentioned in the list of files to be compressed, gzip compression will not be possible, a log will be added to the execution report.

Creating directories in generated archives is not supported.

Only a Path mentioning a file at the root of the archive is supported. Otherwise, the archive is not generated and a log will be added to the execution report.

Compression levels

The compression levels available are:

• NONE
• FASTEST
• FAST
• NORMAL : Default value
• MAX
• ULTRA

Outputs

TIP

The outputs of the task defined below are always available if the task completes its execution (status COMPLETED and FAILED). If the task is stopped before the end of its execution (if the task is canceled by the user for example - CANCELED status, or if it exceeds the authorized execution time - TIMED_OUT), the outputs are not available.

Property	Type	Description
		TThe first archive defines in the request parameter.
		The files generated.
		An xml file representing the list of the temporary files.
		YES | NO. YES: all files have been generated successfully. NO: at least one file has not been generated.
		the import report in XML format. See below

Output listing

WARNING

If an archive could not be generated, it will not appear in the listing document.

Example

<Files>
  <File>
    <Url>https://app.product-live.com/files-data-factory/d05a74cf11788d8f3ae9bf0e0e028dde66g0c83005c5e0d1211b0069945c0c11</Url>
    <File-Name>test-2-1.zip</File-Name>
  </File>
</Files>

Definition

XPath	Description	Occurs
Files	Root of the document.	1
./File	For each file.	0..*
./File/Url	The url of the temporary file.
./File/File-Name	The file name.	1

Limits and additional notes

• The maximum size of a file to add in an archive is 2GB.
• While there is not hard limit regarding the number of files that can be added in an archive, it is recommended to not exceed 500 files. If you need to add more files, you can split the archive in multiple archives.