File generation / Generate Archive

Allows to generate an archive with a .zip extension. For example it can be used to generate an archive of images or files generated by another task.

TIP

This task allows generating one or more archives.

Task name:

Example

{
  "name": "file-generation-archive",
  "taskReferenceName": "generate_archive",
  "description": "The business description of the task",
  "type": "SUB_WORKFLOW",
  "optional": false,
  "inputParameters": {
    "request": "${previous_taskReferenceName.output.file}",
    "allowPartial": true
  }
}

Definition

Property	Type	Required	Description
			The name (or type) of the task
			The unique name of the task in your job. It is used to reference this task in the workflow.
			The functional description of this task in your job.
			The type of the task. It must be SUB_WORKFLOW.
	:		true: the job continues if there is an error on this task. false: the job fails.
			Input parameters of the task. See below

Inputs

TIP

If at least one task parameter (whether mandatory or not) is invalid, task execution is stopped and the returned status is FAILED. For example:

• The proposed value for the allowPartial property is invalid.
• The file specified for the request property does not exist

Property	Type	Required	Description
			See the complete definition below. It can reference the output of a previous task, example: ${previous_taskReferenceName.output.file} , or a static file, example: file://assets/file.xml, or a user input, example: ${workflow.input.input_key}.
			true: if there is an error adding a file to an archive, the archive is still generated although it is incomplete false: If there is an error adding a file, the archive is not generated, a log is added to the execution report and the task continues to run.
	:		Default: false When set to true, the report generation is skipped and thus the report is not available in the output parameters.
	:		Default: info It defines the log level to be recorded in the report. error < warning < info < debug

false: If there is an error adding a file, the archive is not generated, a log is added to the execution report and the task continues to run

Input request

Example

<Archives>
    <Archive>
        <File-Name>test-2-1.zip</File-Name>
        <Compression-Method>7z</Compression-Method>
        <Compression-Level>NORMAL</Compression-Level>
        <Password>********</Password>
        <Files>
            <File>
                <Path>/folder-1/image-1.jpg</Path>
                <Url>https://example.com/ee0c68fabf0a2b3c76b2643bff3cc215030f0caf.jpg</Url>
            </File>
            <File>
                <Path>/image-2.jpg</Path>
                <Url>https://example.com/7a597747dbc536e53eeb47760b5145d002b5bc25.jpg</Url>
            </File>
        </Files>
    </Archive>
</Archives>

Definition

You can use a static file stored in the asset folder, or you can generate this file from a Transform XSLT task.

XPath	Description	Occurs
Archives	Root	1
Archives/Archive	For each archive to be generated.	1..*
Archives/Archive/File-Name	The name of the generated archive. The name must include the extension.	1
Archives/Archive/Compression-Method	The type of compression to use. Possible values: zip, 7z, tar, gzip Default: zip	0..1
Archives/Archive/Compression-Level	The level of compression to use. Possible values: NONE, FASTEST, FAST, NORMAL, MAX, ULTRA Default value: NORMAL Compression tar only accepts NORMAL compression level	0..1
Archives/Archive/Password	The password to use for compression. Only available for 7z and zip compression	0..1
Archives/Archive/Files	The files contained in the archive	1
Archives/Archive/Files/File	For each file to put in the archive. Creating an empty archive is allowed.	0..*
Archives/Archive/Files/File/Path	The path of the file in the archive. Notes: We use here a representation of paths of the "Unix" type (more details below)	1
Archives/Archive/Files/File/Url	The URL of the source file. The source and target extension may not match.	1

Zips / Zip: the former element names still work

Archives and Archive used to be called Zips and Zip. Those older names are deprecated but still accepted, at both levels and even mixed with the current ones - a document opening on <Zips> whose children are <Archive> is read exactly like a current one. Any other root element is rejected.

Prefer Archives / Archive for new jobs.

Representation of paths for files within an archive

The representation for the path of a file within an archive is expected to be a "Unix" type of representation.

Also:

• Paths are relative to the root of the archive
• The root of the archive is represented by /. Paths can start with / or not. The paths below are equivalent:
  • /folder-1/image-1.jpg
  • folder-1/image-1.jpg

Compression modes

The compression modes available are:

• 7z

7Z is a modern, open source archive format, featuring AES encryption, native volume spanning, and high compression ratio (best one in mainstream formats), in many cases - depending on the properties of the input data - better than competing RAR and ZIPX formats.

• zip : Default value

ZIP is a popular compressed file archiving format, mainstream on Microsoft Windows systems ZIP support is nearly ubiquitous (i.e. most Linux distributions and Windows since XP have basic support for ZIP standard out of the box), making ZIP format the ideal choice when distributing files i.e. downloadable packages, mail attachments etc.

• tar

TAR is a pure-archiving format popular on Unix and Unix-like systems (often used for backup and for content distribution on those platforms). It does provide only archiving (concatenate input data and metadata in a single output file), delegating functions as compression, encryption, parity/integrity check, to external software.

• gzip

gzip is a file format and a software application used for file compression and decompression. The decompression of the gzip format can be implemented as a streaming algorithm, an important feature for Web protocols, data interchange and ETL (in standard pipes) applications.

gzip compression only compresses a single file

Compression is not available for a list of files organized within directories. It is only possible to compress a single file with the gzip compression method. It is generally advisable to first assemble the target files within a tar archive which can then be compressed with the gzip compression method.

If several files are mentioned in the list of files to be compressed, gzip compression will not be possible, a log will be added to the execution report.

Creating directories in generated archives is not supported.

Only a Path mentioning a file at the root of the archive is supported. Otherwise, the archive is not generated and a log will be added to the execution report.

Compression levels

The compression levels available are:

• NONE
• FASTEST
• FAST
• NORMAL : Default value
• MAX
• ULTRA

Outputs

TIP

The outputs of the task defined below are always available if the task completes its execution (status COMPLETED and FAILED). If the task is stopped before the end of its execution (if the task is canceled by the user for example - CANCELED status, or if it exceeds the authorized execution time - TIMED_OUT), the outputs are not available.

Property	Type	Description
		The first archive defined in the request parameter.
		The files generated.
		An xml file representing the list of the temporary files.
		YES | NO. YES: all files have been generated successfully. NO: at least one file has not been generated.
		the import report in XML format. See below

Output listing

WARNING

If an archive could not be generated, it will not appear in the listing document.

Example

<Files>
  <File>
    <Url>https://app.product-live.com/files-data-factory/3456...cdef</Url>
    <File-Name>test-2-1.zip</File-Name>
  </File>
</Files>

Definition

XPath	Description	Occurs
Files	Root of the document.	1
./File	For each file.	0..*
./File/Url	The url of the temporary file.
./File/File-Name	The file name.	1

Execution report log codes

Each log carries an xpath locating what it is about, of the form /Archives/Archive[position()=N]/Files/File[position()=M]/.

Code	Severity	Meaning
ARCHIVE_GENERATED	info	The archive was generated. One log per generated archive.
ARCHIVE_NOT_GENERATED	warning	The archive was not generated. Always present when an archive is missing, in addition to the log explaining why.
ARCHIVE_IS_EMPTY	warning	No file was provided, so an empty archive was generated.
INVALID_ARCHIVE_NAME	warning	The archive name is not valid, so that archive was not generated.
FILENAME_ALREADY_EXISTS	warning	A file of that name is already in the archive; the duplicate was not added.
URL_NOT_REACHABLE	warning	A file url could not be reached. With allowPartial = true the archive is generated without that file; with false the archive is not generated.
INVALID_COMPRESSION_METHOD	warning	The requested compression method is not valid.
INVALID_COMPRESSION_LEVEL	warning	The requested compression level is not valid.
EMPTY_TAG	warning	No compression level was provided; NORMAL was used.
GZIP_COMPRESSION_IS_NOT_SUPPORTED_FOR_MULTIPLE_FILES	warning	gzip was requested for an archive holding several files.
GZIP_COMPRESSION_INVALID_PATH	warning	gzip was requested for a file path it does not support.
INVALID_INPUT	error	An input value is not valid. The log names it in inputName.

A log does not mean the task failed

Two different things can go wrong, and only one of them fails the task:

• Before archiving - an invalid task parameter stops the execution and returns FAILED, as stated at the top of Inputs.
• While archiving - everything in the table above is reported as a log and the task carries on to the next archive. Whatever the severity of those logs, info, warning or error, the task ends in success. It only fails here on an unexpected technical error, one outside that list; the report is then still attached to the failed task.

So once archiving has started, a COMPLETED status tells you nothing about how many archives were actually produced. Read the listing and the report to find out.

Limits and additional notes

• The maximum size of a file to add in an archive is 2GB.
• While there is not hard limit regarding the number of files that can be added in an archive, it is recommended to not exceed 500 files. If you need to add more files, you can split the archive in multiple archives.