Jobs
Requirement
- Complete the Create my first job tutorial.
- Install the VS Code extension.
Structure of the ZIP File
A job is the main execution unit of the Data Factory. It orchestrates one or more tasks (for example, importing files, transforming data, or generating exports) into a repeatable, auditable process that can be triggered by a user.
In Data Factory, a job is packaged as a ZIP archive. This archive is the single source of truth for the job definition and all the assets required at runtime:
| File/Folder | Description |
|---|---|
assets | Contains all files used by your job (for example, XSLT, templates, or other resources). |
tests | Contains test files (for example, input XML files for testing workflows). |
.jobignore | Lists files and folders that must be excluded when generating the final ZIP. |
CHANGELOG.md | Change log describing the history and versions of your job. |
job.json | Core job definition: metadata, tasks to execute, user inputs, and outputs. |
README.md | Job documentation for other users or for your future self. |
job.json
The job.json file is the heart of your job. It describes what the job does (metadata), how users interact with it (user inputs), which steps are executed (tasks), and what results are exposed at the end (output parameters).
The most important property is the tasks array, which defines the ordered flow of tasks that Data Factory will execute.
json
{
"schema": "1.0",
"key": "unique-job-key-in-your-account",
"title": "My job title",
"tasks": []
}json
{
"schema": "1.0",
"key": "unique-job-key-in-your-account",
"title": "My job title",
"icon": "file-add",
"titleLocal": {
"fra": "Le titre de mon job"
},
"description": "Description of the job",
"descriptionLocal": {
"fra": "La description du job"
},
"userInputs": [],
"tasks": [],
"outputParameters": {},
"timeoutSeconds": 0
}| Property | Description |
|---|---|
schema | Schema version of the job definition. Must be set to 1.0. |
key | Stable, unique identifier of the job within your Product-Live account. |
title | Human‑readable name of the job displayed to users. |
icon | Icon shown with the job in the UI. Use the VS Code extension autocomplete to browse the full enum list. |
titleLocal | Localization object for the title, keyed by language code (for example, fra, eng). |
description | Short description explaining what the job does. |
descriptionLocal | Localization object for the description, keyed by language code. |
userInputs | List of form fields presented to the user when they start the job. |
tasks | Ordered array of tasks that define the execution flow of the job. |
outputParameters | Mapping of user‑facing output keys to task outputs. These values are exposed in the settings.product-live.com interface. |
timeoutSeconds | Timeout for the job in seconds. If the job takes longer than the specified timeout, it will be considered failed (status: TIMEDOUT). If timeoutSeconds=0, no timeout is applied. |
Tasks
Tasks are the building blocks of a job. Each task performs a specific action, such as retrieving files from an FTP server, transforming data, or generating an .xlsx file.
The tasks array in job.json defines which tasks run and in which order. Refer to the Tasks documentation for the full catalog of available tasks, and see the Wiring Inputs and Outputs tutorial to learn how to connect task inputs and outputs together.
User Inputs
User inputs let you collect information from the user at runtime (for example, a catalog name, a language, or a file to upload). They behave like workflow input parameters: values entered by the user are then available inside your tasks.
To add a user input in job.json, use the VS Code extension autocompletion: press Ctrl + Space or type user after the title property.
There are three types of user inputs:
User Input: TEXT
json
{
"key": "catalog_name",
"title": "Catalog name",
"description": "The catalog name on the cover page",
"required": true,
"type": "TEXT",
"default": "New catalog"
}json
{
"key": "catalog_name",
"title": "Catalog name",
"titleLocal": {
"fra": "Nom du catalogue"
},
"description": "The catalog name on the cover page",
"descriptionLocal": {
"fra": "Le nom du catalogue affiché sur la page de garde"
},
"required": true,
"type": "TEXT",
"default": "New catalog"
}| Property | Description |
|---|---|
key | Unique identifier of the input. Used to reference the value in tasks. |
title | Label displayed to the user in the form. |
titleLocal | Localized labels for the input title. |
description | Additional explanation displayed under the field. |
descriptionLocal | Localized descriptions. |
required | true|false to indicate whether the field must be filled in. |
type | Must be TEXT. |
default | Default value used when the user does not change the field. |
User Input: SELECT
json
{
"key": "export_language",
"title": "Select language",
"description": "Select the language of the exported catalog",
"required": true,
"mode": "SINGLE",
"type": "SELECT",
"options": [
{
"key": "fra",
"title": "French"
},
{
"key": "spa",
"title": "Spanish"
}
],
"default": "fra"
}| Property | Description |
|---|---|
key | Unique identifier of the input. Used to reference the chosen value in tasks. |
title | Label displayed to the user. |
description | Additional explanation displayed under the field. |
required | true|false to indicate whether the field is mandatory. |
mode | Number of selectable options: SINGLE (default) | MULTIPLE. |
type | Must be SELECT. |
options | Array of selectable options (each with a key and a title). |
default | Default selected value (use the key of the option). |
User Input: FILE
json
{
"key": "zip_file",
"title": "Zip file",
"description": "Use a zip file with images at the root",
"required": true,
"type": "FILE"
}| Property | Description |
|---|---|
| key | Unique identifier of the input. Used to reference the uploaded file in tasks. |
| title | Label displayed to the user. |
| description | Additional explanation displayed under the field. |
| required | true|false to indicate if the file must be provided. |
| type | Must be FILE. |
User-provided parameters can be reused anywhere in your job (for example, in task inputs or templates). Learn more in the User Inputs tutorial.
The maximum file size for user input is 2 GB per file. Note: Upload time is limited to 1 minute, so the maximum size may vary depending on connection speed.
Once uploaded, the following file properties are accessible in your job:
json
{
"input": {
"file": {
"id": "65819ba43ad5b963658ab35d",
"url": "https://asset.product-live.com/data-factory/...",
"name": "my_zip_file.zip",
"hash": "70923562084497906fe165aa6ff2629f2d1e61d39d3e017709f7ac7722ddd2be",
"type": "file",
"key": "...",
"createdAt": "2024-04-22T13:43:21Z",
"updatedAt": "2024-04-22T13:43:21Z"
}
}
}User Outputs
User outputs are the results you want to expose to end users after the job finishes (for example, generated Excel files or ZIP archives). They are defined as key–value pairs in the outputParameters section of job.json, where the key is any label and the value references a task output.
Example:
json
"outputParameters": {
"excel_files": "${file_generation_xlsx.output.files}",
"zip_files": "${file_generation_archive.output.files}"
}Learn more in the User Outputs tutorial.
Variables and Secrets
Use variables and secrets to store:
- Global values for your jobs, such as a brand name.
- Protected content like login credentials, passwords, or tokens.
Access variables inside your job using the following syntax:
json
"${workflow.globalVariables.variable_key}"Learn more in the Variables and Secrets tutorial.
Localization
To support multiple languages for your job’s title and description, use the titleLocal and descriptionLocal objects shown in the “All Properties” example in the job.json section.
Supported languages:
Arabic: ara
Chinese: zho
English: eng
French: fra
German: deu
Hindi: hin
Italian: ita
Japanese: jpn
Korean: kor
Dutch: nld
Portuguese: por
Russian: rus
Spanish: spa
Best Practice
Use the VS Code extension to create and edit your jobs. It validates your job.json, provides autocomplete for icons, tasks, and user inputs, and simplifies packaging the ZIP.