Data / Create logs
Task in alpha phase
This Data Factory task is in phase. You can contact the Product-Live team at contact@product-live.com if you want more details and get an early access.
This task allows to create Data Factory logs. Logs are visible in the job execution details and can be used to track the progress or report issues during a job execution.
Task name:
Log severity levels
The severity property defines the importance level of a log entry. The following values are available:
| Severity | Description |
|---|---|
info | Informational message indicating normal progress (default) |
warning | Warning message indicating a potential issue that did not prevent execution |
error | Error message indicating a failure during execution |
Examples
json
{
"name": "data-log-create",
"taskReferenceName": "create_log",
"description": "Create a log",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "INLINE",
"contentType": "application/json",
"model": "default",
"json": {
"operations": [
{
"key": "a",
"element": {
"message": "Import completed successfully",
"emitterName": "My import job"
}
}
]
}
}
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
json
{
"name": "data-log-create",
"taskReferenceName": "create_warning_log",
"description": "Create a warning log",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "INLINE",
"contentType": "application/json",
"model": "default",
"json": {
"operations": [
{
"key": "a",
"element": {
"severity": "warning",
"message": "Some items could not be processed",
"emitterName": "My import job"
}
}
]
}
}
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
TIP
See below for more examples using JSON and XML inputs.
Definition
| Property | Type | Required | Description |
|---|---|---|---|
| The name (or type) of the task | |||
| The unique name of the task in your job. It is used to reference this task in the workflow. | |||
| The functional description of this task in your job. | |||
The type of the task. It must be SUB_WORKFLOW. | |||
| : | true: the job continues if there is an error on this task. false: the job fails. | ||
| Input parameters of the task. See below |
Inputs Parameters
| Property | Type | Required | Description |
|---|---|---|---|
| The Product-Live API key. If not provided, an automatically generated API key will be used to perform the request on behalf of the current user. If provided, the given API key will be used to perform the request. | |||
| See below | |||
Today, application/json and application/xml are supported. | |||
INLINE: The request body is provided inline in the request object. ATTACHMENT: The request body is provided as an attachment in the request object. | |||
| The request body as a JSON object. | |||
| See below | |||
| A unique key to identify the element in the batch. This key will be associated to the element in the output. | |||
| The element to create or update. See examples below | |||
| Some element may require additional context to be created or updated. | |||
| The request body as a file. | |||
| The outputs of the request. Default: [{"mediaType": "application/json", "outputMode": "INLINE", "outputKey": "json"}]. | |||
The media type of the output. Today, application/json and application/xml are supported. | |||
INLINE: The output is provided inline in the output object. ATTACHMENT: The output is provided as an attachment in the output object. INLINE is only supported for mediaType=application/json | |||
| The key of the output. Either json or xml | |||
Default: DEFAULT. The model/format used in the output. Today, only 'DEFAULT' is supported. |
contentType=application/json
- The expected elements are the entities described in the Product-Live API or the Data Factory API depending on the task.
- See examples below for more details.
json
{
"operations": [
{
"key": "a",
"element": {
"...": "..."
}
}
]
}1
2
3
4
5
6
7
8
9
10
2
3
4
5
6
7
8
9
10
json
{
"operations": [
{
"key": "a",
"element": {
"...": "..."
}
}, {
"key": "b",
"element": {
"...": "..."
}
}
]
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
2
3
4
5
6
7
8
9
10
11
12
13
14
15
contentType=application/xml
- This XML representation follows the one used by the JSON query language of the Product-Live API.
- The expected elements are the entities described in the Product-Live API or the Data Factory API depending on the task.
- See examples below for more details.
xml
<?xml version="1.0" encoding="UTF-8"?>
<map xmlns="http://www.w3.org/2005/xpath-functions">
<array key="operations">
<map>
<string key="key">a</string>
<map key="context"></map>
<map key="element">
<string key="...">...</string>
</map>
</map>
</array>
</map>1
2
3
4
5
6
7
8
9
10
11
12
2
3
4
5
6
7
8
9
10
11
12
Outputs
| Property | Type | Description |
|---|---|---|
An object containing the outputs of the task. Each output key is the key of the output defined in the outputs[][outputKey] input parameter. |
Example
If your input parameters is as follows:
json
{
"...": "...",
"outputs": [
{
"mediaType": "application/json",
"outputMode": "INLINE",
"outputKey": "json_inline"
},
{
"mediaType": "application/xml",
"outputMode": "ATTACHMENT",
"outputKey": "xml_attachment"
}
]
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
2
3
4
5
6
7
8
9
10
11
12
13
14
15
The output will be as follows:
json
{
"outputs": {
"json_inline": {
"...": "...",
"...": "..."
},
"xml_attachment": {
"url": "..."
}
}
}1
2
3
4
5
6
7
8
9
10
11
2
3
4
5
6
7
8
9
10
11
Json output - outputs[].outputMode=INLINE and outputs[].mediaType=application/json
output.jsonis an array of objects as defined in the Data Factory Log API.jsonis the default output key but may be changed by setting theoutputKeyproperty.
XML output - outputs[].outputMode=ATTACHMENT and outputs[].mediaType=application/xml
Additional examples
Create a single log - XML input & output
job.json
json
{
"name": "data-log-create",
"taskReferenceName": "create_log",
"description": "Create a single log",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "ATTACHMENT",
"contentType": "application/xml",
"model": "default",
"file": "@my-request.xml"
},
"outputs": [
{
"mediaType": "application/xml",
"outputMode": "ATTACHMENT",
"outputKey": "xml"
}
]
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
my-request.xml
xml
<?xml version="1.1" encoding="UTF-8"?>
<map xmlns="http://www.w3.org/2005/xpath-functions">
<array key="operations">
<map>
<string key="key">a</string>
<map key="context"></map>
<map key="element">
<string key="object">data_factory_log</string>
<string key="message">Import completed successfully</string>
<string key="emitterName">My import job</string>
</map>
</map>
</array>
</map>1
2
3
4
5
6
7
8
9
10
11
12
13
14
2
3
4
5
6
7
8
9
10
11
12
13
14
Create many logs - XML input & output
job.json
json
{
"name": "data-log-create",
"taskReferenceName": "create_logs",
"description": "Create many logs",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "ATTACHMENT",
"contentType": "application/xml",
"model": "default",
"file": "@my-request.xml"
},
"outputs": [
{
"mediaType": "application/xml",
"outputMode": "ATTACHMENT",
"outputKey": "xml"
}
]
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
my-request.xml
xml
<?xml version="1.1" encoding="UTF-8"?>
<map xmlns="http://www.w3.org/2005/xpath-functions">
<array key="operations">
<map>
<string key="key">a</string>
<map key="context"></map>
<map key="element">
<string key="severity">info</string>
<string key="message">Step 1 completed</string>
<string key="emitterName">My import job</string>
</map>
</map>
<map>
<string key="key">b</string>
<map key="context"></map>
<map key="element">
<string key="severity">warning</string>
<string key="message">Step 2 completed with warnings</string>
<string key="emitterName">My import job</string>
</map>
</map>
</array>
</map>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Create a single log - JSON input & output
job.json
json
{
"name": "data-log-create",
"taskReferenceName": "create_log",
"description": "Create a single log",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "ATTACHMENT",
"contentType": "application/json",
"model": "default",
"file": "@my-request.json"
},
"outputs": [
{
"mediaType": "application/json",
"outputMode": "ATTACHMENT",
"outputKey": "json"
}
]
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
my-request.xml
json
{
"operations": [
{
"key": "a",
"context": {},
"element": {
"message": "Import completed successfully",
"emitterName": "My import job"
}
}
]
}1
2
3
4
5
6
7
8
9
10
11
12
2
3
4
5
6
7
8
9
10
11
12
Create many logs - JSON input & output
job.json
json
{
"name": "data-log-create",
"taskReferenceName": "create_logs",
"description": "Create many logs",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "ATTACHMENT",
"contentType": "application/json",
"model": "default",
"file": "@my-request.json"
},
"outputs": [
{
"mediaType": "application/json",
"outputMode": "ATTACHMENT",
"outputKey": "json"
}
]
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
_my-request.json
json
{
"operations": [
{
"key": "a",
"context": {},
"element": {
"message": "Step 1 completed",
"severity": "info",
"emitterName": "My import job"
}
},
{
"key": "b",
"context": {},
"element": {
"message": "Step 2 completed with warnings",
"severity": "warning",
"emitterName": "My import job"
}
}
]
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Create a log with markdown message
job.json
json
{
"name": "data-log-create",
"taskReferenceName": "create_markdown_log",
"description": "Create a log with markdown formatting",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "INLINE",
"contentType": "application/json",
"model": "default",
"json": {
"operations": [
{
"key": "a",
"element": {
"severity": "info",
"messageType": "MARKDOWN",
"message": "## Import summary\n- **Processed**: 150 items\n- **Skipped**: 3 items",
"emitterName": "My import job"
}
}
]
}
}
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Create an info log
job.json
json
{
"name": "data-log-create",
"taskReferenceName": "create_info_log",
"description": "Create an info log",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "INLINE",
"contentType": "application/json",
"model": "default",
"json": {
"operations": [
{
"key": "a",
"element": {
"severity": "info",
"message": "Import completed successfully: 150 items processed",
"emitterName": "My import job"
}
}
]
}
}
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Create a warning log
job.json
json
{
"name": "data-log-create",
"taskReferenceName": "create_warning_log",
"description": "Create a warning log",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "INLINE",
"contentType": "application/json",
"model": "default",
"json": {
"operations": [
{
"key": "a",
"element": {
"severity": "warning",
"message": "3 items were skipped due to missing required fields",
"emitterName": "My import job"
}
}
]
}
}
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Create an error log
job.json
json
{
"name": "data-log-create",
"taskReferenceName": "create_error_log",
"description": "Create an error log",
"type": "SUB_WORKFLOW",
"optional": false,
"inputParameters": {
"request": {
"method": "INLINE",
"contentType": "application/json",
"model": "default",
"json": {
"operations": [
{
"key": "a",
"element": {
"severity": "error",
"message": "Import failed: unable to connect to the remote server",
"emitterName": "My import job"
}
}
]
}
}
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26