Data Factory API
Before going further
We recommend that you have familiarized yourself with how to connect Product-Live to your systems.
Overview
Product-Live offers a series of REST APIs enabling project managers and developers to leverage the full potential of the Data Factory platform. These APIs can be used to launch a job or retrieve the result, but they can also be used to create a Data Factory task tailored to answer a specific need.
The API portal is available here and you can find the OpenAPI definition here.
A Typescript SDK is available to speed up the development of your integration.
Requirements for using Product-Live Data Factory APIs
- An active Product-Live account
- A valid API key
- A Data Factory pipeline
API scopes and permissions
To date, an API key allow its owner to perform any action on the Data Factory platform.
Authentication
Authentication is the process of identifying the user who is making the request. Authorization, on the other hand, is the process of granting access to a certain type of resource. The Data Factory APIs enable you to perform any action on the Data Factory platform on your account.
Product-Live uses API Key authentication strategy. To generate or manage your API key, go to Product-Live Settings > API. An API key may be revoked at any time by its owner.
Authenticate an API request
To authenticate an API request, you should provide your API key in the HTTP X-Api-Key header. For example to simply fetch a job execution:
bash
curl 'https://api.product-live.com/v1/data_factory/job_executions/[job execution id]' -H 'X-Api-Key: <REDACTED>'1
ts
import axios from 'axios';
const jobExecutionId = '[job execution id]';
const apiKey = '<token>';
const url = `https://api.product-live.com/v1/data_factory/job_executions/${jobExecutionId}`;
axios.get(url, {
headers: {
'X-Api-Key': apiKey,
},
})
.then(response => {
// Handle the response here
console.log(response.data);
})
.catch(error => {
// Handle errors here
console.error(error);
});1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
ts
import { createConfiguration, ServerConfiguration, JobExecutionApi } from '@product-live/api-sdk';
export async function main(): Promise<void> {
const jobExecutionId = 'job execution id';
const configuration = createConfiguration({
baseServer: new ServerConfiguration(process.env.API_BASE_PATH || '', {}),
authMethods: {
ApiKeyAuthHeader: process.env.API_ACCESS_TOKEN
}
});
const jobExecutionApi = new JobExecutionApi(configuration);
const jobExecution = await jobExecutionApi.getJobExecutionById(jobExecutionId);
}1
2
3
4
5
6
7
8
9
10
11
12
13
2
3
4
5
6
7
8
9
10
11
12
13
go
package main
import (
"fmt"
"net/http"
"io/ioutil"
)
func main() {
url := "https://api.product-live.com/v1/data_factory/job_executions/[job execution id]"
token := "<token>"
req, err := http.NewRequest("GET", url, nil)
if err != nil {
fmt.Println("Error creating HTTP request:", err)
return
}
req.Header.Set("X-Api-Key", token)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
fmt.Println("Error sending HTTP request:", err)
return
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println("Error reading HTTP response:", err)
return
}
fmt.Println("Response Body:", string(body))
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
javascript
const fetch = require('node-fetch');
const url = 'https://api.product-live.com/v1/data_factory/job_executions/[job execution id]';
const token = '<token>';
fetch(url, {
method: 'GET',
headers: {
'X-Api-Key': token
}
})
.then(response => response.json())
.then(data => {
// Process the response data
console.log(data);
})
.catch(error => {
// Handle any errors
console.error('Error:', error);
});1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
python
import requests
url = 'https://api.product-live.com/v1/data_factory/job_executions/[job execution id]'
headers = {
'X-Api-Key': '<token>'
}
response = requests.get(url, headers=headers)
# Access the response data
data = response.json()1
2
3
4
5
6
7
8
9
10
11
2
3
4
5
6
7
8
9
10
11
java
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class Main {
public static void main(String[] args) throws IOException {
String url = "https://api.product-live.com/v1/data_factory/job_executions/[job execution id]";
String apiKey = "<token>";
URL apiUrl = new URL(url);
HttpURLConnection connection = (HttpURLConnection) apiUrl.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("X-Api-Key", apiKey);
int responseCode = connection.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String inputLine;
StringBuilder response = new StringBuilder();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
System.out.println(response.toString());
} else {
System.out.println("Request failed with response code: " + responseCode);
}
connection.disconnect();
}
}1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Note that it is also possible to pass the API key within the query string using the api_key parameter. For example:
bash
curl 'https://api-next.product-live.com/v1/data_factory/job_executions/[job execution id]?api_key=<REDACTED>'1
Versioning
API versioning enables Product-Live to evolve its platform consistently, while providing third-party developers with a reliable roadmap for feature enhancements and deprecations.
To stay informed about upcoming API modifications, we recommend following our Release Notes. Product-Live ensures full backward compatibility of its API within a major version.
Rate limits
Product-Live APIs are subject to rate limiting. The default rate limit for all Product-Live APIs is 50 requests per minute. However, some limitations may apply to specific endpoints. The rate limit is applied to the API key making the requests.
In addition to the rate limit per minute, the following limits apply:
- Maximun request per second: 5 requests per second.
- Maximun request per month: 40,000 requests per month.
When the rate limit is exceeded, the API may return a 429 Too Many Requests status code.
For large-scale applications
If you need a higher rate limit, our team can help you. Please contact us for more information.
Pagination
The API supports pagination for all endpoints that return a list of entities.
Query parameters
| Parameter | Type | Description | Default | Required |
|---|---|---|---|---|
page | integer | The page number to retrieve. Starts at 0. | 0 | false |
size | integer | The number of items per page. The maximum value is 1000. | 20 | false |
Example
For instance, the List Variables endpoint returns a list of variables. This list can be paginated by using the size and page query parameters.
bash
curl --request GET \
--url 'https://api.product-live.com/v1/data_factory/variables?page=1&size=5' \
--header 'Content-Type: application/json' \
--header 'X-Api-Key: <REDACTED>'1
2
3
4
2
3
4
HTTP response codes
The API uses standard HTTP status codes to indicate the success or failure of a request. The following table lists the most common status and error codes to indicate the specific error that occurred.
TIP
Specific error codes to certain endpoints are listed in the documentation for that endpoint.
| Status code | Error Code | Description |
|---|---|---|
| 200 | OK. The request was successful. | |
| 201 | Created. The request was successful and a new resource was created. | |
| 400 | BAD_REQUEST | Bad request. The request was unacceptable, often due to missing a required parameter. See error code below. |
| 401 | UNAUTHENTICATED | Unauthorized. The provided API key is invalid. |
| 403 | Forbidden. The provided API key does not have the required permissions. | |
| 404 | NOT_FOUND | Not found. The requested resource does not exist. |
| 429 | RATE_LIMIT | Too many requests. The rate limit has been exceeded. |
ALREADY_EXISTS | A resource with this id already exists. | |
| 5xx | Internal server error. |
json
{
"date": "Thu Aug 10 2023 12:00:00 GMT+0000 (Coordinated Universal Time)",
"code": "BAD_REQUEST",
"message": "Missing property 'name'",
"httpStatusCode": 400
}1
2
3
4
5
6
2
3
4
5
6
Date and time format
The format used is always the ISO 8601 format.
ISO 8601 is an international standard covering the worldwide exchange and communication of date- and time-related data. The standard aims to provide a well-defined, unambiguous method of representing calendar dates and times in worldwide communications, especially to avoid misinterpreting numeric dates and times when such data is transferred between countries with different conventions for writing numeric dates and times.
SDKs
The API portal is available here and you can find the OpenAPI definition here. The OpenAPI definition can be used to generate SDKs for different programming languages.
A Typescript SDK is available to speed up the development of your integration. The updated of this SDK follows the release of new versions of the API.
Entities
| Entity | Description |
|---|---|
| Files | A Data Factory file to be used in a job |
| Jobs | A Data Factory job. |
| Job Executions | A job execution represents the execution of a job. |
| Tasks (or custom tasks) | A task is a step of a job. |
| Task Executions | A task execution represents the execution of a task. |
| Variables | A variable is a key-value pair that can be used to store reusable values. |