HTTP

This task allows you to perform HTTP calls with advanced features including multiple authentication methods, retry mechanisms, redirect handling, and response assertions.

Authentication Guides

For detailed step-by-step instructions on setting up cloud provider authentication, see the Third-Party Integrations section:

• AWS S3 - Configure AWS Signature V4 for S3, API Gateway, etc.
• Azure Blob Storage - Configure Azure Blob Storage with SAS tokens.
• GCP Cloud Storage - Configure GCP Service Account for Cloud Storage, BigQuery, etc.
• OVH Object Storage - Configure OVH Object Storage (S3-compatible) credentials.
• Scaleway Object Storage - Configure Scaleway Object Storage (S3-compatible) credentials.

Examples

Simple GET Request

{
    "name": "protocol-http-perform",
    "taskReferenceName": "simple-get-request",
    "description": "A simple GET request",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.example.com",
        "path": "/v1/items",
        "headers": {
            "Accept": "application/json"
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

Header API Key Authentication

{
    "name": "protocol-http-perform",
    "taskReferenceName": "api-key-auth",
    "description": "Request with API key in header",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.product-live.com",
        "path": "/v1/items",
        "headers": {
            "X-Api-Key": "<REDACTED>",
            "Accept": "application/json"
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

Product-Live Key (Ephemeral)

{
    "name": "protocol-http-perform",
    "taskReferenceName": "pl-key-ephemeral",
    "description": "Automatically generated Product-Live API key",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.product-live.com",
        "path": "/v1/items",
        "headers": {
            "Accept": "application/json"
        },
        "connectionTimeOutMilliseconds": 10000,
        "authentication": {
            "useAuthentication": true,
            "type": "PL_KEY"
        }
    }
}

Product-Live Key (Explicit)

{
    "name": "protocol-http-perform",
    "taskReferenceName": "pl-key-explicit",
    "description": "User-defined Product-Live API key",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.product-live.com",
        "path": "/v1/items",
        "headers": {
            "Accept": "application/json"
        },
        "connectionTimeOutMilliseconds": 10000,
        "authentication": {
            "useAuthentication": true,
            "type": "PL_KEY",
            "plKey": "<REDACTED>"
        }
    }
}

Basic Authentication

{
    "name": "protocol-http-perform",
    "taskReferenceName": "basic-auth",
    "description": "Request with Basic authentication",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.example.com",
        "path": "/v1/protected",
        "authentication": {
            "useAuthentication": true,
            "type": "BASIC",
            "username": "myuser",
            "password": "<REDACTED>"
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

Bearer Token Authentication

{
    "name": "protocol-http-perform",
    "taskReferenceName": "bearer-auth",
    "description": "Request with Bearer token",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.example.com",
        "path": "/v1/protected",
        "authentication": {
            "useAuthentication": true,
            "type": "BEARER",
            "token": "<REDACTED>"
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

API Key Authentication

{
    "name": "protocol-http-perform",
    "taskReferenceName": "api-key-header-auth",
    "description": "Request with API key in custom header",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.example.com",
        "path": "/v1/protected",
        "authentication": {
            "useAuthentication": true,
            "type": "API_KEY",
            "apiKey": "<REDACTED>",
            "headerName": "X-Custom-Api-Key"
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

OAuth2 Client Credentials

{
    "name": "protocol-http-perform",
    "taskReferenceName": "oauth2-client-credentials",
    "description": "Request with OAuth2 Client Credentials flow",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.example.com",
        "path": "/v1/protected",
        "authentication": {
            "useAuthentication": true,
            "type": "OAUTH2_CLIENT_CREDENTIALS",
            "tokenUrl": "https://auth.example.com/oauth/token",
            "clientId": "<CLIENT_ID>",
            "clientSecret": "<CLIENT_SECRET>",
            "scope": "read write"
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

AWS Signature V4 - S3

{
    "name": "protocol-http-perform",
    "taskReferenceName": "aws-s3-list",
    "description": "List objects in AWS S3 bucket using Signature V4",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "my-bucket.s3.eu-west-1.amazonaws.com",
        "path": "/",
        "queryParameters": {
            "list-type": "2",
            "max-keys": "10"
        },
        "authentication": {
            "useAuthentication": true,
            "type": "AWS_SIGNATURE_V4",
            "accessKeyId": "<AWS_ACCESS_KEY_ID>",
            "secretAccessKey": "<AWS_SECRET_ACCESS_KEY>",
            "region": "eu-west-1",
            "service": "s3"
        },
        "responses": ["STRING"],
        "connectionTimeOutMilliseconds": 10000
    }
}

GCP Service Account

{
    "name": "protocol-http-perform",
    "taskReferenceName": "gcp-storage",
    "description": "Upload to GCP Cloud Storage",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "POST",
        "domain": "storage.googleapis.com",
        "path": "/upload/storage/v1/b/my-bucket/o?uploadType=media&name=test.txt",
        "headers": {
            "Content-Type": "text/plain"
        },
        "body": {
            "type": "BINARY_FILE",
            "file": {
                "url": "file://assets/test.txt"
            }
        },
        "authentication": {
            "useAuthentication": true,
            "type": "GCP_SERVICE_ACCOUNT_KEY",
            "gcpServiceAccountKeyProjectId": "my-project-id",
            "gcpServiceAccountKeyPrivateKeyId": "my-private-key-id",
            "gcpServiceAccountKeyPrivateKey": "-----BEGIN PRIVATE KEY-----\n...",
            "gcpServiceAccountKeyClientEmail": "my-sa@my-project.iam.gserviceaccount.com",
            "gcpServiceAccountKeyClientId": "my-client-id",
            "gcpServiceAccountKeyScope": ["https://www.googleapis.com/auth/cloud-platform"]
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

Query Parameters

{
    "name": "protocol-http-perform",
    "taskReferenceName": "query-params",
    "description": "Request with query parameters",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.product-live.com",
        "path": "/v1/items",
        "headers": {
            "X-Api-Key": "<REDACTED>",
            "Accept": "application/json"
        },
        "queryParameters": {
            "page": "0",
            "size": "10"
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

JSON Body

{
    "name": "protocol-http-perform",
    "taskReferenceName": "json-body",
    "description": "POST request with JSON body",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "POST",
        "domain": "api.product-live.com",
        "path": "/v1/items/find",
        "headers": {
            "X-Api-Key": "<REDACTED>",
            "Accept": "application/json"
        },
        "connectionTimeOutMilliseconds": 10000,
        "body": {
            "type": "JSON",
            "json": {
                "keyName": "EAN",
                "stringEquals": ["3541361805066"]
            }
        }
    }
}

Plain Body (Text/XML/CSV)

{
    "name": "protocol-http-perform",
    "taskReferenceName": "plain-body",
    "description": "POST request with plain text body",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "POST",
        "domain": "api.example.com",
        "path": "/v1/data",
        "body": {
            "type": "PLAIN",
            "contentType": "text/xml",
            "text": "<?xml version=\"1.0\"?><root><item>value</item></root>"
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

GraphQL Query

{
    "name": "protocol-http-perform",
    "taskReferenceName": "graphql-query",
    "description": "GraphQL query request",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "POST",
        "domain": "api.example.com",
        "path": "/graphql",
        "body": {
            "type": "GRAPHQL",
            "query": "query GetUser($id: ID!) { user(id: $id) { name email } }",
            "variables": {
                "id": "123"
            }
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

Multipart Form Body

{
    "name": "protocol-http-perform",
    "taskReferenceName": "multipart-form",
    "description": "Multipart form with file upload",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "POST",
        "domain": "api.product-live.com",
        "path": "/v1/data_factory/files",
        "headers": {
            "X-Api-Key": "<REDACTED>",
            "Accept": "application/json"
        },
        "connectionTimeOutMilliseconds": 10000,
        "body": {
            "type": "MULTIPART_FORM",
            "form": [
                {
                    "name": "file",
                    "type": "FILE",
                    "contentType": "multipart/form-data",
                    "file": "file://assets/table-import-schema.xml",
                    "filename": "table-import-schema.xml"
                }
            ]
        }
    }
}

Form URL Encoded

{
    "name": "protocol-http-perform",
    "taskReferenceName": "form-url-encoded",
    "description": "Form URL encoded body",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "POST",
        "domain": "api.example.com",
        "path": "/v1/login",
        "body": {
            "type": "FORM_URL_ENCODED",
            "form": [
                { "name": "username", "text": "myuser" },
                { "name": "password", "text": "mypassword" }
            ]
        },
        "connectionTimeOutMilliseconds": 10000
    }
}

Response as File

{
    "name": "protocol-http-perform",
    "taskReferenceName": "response-file",
    "description": "Download response as file",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.example.com",
        "path": "/v1/export.csv",
        "headers": {
            "X-Api-Key": "<REDACTED>"
        },
        "connectionTimeOutMilliseconds": 10000,
        "responses": ["FILE"]
    }
}

Retry Settings

{
    "name": "protocol-http-perform",
    "taskReferenceName": "with-retry",
    "description": "Request with automatic retry on failure",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.example.com",
        "path": "/v1/unstable-endpoint",
        "connectionTimeOutMilliseconds": 10000,
        "retrySettings": {
            "maxRetries": 3,
            "retryDelayMs": 1000,
            "backoffStrategy": "EXPONENTIAL",
            "maxRetryDelayMs": 30000,
            "retryOnStatusCodes": [429, 500, 502, 503, 504],
            "retryOnNetworkError": true,
            "retryOnTimeout": true
        }
    }
}

Redirect Settings

{
    "name": "protocol-http-perform",
    "taskReferenceName": "manual-redirect",
    "description": "Manual redirect handling",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.example.com",
        "path": "/v1/redirect-test",
        "connectionTimeOutMilliseconds": 10000,
        "redirectSettings": {
            "mode": "FOLLOW",
            "maxRedirects": 10,
            "followSameHost": false,
            "followSameProtocol": false,
            "followDowngrade": true,
            "preserveMethodOnRedirect": false
        }
    }
}

Assertions

{
    "name": "protocol-http-perform",
    "taskReferenceName": "with-assertions",
    "description": "Request with response assertions",
    "type": "SUB_WORKFLOW",
    "optional": false,
    "inputParameters": {
        "scheme": "HTTPS",
        "method": "GET",
        "domain": "api.example.com",
        "path": "/v1/health",
        "connectionTimeOutMilliseconds": 10000,
        "assertions": {
            "enabled": true,
            "failOnError": true,
            "assertions": [
                {
                    "name": "Status is 200",
                    "target": "STATUS_CODE",
                    "operator": "EQUALS",
                    "expectedValue": 200
                },
                {
                    "name": "Response time under 2s",
                    "target": "RESPONSE_TIME",
                    "operator": "LESS_THAN",
                    "expectedValue": 2000
                },
                {
                    "name": "Content-Type is JSON",
                    "target": "HEADER",
                    "targetKey": "content-type",
                    "operator": "CONTAINS",
                    "expectedValue": "application/json"
                },
                {
                    "name": "Status field is healthy",
                    "target": "BODY_JSON_PATH",
                    "targetKey": "$.status",
                    "operator": "EQUALS",
                    "expectedValue": "healthy"
                }
            ]
        }
    }
}

Definition

Property	Type	Required	Description
			The name (or type) of the task
			The unique name of the task in your job. It is used to reference this task in the workflow.
			The functional description of this task in your job.
			The type of the task. It must be SUB_WORKFLOW.
	:		true: the job continues if there is an error on this task. false: the job fails.
			Input parameters of the task. See below

Inputs

Base Configuration

Property	Type	Required	Description
	: ,		The scheme of the service Default: HTTPS
			The complete domain of the service No validation is performed on the domain name. The domain name must not contain the scheme, the port or the path The domain must be at least 1 character long example: api.product-live.com
			The port of the service Default if scheme = HTTP : 80, if scheme = HTTPS : 443 example: 8080
			The path of the service example: /v1/items
	:		The method of the request Default: GET example: POST
			A JSON object containing the headers of the request Max number of headers is 100 Max length of a header name is 100 characters Max length of a header value is 10 000 characters example: {"X-Api-Key": "<REDACTED>"}
			A JSON object containing the query parameters of the request Max number of query parameters is 100 Max length of a query parameter name is 100 characters Max length of a query parameter value is 10 000 characters example: {"page": "1"}
			The body of the request
	: , ,		The type of the body Default: JSON
			The body of the request in JSON format example: {"name": "example"}
			The body of the request in Form format
			The name of the form data example: name
	: ,		The type of the form data example: TEXT
			The text value example: text value
			The content type of the form data if the type is file. If not specified, the content type is deduced from the filename. example: image/jpeg
			The filename of the form data if the type is file. If not specified, the filename is deduced from the url. example: file.txt
			The file representation example: {"object": "data_factory_file","id": "string","url": "string","filename": "string"}
			The authentication of the request, see below
			true: use authentication, false: do not use authentication, every other value is considered as false Default: true
			Enum - NONE | PL_KEY | GCP_SERVICE_ACCOUNT_KEY
			If mentioned, the authentication is made using this PL key, if not, an API key is automatically generated to perform requests against the Product-Live API on behalf of the user who started the current job
			The project id of the GCP service account key example: my-project-id
			The private key id of the GCP service account key example: my-private-key-id
			The private key of the GCP service account key example: -----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQ...
			The client email of the GCP service account example: my-service-account@xxx.iam.gserviceaccount.com
			The client id of the GCP service account key example: my-client-id
			The scope of the GCP service account key example: ["https://www.googleapis.com/auth/cloud-platform"]
			The connection timeout in milliseconds, if the value is 0 then no timeout is set Default: 1000 example: 5000
			The read timeout in milliseconds, if the value is 0 then no timeout is set Default: 150 example: 1000
	: , ,		The type of the response Default: [JSON, FILE, STRING]
	:		Default: false When set to true, the report generation is skipped and thus the report is not available in the output parameters.
	:		Default: INFO It defines the log level to be recorded in the report. ERROR < WARNING < INFO < DEBUG

Body

Property	Type	Required	Description
			The body of the request
	: , , , , ,		The type of the body Default: JSON

JSON Body (body.type = JSON)

Property	Type	Required	Description
			The body of the request in JSON format example: {"name": "example"}

Plain Body (body.type = PLAIN)

Property	Type	Required	Description
			The content type (e.g., text/plain, text/xml, text/csv, application/xml)
			The text content of the body
			Alternative: a file containing the body content

GraphQL Body (body.type = GRAPHQL)

Property	Type	Required	Description
			The GraphQL query
			GraphQL variables

Multipart Form Body (body.type = MULTIPART_FORM)

Property	Type	Required	Description
			Array of form fields
			The name of the form field
	: ,		The type of the form field
			The text value
			The file reference example: {"url": "file://assets/data.xml"}
			The content type of the form field (auto-detected if not specified)
			The filename (auto-detected from URL if not specified)

Form URL Encoded Body (body.type = FORM_URL_ENCODED)

Property	Type	Required	Description
			Array of form fields
			The field name
			The field value

Binary File Body (body.type = BINARY_FILE)

Property	Type	Required	Description
			The file to send as binary example: {"url": "file://assets/image.png"}

Authentication

Property	Type	Required	Description
			The authentication configuration
			true: use authentication, false: do not use authentication Default: true
			One of: NONE, PL_KEY, BASIC, BEARER, API_KEY, OAUTH2_CLIENT_CREDENTIALS, AWS_SIGNATURE_V4, GCP_SERVICE_ACCOUNT_KEY

No Authentication (authentication.type = NONE)

No additional properties required.

Product-Live Key (authentication.type = PL_KEY)

Property	Type	Required	Description
			If provided, uses this API key. Otherwise, an ephemeral API key is automatically generated for the current job's user context

Basic Authentication (authentication.type = BASIC)

Property	Type	Required	Description
			The username
			The password

Bearer Token (authentication.type = BEARER)

Property	Type	Required	Description
			The bearer token

API Key (authentication.type = API_KEY)

Property	Type	Required	Description
			The API key value
			The header name Default: X-Api-Key

OAuth2 Client Credentials (authentication.type = OAUTH2_CLIENT_CREDENTIALS)

Property	Type	Required	Description
			The OAuth2 token endpoint URL
			The OAuth2 client ID
			The OAuth2 client secret
			OAuth2 scope(s)

AWS Signature V4 (authentication.type = AWS_SIGNATURE_V4)

Property	Type	Required	Description
			AWS Access Key ID
			AWS Secret Access Key
		<	AWS Region (e.g., us-east-1, eu-west-1)
			AWS Service name (e.g., s3, execute-api, es)
			AWS Session Token for temporary credentials (STS)

GCP Service Account Key (authentication.type = GCP_SERVICE_ACCOUNT_KEY)

Property	Type	Required	Description
			The project ID of the GCP service account key
			The private key ID
			The private key (PEM format)
			The client email
			The client ID
			The OAuth2 scopes example: ["https://www.googleapis.com/auth/cloud-platform"]

Retry Settings

Property	Type	Required	Description
			Configuration for automatic retries on failure
			Maximum retry attempts (0-10) Default: 0 (no retries)
			Initial delay between retries in milliseconds (0-300,000) Default: 1000
	: , ,		Backoff strategy: - FIXED: Always use retryDelayMs - LINEAR: retryDelayMs * attemptNumber - EXPONENTIAL: retryDelayMs * (2 ^ attemptNumber) Default: EXPONENTIAL
			Maximum delay between retries (caps the backoff) (0-300,000) Default: 30000
			HTTP status codes that trigger a retry Default: [429, 500, 502, 503, 504]
			Retry on network errors (connection refused, etc.) Default: true
			Retry on timeout errors Default: true

Redirect Settings

Property	Type	Required	Description
			Configuration for redirect handling
	: , ,		Redirect mode: - FOLLOW: Automatically follow redirects - MANUAL: Do not follow, return redirect response - ERROR: Fail on redirect Default: FOLLOW
			Maximum number of redirects to follow (0-50) Default: 10
			Only follow redirects to the same host Default: false
			Only follow redirects with the same protocol Default: false
			Allow HTTPS to HTTP downgrade Default: true
			Preserve HTTP method on redirect (instead of converting to GET) Default: false

Assertions

Property	Type	Required	Description
			Configuration for response assertions
			Enable assertions Default: false
			Fail the task if any assertion fails Default: false
			Array of assertion definitions

Assertion Definition

Property	Type	Required	Description
			Name for the assertion
			Target to assert: STATUS_CODE, RESPONSE_TIME, RESPONSE_SIZE, HEADER, COOKIE, BODY_STRING, BODY_JSON_PATH
			For BODY_JSON_PATH, use JSONPath syntax (e.g., $.data.id)
			Assertion operator (see table below)
			The expected value (for operators that require it)
			For IS_TYPE operator: STRING, NUMBER, BOOLEAN, OBJECT, ARRAY, NULL
			For IN, NOT_IN operators: array of expected values

Assertion Operators

Operator	Description
EQUALS	Value equals expected
NOT_EQUALS	Value does not equal expected
CONTAINS	Value contains expected string
NOT_CONTAINS	Value does not contain expected string
STARTS_WITH	Value starts with expected string
ENDS_WITH	Value ends with expected string
MATCHES_REGEX	Value matches regular expression
EXISTS	Property exists
NOT_EXISTS	Property does not exist
LESS_THAN	Value is less than expected
LESS_THAN_OR_EQUALS	Value is less than or equal to expected
GREATER_THAN	Value is greater than expected
GREATER_THAN_OR_EQUALS	Value is greater than or equal to expected
IS_NULL	Value is null
IS_NOT_NULL	Value is not null
IS_EMPTY	Value is empty (string, array, object)
IS_NOT_EMPTY	Value is not empty
IS_TYPE	Value is of expected type
IN	Value is in expected values array
NOT_IN	Value is not in expected values array

Outputs

Base Response

Property	Type	Description
		The HTTP status code of the response example: 200
		The reason phrase of the response example: OK
		The headers of the response example: {"Content-Type": "application/json"}
		If responses includes JSON, the body in JSON format
		If responses includes FILE, the body as a file reference
		If responses includes STRING, the body as a string
		The cURL command to reproduce the request (API keys are obfuscated)
		The full URL of the request
		An XML file containing a set of logs related to the execution of the task. This output is not available if the input skipReportGeneration=true.

Metrics

Property	Type	Description
		Response time in milliseconds
		Response body size in bytes

Cookies

Property	Type	Description
		Cookies set by the response example: {"session_id": "abc123"}

Assertion Results

Property	Type	Description
		Results of assertion evaluation (if assertions enabled)
		Whether all assertions passed
		Total number of assertions evaluated
		Number of passed assertions
		Number of failed assertions
		Detailed results for each assertion

Redirect Info

Property	Type	Description
		Information about redirects (if any occurred)
		Whether the request was redirected
		Number of redirects followed
		The original request URL
		The final URL after redirects
		Array of redirect steps with status codes and URLs

Retry Info

Property	Type	Description
		Information about retries (if any occurred)
		Total attempts made (1 = no retries)
		Number of retries performed
		Whether any retries were performed
		Total time spent waiting between retries
		Details of each attempt including status codes, errors, and timing

See Also

• Data Factory Variables - How to manage credentials securely