Learning Center

Data Factory Use Cases ​

Create an asset in a GCP bucket ​

Prerequisites ​

To execute this use case, you need:

  • A Product-Live account with access to the Data Factory platform
  • A GCP account with a bucket created and accessible from the Product-Live platform

Setup ​

  1. Create a GCP service account (to this date, service account is the only authentication method supported by Product-Live to access GCP buckets) with the right permission to access the target bucket. The service account must have the following roles. See GCP documentation for more information on how to create a service account and assign roles: https://cloud.google.com/iam/docs/service-accounts-create
  2. Gather the necessary information to authenticate against the GCP API:
  • The project ID
  • The private key ID
  • The private key
  • The client email
  • The client ID
  • the scopes
  1. Create the necessary Data Factory variables (used in the job we will create in the next step):
  • gcp_service_account_key_project_id
  • gcp_service_account_key_private_key_id
  • gcp_service_account_key_private_key
  • gcp_service_account_key_client_email
  • gcp_service_account_key_client_id
  1. Download and edit this example and edit the GCP bucket name ([bucket-name]) to match your GCP bucket name.
  2. Upload the edited job to your Data Factory account. This job will create a single file in the target bucket named test.txt with the content Hello World!.

You are now ready to launch the job, if everything is set up correctly, you should see the file test.txt in your GCP bucket.

Data Factory Job details ​

TIP

The job presented here is very basic and is intended to illustrate the process of creating items using a Data Factory job triggered by an API. It can be enhanced and enriched to suit your needs. For more information, please refer to the Data Factory platform documentation.

mermaid
flowchart TD
	_start(("Start"))
	_end(("End"))
	protocol_http_perform["<b><a href='/data-factory/references/tasks/protocol-http-perform/'>protocol-http-perform</a></b><br />"]
	_start --> protocol_http_perform
	protocol_http_perform --> _end