12. Scalability and performance

Jobs are executed in Pipelines with the rule first in first out. If you need more resources you can add as many Pipelines as you want into your account. Contact your account manager for more information.

Below you will find a list of the best practices to optimise your Jobs.

Task Import Items

• In the Import Items XML file do not import elements that are empty. Even if they are ignored it can be significant in file size.
• Use differential to import only items that have been updated, and only fields that have been updated. You can use the XSLT differential items for this, you can find it in the examples of the XSLT examples : View tutorial. If your data are not in the Import Items format, you should first convert it to the Import Items XML format, and then use the differential.

Task Import Table Schema

• You should only import the attributes that have been updated since the last import. Read the full documentation of Import Table Schema task to learn more about the mode = INCREMENTAL. Note that if you use the mode = EXHAUSTIVE under the hood a differential is done before updating the schema.

XSLT

• Never use twice the same XPath, but use a variable that store the result.
• Do not use // to find elements
• If you are trying to achieve differential or trying to merge files use the xsl:merge function. You can find an example in the examples of the XSLT examples : View examples

Jobs periodicity

• Do not set periodicity less than every 30 minutes. If you need to, you should consider using another protocol like using a REST or SOAP api call.

Next

To learn more you can read the References and Examples sections.