To enable flexible pipelines and mapping groups, you can use variable values in transformations and mapping rules.
Variable types
You can use different types of variables with a pipeline, based on the value to return.
Variable type | Description |
---|---|
System | Provides information about when the pipeline runs:
|
Column | Returns the values for any column defined in the pipeline or mapping group |
Runtime | Enables variable values to be set when the pipeline runs |
Input a variable value
You can select a variable for some values within a transformation or mapping rule:
- To select a variable for a transformation, click Variable
- To select a variable for a mapping rule, click the From or To field.
You can include multiple variable values in a field, or combine variables with static values.
Create runtime variables for a pipeline or mapping group
You can create runtime variables to enable values to be set for transformations or mapping groups when the pipeline runs.
Note: You can use a runtime variable only with columns of the same data type. To use a variable with multiple columns of different data types, create a separate variable for each type.
- In the pipeline or mapping group, select the Variables tab.
- Add the variable:
- If the first variable, click Create variables.
- Otherwise, under Edit variables, click Add variable.
- If the first variable, click Create variables.
- Select the data type of the columns to use the variable, such as String or Number.
- If the variable's value is needed for the pipeline to run, check Required.
Note: If Required is checked and the value is not entered at runtime, the pipeline won't run, even if the variable has a default value.
- In Display name, enter a name to help identify the variable.
- Enter the variable's default value, if applicable.
- Click Save.
Tip: If you create a runtime variable for a mapping group, create a corresponding variable for any pipeline that uses the mapping group in a Mapping transformation. To help map the variables between the pipeline and mapping group, use the same name for both. To distinguish the mapping group's variable, append a prefix to its name, such as MG_
.
Set preview values for a pipeline's runtime variables
If a pipeline uses runtime variables, set the values to use to preview its transformations.
Note: If a runtime variable is required, the pipeline requires its value to preview transformations. If a runtime variable is not required, null
appears in the preview until its value is set.
- In the pipeline, select the Variables tab, and click Edit variables.
- Under Set preview variables, enter the values for the pipeline's runtime variables.
- Click Set values.
Note: The message, There was an error processing your transformation. Exception previewing pipeline #### on sample file ####. Variables [] are marked as required but do not have a non-empty value set.
will be displayed if a runtime variable is required but a preview is not populated.
Edit runtime variables
You can edit runtime variables at any time.
- In the pipeline or mapping group, select the Variables tab, and click Edit variables.
- Update the variables' data type, Required status, name, or default value as necessary.
- Click Save.
- If you edit the data type, also update any transformation that uses the variable value.
Delete runtime variables
If you no longer need a runtime variable, you can delete it from the pipeline or mapping group.
- In the pipeline or mapping group, select the Variables tab, and click Edit variables.
- For the variable to remove, click Delete .
- Click Save.
- Update any transformation that used the variable with a new value.