To use pipelines to transform or map tabular input files within a chain, add a step with the Data Prep connector's Run pipeline command.
To enable this command, an IT admin first sets up a Data Prep connection.
To run a pipeline to transform or map a tabular input file, use a Run pipeline command.
Note: To use this command, first create the pipeline with the transformations to apply.
|Pipeline||Select the pipeline to run.|
|Runtime variables||If Pipeline includes runtime variables, enter their values.|
|File||Select the tabular data file to transform with the pipeline.
Tip: The pipeline requires a header row. If the file from your source doesn't have a header row, include the Tabular Transformation connector's Add header command earlier in the chain to insert it before this command.
|Field mappings||Map the columns from File to the columns defined for the pipeline.
Note: Field names are case-sensitive.
|Parallel processing||To run multiple instances of the pipeline at the same time, check this box.
Note: In general, parallel processing improves pipeline performance. However, performance may degrade with limited resources or when running a large number of instances at the same time.
|Delimiter||Select the delimiter used in File.|
|Output delimiter||Select the delimiter to use in Pipeline output.|
|Keep unmapped fields||If File contains more fields than included in the pipeline's column definition, check this box to output all fields, not just those in the column definition.
Note: Use caution with this property if the pipeline includes a Group by transformation.
Regardless of pipeline, the Run pipeline command always outputs:
If the pipeline includes a Mapping transformation, the command also outputs:
- A file that lists the unique field values for columns assigned the transformation but not transformed by a mapping rule
- A numeric count of the unmapped fields
If the pipeline includes a Sum if transformation, the command also outputs the numeric result of that transformation.
|0||Success||Pipeline completed successfully|
|1||Error||Pipeline failed to run or found unmapped rows|