To use pipelines to transform or map tabular input files within a chain, add a step with the Data Prep connector's Run pipeline command.
To enable this command, an IT admin first sets up a Data Prep connection.
Run pipeline
To run a pipeline to transform or map a tabular input file, use a Run pipeline command.
Note: To use this command, first create the pipeline with the transformations to apply.
Equivalent API call: https://support.workiva.com/hc/en-us/articles/4405086491156-Data-Prep-connector
Properties
| Property | Detail |
|---|---|
| Pipeline | Select the pipeline to run. |
| Runtime variables | If Pipeline includes runtime variables, enter their values. |
| File | Select the tabular data file to transform with the pipeline.
Tip: The pipeline requires a header row. If the file from your source doesn't have a header row, include the Tabular Transformation connector's Add header command earlier in the chain to insert it before this command. |
| Field mappings | Map the columns from File to the columns defined for the pipeline.
Note: Field names are case-sensitive. |
| Parallel processing | To run multiple instances of the pipeline at the same time, check this box.
Note: In general, parallel processing improves pipeline performance. However, performance may degrade with limited resources or when running a large number of instances at the same time. |
| Delimiter | Select the delimiter used in File. |
| Output delimiter | Select the delimiter to use in Pipeline output. |
| Keep unmapped fields | If File contains more fields than included in the pipeline's column definition, check this box to output all fields, not just those in the column definition.
Note: Use caution with this property if the pipeline includes a Group by transformation. |
Outputs
Regardless of pipeline, the Run pipeline command always outputs:
| Output | Output type |
|---|---|
| Record count | Integer |
| Pipeline output | File |
If the pipeline includes a Mapping transformation, the command also outputs:
- A file that lists the unique field values for columns assigned the transformation but not transformed by a mapping rule
- A numeric count of the unmapped fields
If the pipeline includes a Sum if transformation, the command also outputs the numeric result of that transformation.
Exit codes
| Code | Type | Detail |
|---|---|---|
| 0 | Success | Pipeline completed successfully |
| 1 | Error | Pipeline failed to run or found unmapped rows |