Workiva Support Center

In this Connected Learning Path, we will create a Chain that illustrates how to convert a JSON dataset to CSV. In this example dataset, there is an array of nested objects.
 
This path reinforces the looping concept that we learned in the JSON Data exercise as well as the Cartesian join concept in the JSON Nested Objects path. If you have not completed these paths, we strongly recommend that you complete these prior to completing this one.

Step 1: Create a Chain

• Add a new Chain
• Name the Chain: CLP | Accessing JSON Array of Nested Objects
• Create a chain variable
  • Name: cv-JSON-AllDonut
  • Value: https://cs-sftp-training-bucket.s3.amazonaws.com/cs-training/transformation-qs/all_donuts.json
• Save the Chain

Step 2: Retrieve JSON Data

Use the HTTP Connector to retrieve donut data from a web location. This dataset illustrates an example of a more complex JSON dataset with an array (multiple items) of nested objects (attributes).

• Add a GET Command from the HTTP Connector to the Start node.
• Configure the Command with the following:

• Save the Command

Step 3: Convert JSON to CSV

Use the JSON Connector to extract keys from the JSON array to identify the different donut varieties. 

For reference, below is the schema of the all-donuts JSON:

Schema

[{"id":"0001","type":"donut","name":"Cake","ppu":0.55,"batters":{"batter":[{"id":"1001","type":"Regular"},{"id":"1002","type":"Chocolate"},{"id":"1003","type":"Blueberry"},{"id":"1004","type":"Devil's Food"}]},"topping":[{"id":"5001","type":"None"},{"id":"5002","type":"Glazed"},{"id":"5005","type":"Sugar"},{"id":"5007","type":"Powdered Sugar"},{"id":"5006","type":"Chocolate with Sprinkles"},{"id":"5003","type":"Chocolate"},{"id":"5004","type":"Maple"}]},{"id":"0002","type":"donut","name":"Raised","ppu":0.55,"batters":{"batter":[{"id":"1001","type":"Regular"}]},"topping":[{"id":"5001","type":"None"},{"id":"5002","type":"Glazed"},{"id":"5005","type":"Sugar"},{"id":"5003","type":"Chocolate"},{"id":"5004","type":"Maple"}]},{"id":"0003","type":"donut","name":"Old Fashioned","ppu":0.55,"batters":{"batter":[{"id":"1001","type":"Regular"},{"id":"1002","type":"Chocolate"}]},"topping":[{"id":"5001","type":"None"},{"id":"5002","type":"Glazed"},{"id":"5003","type":"Chocolate"},{"id":"5004","type":"Maple"}]}]

• Add an Array to CSV Command from the JSON Connector to the Chain.
• Connect the Start Node (GET - All Desserts) to the Array to CSV Command.
• Name the Command: Array to CSV - Get Variety.
• In the JSON Data parameter, select the Response Output from the GET - All Desserts Command.
• Leave the Input Text, Path to root, and Filter parameters blank.
• Leave the Multi-value Delimiter parameter as a comma (,).
• Check the Preview Result option.
• Enter variety for the Column Name and .name for the JSONPath parameters.
• Select Pipe for the Delimiter parameter.
• Save the Command

Step 4: Create an Iterable Data Set

As noted, the dataset has an array (i.e., multiple items) of nested objects. To process each of the varieties individually, row numbers must be added to the dataset. These row numbers will be used in subsequent Array to CSV Commands to identify the array item (donut) for which the nested object (attributes) will be parsed.

For those familiar with Chain Builder, you may wonder why we use Advanced Query as opposed to the Add Row Numbers Command. The Add Row Numbers Command starts counting at one (1) and we need to start counting at zero (0) since JSON Arrays are zero-based.

• Add an Advanced Query Command from the Tabular Transformation Connector to the Chain.
• Connect the Array to CSV - Get Variety Command to the Advanced Query Command.
• Name the Command: Advanced Query - Add Row Numbers.
• Configure the Tables section
  • In the File field, specify the Converted File Output from the Array to CSV - Get Variety Command.
  • Enter varieties in the Table Name parameter. The query that is specified will be run against this table.

• In the Query parameter, enter the following:

SELECT

 ROW_NUMBER () OVER (
 
 ORDER BY variety
 
 )-1 RowNum,

 variety
FROM

 varieties

• Specify Pipe & Comma for the Input Delimiter and Output Delimiter parameters, respectively.
• Check the Preview results parameter.
• Save the Command.

Step 5: Create a JSON Array

Use the CSV to JSON Command from the JSON Connector to convert the unique list of varieties to a JSON array. A Command Group can then iterate over each item in this array.

• Add a CSV to JSON Command from the JSON Connector to the Chain.
• Connect the Advanced Query - Add Row Numbers Command to the CSV to JSON Command.
• Configure the Command using the following:

• Save the Command

Step 6: Add a Command Group

Add a Command Group to the Chain to enable iteration over each of the donut varieties identified in the Advanced Query step.

• Add a Command Group to the Chain.
• Connect the CSV to JSON - Varieties Command to the In section of the Command Group.
• Click the Command Group and then select the dropper icon to change the Command color to yellow.
• Name the Group: Donut Varieties.
• Enable the Iterations toggle and select the JSON File Output from the CSV to JSON - Varieties Command.
• Save the Command Group.

Step 7: Get the List of Toppings

Use the Array to CSV Command of the JSON Connector to get the list of toppings in the nested JSON array. This action is repeated for each of the donut varieties.

⚠ Please pay particular attention to the cautionary statement in the below steps indicated by the red exclamation point.

• Add an Array to CSV Command from the JSON Connector to the Chain.
• Connect the Group Start from the Donut Varieties Command Group to the Array to CSV Command.
• Name the Command: Array to CSV - Toppings.
• In the JSON Data parameter, select the Response Output from the GET - All Desserts Command.
• Leave the Input Text parameter blank.
• In the Path to root parameter, we will leverage the iteration value to specify the array item on which the iteration will operate.
  • The following steps are critical and should be read in their entirety before completing this step.
    • Click in the Path to root parameter.  From the Variable selection pane, expand the Group Iterator and click JSON File Iteration.
    • Click the green JSON File Iteration bubble in the Path to root parameter. This will open the Variable transformation form. Select Get Value from JSON and click the plus (+) sign. Type RowNum (case-sensitive) In the Value field and press the enter key. Click Accept.
    • The variable should have an asterisk indicating a Variable transformation has been applied.
      • 🛑 If any of these steps fail, delete the JSON File Iteration Variable from the Path to root and start again.
  • After the JSON File Iteration bubble, type topping (lower case) and press the enter key.
• Leave the Filter parameter blank.
• Leave the default value as comma (,), for the Multi-value Delimiter parameter.
• Check the Preview Result option.
• In the Columns section, we specify the key(s) in the JSON array for which to extract the value(s) to a column in the resulting CSV.
  • In the Column name parameter, enter topping and in the JSONPath parameter, enter .type.
• In the Delimiter parameter, select Comma.
• Save the Command.

Step 8: Get the List of Batters

Use the Array to CSV Command of the JSON Connector to get the list of batters in the nested JSON array. This action is repeated for each of the donut varieties. Since this step is very similar to the previous step, copy the Command configured in the prior step and simply update it for the batters information.

• Copy the Array to CSV - Toppings Command.
• Connect the Group Start from the Donut Varieties Command Group to the Array to CSV - Toppings (Copy) Command.
• Name the copied Command: Array to CSV - Batters.
• In the Path to root parameter, delete the topping bubble by clicking the X. Type batters (case-sensitive) and press enter. Then type batter and press enter. There should be three bubbles in the Path to root parameter.
  • JSON File Iteration *
  • batters
  • batter
• In the Columns section, change the Column Name from topping to batter.
• Save the Command

Step 9: Flatten the Data

Similar to the last exercise, use a Cartesian join in an Advanced Query Command from the Tabular Transformation Connector to flatten the dataset.

• Add an Advanced Query Command from the Tabular Transformation Connector to the Chain.
• Connect the Array to CSV - Batters and Array to CSV - Toppings Commands to the Advanced Query Command.
• Name the Command: Advanced Query - Flatten JSON Object.
• In the Tables section, click the Add button once so that there are two available tables. Complete the Tables section per the below:

• In the Query parameter, enter the below query:

Select '<JSON FILE ITERATION>' as Variety,Batter,Topping from Batter,Topping

• • <JSON FILE ITERATION> should be selected and replaced by expanding Group Iterator from the Variable pane and selecting JSON File Iteration.
  • After the JSON File Iteration Variable is selected, click the green bubble, this will open the Variable transformation form.
    • Apply the Get Value from JSON Variable transformation using variety for the Value field. 
    • Be sure to press enter after typeing variety and Accept the transformation!
• Specify Comma for the Input Delimiter and Output Delimiter parameters.
• Check the Preview results option.
• Save the Command.

Step 9: Test the Exercise

• Publish the Chain.
• Click Execute and then select Run Chain.
• Once the Chain has completed,
  • Click the Advanced Query - Flatten JSON Object node and select the Outputs tab. Confirm the Record Count for each iteration:
    • Iteration 1: 28 records
    • Iteration 2: 5 records
    • Iteration 3: 8 records
  • Click the Preview Results (eye icon) Confirm the data matches for each iteration.

To learn more about data transformation using Chains, check out the Connected Learning Paths - Transformation Introduction!