Classic file types are no longer available for use as of January 2021. You can transition your classic files or download a PDF. Learn More

Array to CSV - Performance

Answered
Request Demo

I have JSON data from an API call which contains about 500k records and 90+ fields and I am using the Array to CSV command to convert the data to CSV before importing to a Wdata table.  The Array to CSV command currently takes about 7-8 minutes to complete.  Is this performance normal for a JSON with this size?  Thanks!

0

Comments

1 comment

Date Votes
  • Jeff Hickey

    Hi Mark. The size of data can impact performance. There are several other factors that also influence run time. For instance:

    • Large datasets naturally take longer to process
    • If you're using GroundRunner, server load may further increase run time

    To reduce run time, you can consider the following:

    • Schedule the Chain to run overnight or during periods of low usage. This ensures the data is ready when needed
    • Reduce the data size by transforming the data to remove unnecessary columns and rows
    • Reduce the data size by only loading new and updated data instead of reloading previously loaded data

    These are just examples, but hopefully they give you a good starting point on areas that you may be able to trim down overall run time.

    0

Please sign in to leave a comment.