Batching and Looping creating multiple output files

Questions
#1

Describe the issue/error/question

I have am using an API that allows 240 requests a minute. So I have split into batches and looped the API call with a delay of 30 seconds per 100 request. Everything works great until the end of my workflow. I want to output one CSV and upload it to google drive. But because the data is coming in batches I end up with five CSV’s being uploaded. I have read every thread and the docs and cannot figure out (A) how to merge the five binary data optputs into one or (B) to have a stopgap to where everything catches up and it sends it all at once so I just have one csv. Thank you VERY much.

What is the error message (if any)?

No error message. Outputting multiple CSV’s (one for every batch run) instead of one large CSV

RunBatHere - Google Drive
RunBatHere - Google Drive1701×529 30.2 KB

Please share the workflow

Share the output returned by the last node

Information on your n8n setup

  • n8n version: 0.218.0
  • Database you’re using (default: SQLite): Default
  • Running n8n with the execution process [own(default), main]:
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: Docker
#2

You can use a code node to merge the items from all batches until your loop finishes.

Here’s a template workflow Merge multiple runs into one | n8n workflow template.

2 Likes
#3

Thank you very much FelixL. I think that will do it!

#4

Hi @wtheory,

just saw that with the new n8n Version (0.220.0) dropped a new feature to make merging the output of the “split in batches” node easier!