How to merge output of multiple Split In Batches runs?

[joeldeteves]

Describe the issue/error/question

I have a workflow that splits data using the Split In Batches Node + a series of SQL queries. I have the data split according to the name of the customer in my ticket database, as well as their ticket count which is exactly what I want.

Now I’m wondering how I can “break” i.e. aggregate the Split In Batches at the end so I can output the data as rows in a single spreadsheet? Ideally with headers.

What is the error message (if any)?

Please share the workflow

Share the output returned by the last node

Information on your n8n setup

• n8n version: 0.199.0
• Database you’re using (default: SQLite): Postgres
• Running n8n with the execution process [own(default), main]: own
• Running n8n via [Docker, npm, n8n.cloud, desktop app]: K8s

[BramKn]

Hi @joeldeteves

Do you mean like talked about in this topic?

Get all results after batch split node?

Hi everybody and congrats to n8n team for this amazing product. Firts post here, but I’m lost and I need help What I want to do : I need to perform some operations on data (call an external API and wait for data enrichment). To avoid errors from external api, I need to split into batches the data and call the api for each batch BUT I need to retreives all the results of all batches : If I have 100 object as inputs splited into 5 batches, I need to get 100 objects after the operation. To sim…

[joeldeteves]

Yes however I tried that method and it doesn’t work, at least not with the workflow I have here

[BramKn]

The workflow you shared does not look like the example given.
If you changed it after please use the example and if it doesn’t work share that workflow.

[pemontto]

I’m not sure I understand the workflow, I’d expect the IF node to come after the operations you split for.

Maybe something like this:

Without sample data that will probably fail, but you should get the gist of it.

[pemontto]

Or potentially just this if the rest of your workflow is working OK

[joeldeteves]

Using the modified workflow, nothing gets outputted by the Code node:

[joeldeteves]

Tried this one too - same as the other modification, nothing gets output by the code node

[joeldeteves]

I got a bit further with this - it turns out, one of the SQL queries is returning an empty result.

It works fine with my original loop but not in the one demonstrated in the second example.

I’m not sure why this is the case - have to do some more troubleshooting to figure out how to bypass empty items

[joeldeteves]

Solved!

[joeldeteves]

Here’s an even better version. This one removes the requirement for merging the SQL outputs, and properly outputs ‘0’ if the ticket count is 0 (instead of counting the output as ‘1’):

While this works well, it certainly was a lot of extra work to figure out. I hope to see the “combine” functionality get added into the main Split In Batches node in the future.