[Merge node] merging multiple outputs

Describe the issue/error/question

Basically I have a workflow that gets data and sorts the data to run 4 separate loops and in the end i just want it to collect all the results and create a spreadsheet with a report on what happened for each item.

The problem I am running into is the append merge nodes, i have daisy chained 3 merge nodes like this to combine all the outputs to a single output. like this

then when usually the workflow will stop at the merge nodes if either node has a missing input.
if i have always output data it will just output empty items out and totally disregard the data.

I also use a community node which is the advanced split in batches node which really helped to simplify my workflow a tonne @Bram but sometimes it doesn’t output the done branch data (which is combined result of all the looping) perhaps its because the merge nodes skip them.

Anyway is there anyway to merge all that data without excluding any data and in the case where one of the 4 outputted data is not present, the merge should continue without it??

What is the error message (if any)?

Please share the workflow

its too big you will have to go here to retrieve it

Share the output returned by the last node

Information on your n8n setup

  • using the latest version of n8n on docker container

Here is an example where data is present from all streams but the merge node fails

Into the node that stopped and didnt output you can see the data coming in but nothing being output

So if i set it to always output data it will just output a emtpy data in its place.

updated the google drive link for public sharing im so sorry about that

Hi @Josh-Ghazi, the Merge node is a bit tricky, especially when you have a differing number of executions on each stream.

Perhaps it’s easier to process your data sequentially rather than in multiple branches which have to merged afterwards? It seems to me that the only job of your Merge nodes is to wait until everything else is done anyway. So perhaps you could do something like this to get data from different nodes after a lengthy sequential flow:

After the “preparing the result” step you’d have all the data back together and can write it to a spreadsheet, regardless of what has been previously. You can also cover cases where a previous node might not have returned any suitable result and you want to exclude an empty item etc. in the final node.

1 Like

Thank you @MutedJam, I will give this a try, this looks like exactly what ive been looking for.

1 Like

Okay ive also given this a try, I have removed all of the merge nodes from the end of the workflow, instead what I have done in use if and set nodes to sort out all of the items and process all items through the 2 loops instead of trying to separate items into one of the loops, just as you have demonstrated up above.

Instead of using the split in batches node and then a function node to carry the data forward and using function nodes to compile all the results, I have just used @BramKn 's Split in batches - ADVANCED node to run the loops and combine all the data from every loop into a done branch. Then moved all of the items to the next loops to perform the workflow after.

So my workflow is now 100% merge free and there are no more problems regarding missing data or failures to complete the workflow! I couldnt be happier

the workflow is still too big to be posted on here, but ill leave a picture instead

Kindest regards
Josh

2 Likes

Amazing, really glad to hear this works for you!

If the mentionned solution doesn’t work for you, here is a code snippet that allows you to merge any number of inputs, and then pass it to an if where you can assert on the “runs” parameters that your merge is actually over:

const staticData = $getWorkflowStaticData('node');
// set the node global variabl
if(!staticData.runs){
  staticData.runs = 0;
}

staticData.runs +=1;
const jsonObject = $input.item.json;

const result = Object.assign({}, jsonObject, staticData.currentResult);

result["runs"]= staticData.runs;
staticData.currentResult = result;

return { json: result};
1 Like