Basically I have a workflow that gets data and sorts the data to run 4 separate loops and in the end i just want it to collect all the results and create a spreadsheet with a report on what happened for each item.
The problem I am running into is the append merge nodes, i have daisy chained 3 merge nodes like this to combine all the outputs to a single output. like this
then when usually the workflow will stop at the merge nodes if either node has a missing input.
if i have always output data it will just output empty items out and totally disregard the data.
I also use a community node which is the advanced split in batches node which really helped to simplify my workflow a tonne @Bram but sometimes it doesn’t output the done branch data (which is combined result of all the looping) perhaps its because the merge nodes skip them.
Anyway is there anyway to merge all that data without excluding any data and in the case where one of the 4 outputted data is not present, the merge should continue without it??
Hi @Josh-Ghazi, the Merge node is a bit tricky, especially when you have a differing number of executions on each stream.
Perhaps it’s easier to process your data sequentially rather than in multiple branches which have to merged afterwards? It seems to me that the only job of your Merge nodes is to wait until everything else is done anyway. So perhaps you could do something like this to get data from different nodes after a lengthy sequential flow:
After the “preparing the result” step you’d have all the data back together and can write it to a spreadsheet, regardless of what has been previously. You can also cover cases where a previous node might not have returned any suitable result and you want to exclude an empty item etc. in the final node.
Okay ive also given this a try, I have removed all of the merge nodes from the end of the workflow, instead what I have done in use if and set nodes to sort out all of the items and process all items through the 2 loops instead of trying to separate items into one of the loops, just as you have demonstrated up above.
Instead of using the split in batches node and then a function node to carry the data forward and using function nodes to compile all the results, I have just used @BramKn 's Split in batches - ADVANCED node to run the loops and combine all the data from every loop into a done branch. Then moved all of the items to the next loops to perform the workflow after.
So my workflow is now 100% merge free and there are no more problems regarding missing data or failures to complete the workflow! I couldnt be happier
the workflow is still too big to be posted on here, but ill leave a picture instead
If the mentionned solution doesn’t work for you, here is a code snippet that allows you to merge any number of inputs, and then pass it to an if where you can assert on the “runs” parameters that your merge is actually over:
const staticData = $getWorkflowStaticData('node');
// set the node global variabl
if(!staticData.runs){
staticData.runs = 0;
}
staticData.runs +=1;
const jsonObject = $input.item.json;
const result = Object.assign({}, jsonObject, staticData.currentResult);
result["runs"]= staticData.runs;
staticData.currentResult = result;
return { json: result};
Even if old thread, I reply here in case it helps someone in my situation. I was trying to use this solution, but the code wasn’t working and returning an empty output.
In my case the all() was the problem. Instead of all(0,0) I had to use all().
Here’s my node to merge 3 sequential branches:
Although honestly I think this function should be added to the merge node, giving it an option to “Wait for both inputs” even in Append mode.
I’m facing the same issue. I have an IF node that goes into two parallel paths A and B, which then get appended back into the same set at a Merge node. The problem is the Merge node fails whenever the IF node does not produce outputs for one of the paths.
For example, say the flow processes two items i1 and i2. If i1 goes through A and i2 goes through B, then it works. However, if both items go through A, the merge fails because there’s no output from B, and vice versa. The Merge node should have the option to be configured to handle this better. Otherwise it’s pretty useless in this scenario.
I tried merging in code, as shown by @MutedJam , but this also fails quite often because the Code node is somehow not waiting for both A and B to finish. So when $('B').all(0,0) runs, it fails with something like Cannot read length of undefined.
The only viable workaround I found was to split the workflow in two. The first part includes the IF and paths A and B. Then A ends in a Call Merge workflow 1 node, and B ends in Call Merge workflow 2 node. So there’s a duplicated node to call the second part of the workflow, but it’s better than having to duplicate the whole second part just because the Merge doesn’t work.
I also considered to introduce something like a node to wait for both A and B to finish, but it felt hacky.
