How to handle merging output from a split node with varying data types and potential non execution on path

[mxkel]

Describe the problem/error/question

I have part of a workflow that extracts data from resumes. The incoming resumes are processed in batches, some batches are all PDF, some have a mixture of PDF, DOCx and Image, some just two. I need to merge all the outputs together after they have been extracted and maintain the order.

While this works if the batch has all 3 types in it, if a batch passes through with just PDF for example then the other two paths wont execute causing the merge node to hang and not execute.

I have tried a code node, but that creates a race condition and will continue the workflow when the first item is complete.

Any idea how to handle this situation where some paths may not execute but sometimes they all do?

What is the error message (if any)?

Please share your workflow

Information on your n8n setup

• n8n version: Version 1.111.0

• Database (default: SQLite): Postgres

• n8n EXECUTIONS_PROCESS setting (default: own, main): default

• Running n8n via (Docker, npm, n8n cloud, desktop app): Docker on Railway

• Operating system: MacOS Tahoe

  

[krisn0x]

Hey, could you try to share your workflow again in a code box, as the json is broken right now. You can also export the workflow and share it through G Drive.

[mxkel]

I can’t seem to get it to show correctly, I have attached an image of the workflow.

[krisn0x]

Can you use the 3 dots (top-right) > export workflow and share through Google Drive? Would be faster as I can make the changes directly Otherwise I will have to keep requesting screenshots.

[mxkel]

I exported that part of the workflow. All i need is a way to dynamically merge the paths dependant on how many execute so one or two paths not executing doesnt hang the workflow. https://drive.google.com/file/d/16Ol7QGFJTsTO7bu1o4leESQypEdek1jd/view?usp=sharing

[krisn0x]

Hmm, could you run yours and grab a screenshot of what happens when the Merge node hangs? I tried to simulate on my end and if Detect File Types outputs only 2 file types:

.. then the merge node doesn’t have issues with continuing:

What would fail it on my end, is if one of the branches starts executing but fails, but that throws an error. Need to compare this with what happens on your end.

[mxkel]

The merge node just doesn’t execute. In this run we have 4 PDFs and 1 DOCx.

The merge node receives the 5 items, but just doesn’t continue.

I presume because it is waiting for the 3rd branch. I have sporadically had this work and the merge node output on 2 executed branches, but the majority of the time it stops. This will be a production workflow so it needs to be bulletproof.

[krisn0x]

Yeah, so far I’m inclined to think that it should continue and in your case it’s hitting some bug or some edge case I don’t understand yet. Especially after you said it continues sometimes. I will look into it a bit more and see what I can dig up.

[mxkel]

Doesn’t the merge node always wait for all inputs to execute?

[krisn0x]

That’s what I thought too! I’m confused to be honest. It could mean it waits for all branches that have started executing and doesn’t wait for any that haven’t started at all. I will try to summon @barn4k, maybe he can shed some light