What is the best pattern for triggering N sub-workflows in parallel and resuming main workflow when all complete

## Describe the problem/error/question

I need to trigger a dynamic number of sub-workflows in parallel (around 100)

and wait for ALL of them to complete before continuing the main workflow.

I’ve implemented a solution but I’m wondering if there’s a better or more elegant approach.

**My Current Implementation:**

**Main Workflow:**

1. Triggers N sub-workflows via HTTP Request node with `Execute Once: false`

2. Passes the total count of sub-workflows (e.g., 100) to each sub-workflow

3. Uses a Wait node (Wait for Webhook Resume) to pause execution

**Each Sub-Workflow:**

1. Performs its work (calls external LLM via MCP, takes 2-5 minutes)

2. When complete, sends an HTTP callback to a “Collector Workflow”

**Collector Workflow:**

1. Receives callbacks from completed sub-workflows

2. Maintains a counter of completed tasks

3. When counter reaches N/N (all sub-workflows finished), it resumes the main workflow via webhook

**My Questions:**

- Is this a good pattern for handling 100+ parallel sub-workflows?

- Is there a built-in n8n feature or node that can handle this more elegantly?

- Should I be using the Merge node differently, or is the callback + counter approach appropriate?

- Are there any performance considerations or limitations I should be aware of?

**Key Requirements:**

- Dynamic number of sub-workflows (not fixed)

- Each sub-workflow is long-running (2-5 minutes) use callback solution

- Main workflow must wait for ALL sub-workflows to complete

- Need to collect and process all results together

I’m open to completely different approaches if there’s a better way to architect this!

**Main Workflow Structure:**

**Sub-Workflow Structure:**

when callback return from the long process → call to the Collector workflow

**Collector Workflow Structure:**

The final output is a collection of all sub-workflow results that need to be merged and processed together in the main workflow.

Anyone can advice please?

**My Questions:**

- Is this a good pattern for handling 100+ parallel sub-workflows?

- Is there a built-in n8n feature or node that can handle this more elegantly?

- Should I be using the Merge node differently, or is the callback + counter approach appropriate?

- Are there any performance considerations or limitations I should be aware of?

Hey @michalbd !

I think that this is the ‘recommended’ way, since it follows the “fan‑out + wait‑for‑all + fan‑in” with many long‑running sub‑workflows…(async HTTP + callback + counter + resume).

As far I am aware… not yet!

Merge is great when all branches are inside the same execution and finish in the same runn ,it cann t “wait” on separately triggered async workflows or webhooks arriving over time….

Hhmmm… set up n8n to alllow multiple concurirent executions (queue mode with workers if needed)…

Here a similar post:

Cheers!

1 Like

thank you, will do it!

I have a solution that provides true parallel execution of a dynamic number of subflows but I’m thinking about selling it..

Hey @smartflowsagency , you could have answered with an offer or some hint… not a simply “attempt“ to sell a solution…

BTW is your solution using a single main n8n instance?

It requires subflows to execute.

The main process spawns the subflow for the parallelism, that subflow calls your subflows to execute and then data is collected in the parallelizing subflow, then finaly returns all data to the root flow.

Not at all sure what I would charge. Once it’s in the wild it will be hard to charge so..

Dude… no offense, but how exactly do you achieve “true parallelism” here… honestly I don’t get it myself lol.

I know that “parallel” executions occurs with queue mode(that actually are inline since I use Redis to manage them)… OR using different n8n main instances.

Enlighten me please.

Cheers!

It’s similar to what OP proposed but more robust and works on a dynamic list of workflows.

You seem to understand what parallel execution meant when the OP was discussing so I’m not sure what the disconnect is.

From a main workflow, you call this workflow and it will run any number of other subflows without waiting for the workflows to finish serially.

As far a whether it truly runs in parallel, this is somehow up to the hardware you are running and whether you are running que mode.

Ohh… got your point.

One workflow that triggers ‘n’ sub-workflows and the main workflow completes without waiting for sub-workflows… and pipe all data back(or to another workflow) when all sub-workflows are finished…

Yes, that would be “parallel” , but in background actually they are executing inline if not handled as in my previous post.

Yep, coffee time…i am kind of over thinking.

Cheers! And out of topic. :slight_smile: