Hey! I have a ‘main’ workflow that scrapes the web looking for competitor products (up to 20, max). It then passes each competitor to a sub-workflow to process specific details about that product. The sub-workflow takes around 60 seconds to run.
The sub-workflow is set to ‘run once for each item’. This means it can take around 20 minutes for the sub-workflow to complete. How can I configure it so the sub-workflow still runs once for each item, but it kicks off the 20 runs all at the same time, rather than waiting for one to complete before starting the next?
I tried to solve this in Claude, but it suggested ‘Split in Batches’ and branching the main workflow into 4 branches that processed 5 competitors each - it felt very hacky. I’m hoping there’s a more elegant solution out there!
The option you need is already in your node. In the Parameters tab, expand Options and toggle “Wait for Sub-Workflow Completion” to off.
With this disabled, the node fires each sub-workflow and immediately moves to the next item without waiting for the previous one to finish.
But, since the main workflow won’t wait for the results, any nodes after the Execute Sub-workflow node won’t have access to the sub-workflow output. If you need the results back in the main workflow, you’ll need to have each sub-workflow send its results to a webhook or database, and collect them separately.
Thank you! I need the results of the sub-workflow to be passed back to the main workflow…
Maybe I could fire all 20 to the sub workflow, add a wait node to the main workflow, then grab the results from a temporary DB? Is that the best solution?
Yes, that’s exactly the right pattern. But you don’t need an external DB, you can do it entirely within n8n:
Main workflow: Fire all 20 sub-workflows with “Wait for Sub-Workflow Completion” off, passing $execution.resumeUrl and a unique ID (e.g. competitor name) to each one
Main workflow: Add a Wait node right after, set to “On webhook call”
Sub-workflow: At the end, use an HTTP Request node to POST the results back to the resumeUrl you received, including the unique ID
Main workflow: After the Wait node, add a loop that checks if all 20 results have come back. If not, go back to the Wait node
There’s an official template that implements exactly this pattern with a wait-for-all loop. Worth importing it and adapting to your use case rather than building from scratch.
Great solution from @houda_ben! One important thing to keep in mind: on self-hosted n8n, the default concurrency limit for executions is 20 (configurable via N8N_CONCURRENCY_PRODUCTION_LIMIT env var). So 20 parallel sub-workflows should work fine out of the box.
However, if you’re on n8n cloud or have a lower concurrency limit set, you may need to batch them (e.g., fire 10 at a time). The $execution.resumeUrl pattern @houda_ben described is definitely the right approach for collecting results back.
One small addition: make sure to pass a timeout to your Wait node (e.g., 5 minutes) as a failsafe in case any sub-workflow crashes and never sends back the result — otherwise your main workflow could hang indefinitely.
@Adam_Warburton
in addition,
The issue is that you’re running the sub-workflow synchronously, so n8n defaults to processing items one by one. If you need true parallelism, you’d have to go async, like Houda said. If you need the results inline, it’s usually better to redesign the sub-workflow to handle the whole batch in one run instead of calling it 20 times. So it really comes down to choosing between a synchronous and an async architecture.
The key thing to understand here is that n8n’s “Execute Workflow” node (in sync mode) runs sub-workflows sequentially by default when you iterate over items. To get true parallelism, you need the async approach.
Here’s the pattern I use when I need to process many items in parallel:
In your main workflow, split your 20 items into batches
For each item, use an HTTP Request node to call your sub-workflow’s webhook URL (instead of Execute Workflow node)
Set those HTTP requests to not wait for response (fire and forget)
Each sub-workflow runs independently on its own execution thread
The downside is you lose the ability to collect results back in the main workflow easily. If you need results, you can store them in a database (Postgres, Supabase, etc.) and then poll or wait for completion.
For your specific use case of 20 competitors with 60s sub-workflows, this should bring your total from 20min down to roughly 60-90 seconds depending on your server resources.
All great solutions, thank you for everyone’s help! I think I might run into concurrency issues, so a batch then ‘fire and forget’ approach is probably best, and then I’ll add the Wait node (with timeout) and poll for results.