hi there
i am trying to scrape through 3 different HTTPS Request as shown from the image below,
however upon trying to execute my workflow it seems like it only runs the top workflow, and never the bottom 2.
what i want to acheive
- Run the 3 HTTPS request nodes and the same time and once the output is in google sheets, i will remove the duplicated using the node
Hey @chickenrice, welcome to the community!
The issue here is that n8n by default runs nodes sequentially - even if you have 3 separate HTTPS Request branches, the trigger will execute them one after another, not in parallel.
To run all 3 at the same time, you have two options:
Option 1: Use a single trigger + merge approach
Instead of 3 separate sub-workflows, connect all 3 HTTPS Request nodes directly from the trigger. n8n will execute them in sequence but you can merge outputs with a Merge node afterward.
Option 2: True parallel execution
Use the “Execute Workflow” node (with “Wait for sub-workflow completion” turned OFF) to call separate child workflows for each scraper. This fires all 3 at the same time without waiting.
For your use case of scraping 3 lists and deduplicating, Option 1 might be simpler. Could you share a bit more about what the 3 HTTP calls return? That would help suggest the best Merge strategy.
1 Like
hi @nguyenthieutoan thanks for the explanations!
what do you mean buy connecting all 3 HTTPS nodes directly from the trigger? and how does this differ from the current set up?
My understanding is that all 3 HTTPS nodes current flow from the same trigger (as shown below), hence I am a little confused.
As to what the HTTP calls return, im not sure if this if what u mean but:
with regards to the POST node, this is what it returns:
with regards to the GET node, this is what it returns:
the aim from the scrappers is to attain the username and instagram handles from people who have either commented/ tagged/ hashtag the requirements we have set on apify
The key point is you need a Merge node at the end that accepts all 3 inputs.
Here’s how it works:
-
All 3 HTTPS Request nodes fire from the same trigger (which you already have)
-
The Merge node waits for all 3 branches to complete before passing data downstream
-
Only after all 3 return results will the Merge node output combined data to your Deduplicate node
So the flow looks like:
Trigger
├── HTTPS Request 1 (comments)
├── HTTPS Request 2 (tags)
└── HTTPS Request 3 (hashtags)
↓ (all 3 connect into)
Merge Node
↓
Deduplicate Node
↓
Google Sheets
In the Merge node, set mode to “Combine” > “Append” - this stacks all items from the 3 branches into one list, which is exactly what you want before deduplicating.
The reason only 1 node was running before is likely because the subsequent nodes weren’t properly connected to the Merge node - n8n won’t execute a branch unless something downstream is waiting for its output.
1 Like