Hi, im making a workflow that collects websites from a list and http requests them to get info
I have about 2,000 websites, but when i run the node, my console outputs a websocket error and the execution tab says “n8n has run out of memory” so i limit it to 400 websites, doesnt work, then limit it to 10 and it works
So my question is how can i send http requests to hundreds of sites in one node as trying to do 10 or 20 manually at a time is tedious
im on the cloud version
Hi, did you try Split in batches node? Example Workflow Structure
Load URLs → Split In Batches (batch size: 10–25) → HTTP Request → optional: Wait → Next batch.
Hope this works for you. Good luck!
Hey @Joshua_Knox hope all is well. Welcome to the community.
HTTP Request node has an option called “Batching”, which allows you to set “Items per Batch” and “Batch Interval”. Try to limit number of websites per batch and set how long to wait between batches.
Hey I havent had any luck. Ive been playing with the size of the batch and the interval of ms, but no matter how small i make the batch i still get
[WebSocketClient] Connection lost, code=1006 &
403 (Forbidden)
do you have experience running this number of items (400) in an http node at once?
anyway thanks you for your help
If there’s an issue with batching, I would try another option, such as redesigning the workflow, for example, by wrapping the HTTP node inside a sub-workflow and passing the 2,000 websites to it, so it runs as 2,000 separate executions..
