HTTP Request node hangs indefinitely during web scraping – timeout not applied

senda · September 9, 2025, 5:32pm

Describe the problem/error/question

Hello everyone,

I’m having an issue with the “HTTP Request” node for web scraping. I’m running n8n on a server with the latest self-hosted version. After around 15 executions, the node starts running indefinitely. The timeout doesn’t seem to be applied, and as a result, it blocks the proper execution of the rest of my workflow. I’ve tried several solutions, but none of them worked.

Thanks in advance for any help or suggestions!

Please share your workflow

Share the output returned by the last node

Information on your n8n setup

n8n version: last
Database (default: SQLite):
n8n EXECUTIONS_PROCESS setting (default: own, main):
**Running n8n via (Docker, npm, n8n cloud, desktop app):**docker
**Operating system:**linux

jabbson · September 9, 2025, 5:36pm

Hey @senda hope all is good.

Which URL does the workflow try to process when it hangs?

senda · September 9, 2025, 5:39pm

hey @jabbson,
this url: “Talented People Group, le Recruteur Expert - L'expert en recrutement”

jabbson · September 9, 2025, 5:41pm

Doesn’t look like it does in this short test:

Does it work for you?

senda · September 9, 2025, 5:46pm

Not individually, everything works fine. The problem occurs after about 15 loops, and then it runs indefinitely. and it happens with many different URLs; this is just the last example where it crashed

jabbson · September 9, 2025, 5:48pm

Can you provide the list of pages to run sequentially to trigger the issue?
Or is it the list you mentioned earlier? If so, the list has bad urls and images, is that expected?

senda · September 9, 2025, 6:01pm

The list that triggered it contains between 250 and 300 links, and it’s not always the same one that causes the error.

jabbson · September 9, 2025, 6:04pm

Does this mean that you have many links with images? Do you need to download all these images?

What I would try:

[more probable] exclude binaries from the list (like links with images)
[less probable] try to space out the requests in time, to make sure you are not hitting some sort of rate limiting.

senda · September 10, 2025, 9:56am

I just tested my workflow with only the bad URLs and only the image URLs, and it worked perfectly fine.
Could the problem be related to cache or memory?

jabbson · September 10, 2025, 2:21pm

I assume the issue is due to the system getting overwhelmed by the images stored in memory.

If me answer helped you, kindly consider marking it as Solution. Thank you.

Cheers!

senda · September 10, 2025, 2:35pm

how can i delete the memory?

Entrepositive · September 10, 2025, 2:45pm

Depending on your circumstance and the solution you need, you might need to rethink this. It seems the problem you’re faced with is literally overloaded memory. Just like RAM in a computer system, or short term memory in our human brains, we need to intelligently re-sort, organize and prioritize all the bits for longer-term usage. (Memory Manager Node)

You might consider adding a knowledge base/RAG tool and iteratively store the data you need to the knowledge base to reduce the burden on the memory.

EP

jabbson · September 10, 2025, 2:48pm

You don’t need to, most temporary things in memory are auto-cleaned, when not needed anymore.

senda · September 10, 2025, 4:18pm

Thanks! I’ll check that!

system · December 9, 2025, 4:18pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.