i have published my automation. the trigger is a webhook node, when a csv (1000 leads) is sent to that webhook the automation starts, but the autoamtion is forced killed by n8n, the 1st time it ran for 3 hours (processed 260 leads) before getting forced killed and on next try it ran for 6 hours [started from begining and processed 500 leads] before getting force killed:
i am trying a 3rd time with differnt settigns now which are:
please let me know if anyone knows the likely cause and solution
Hey @AbdullahShah, while you wait for a response, here are some things that might help:
Automatically matched to your question.
Docs:
Forum:
@alexm, @Mark_Tic, @Niffzy - you’ve helped with similar issues before, can you take a look?
Automatically suggested by n8n’s community bot. It’s a pilot - please share feedback here.
Two things you can rule in or out straight from your own screenshots.
It is not the workflow timeout. Your settings screenshot shows “Timeout Workflow” toggled off, so n8n is not ending the run on a time limit. The shape is wrong for a timeout anyway — the two runs died at 3h15m and 6h17m, not at the same point.
“Save execution progress” is switched on. That makes n8n write the execution state after every node. Across a run that loops 1000 leads that is a very large number of writes accumulating against one execution record, and it is a common contributor to a long run growing until the process is killed rather than a node failing. That distinction matches your symptom: an execution that ends with no error on any node usually means the process died, not that something threw. Turning that setting off is the cheapest thing to test, and the only thing you give up is resuming a failed run from the middle.
Worth noting your throughput was steady both times — 260 leads in 3h15m and 500 in 6h17m is roughly 45 seconds per lead either way. Nothing degraded before the end. It stopped rather than slowed, which again points at the process being killed.
One thing that is easy to miss, and it has already happened. Your second run started from the beginning and reached 500. The first had already reached 260. If those nodes send email, the first 260 leads got the same cold email twice. Nothing in n8n flags that — the retry looks clean and duplicate sends do not show up as errors anywhere. Before the next attempt it is worth writing each lead to a sheet or table the moment it is sent, and checking that list before sending, so a restart resumes instead of repeating. On cold outreach the double-send usually costs more than the failed run did.
Disclosure: I used Claude to help work through this. The two settings observations are read directly off the screenshots you posted. The memory explanation is the most likely cause given the symptom, not something verified against your instance, and I did not check it against the docs while writing. Happy to be corrected if your setup rules it out.
both failures happened before I turned “save execution progress” on, that setting can’t have caused runs #1 and #2. It might still make things slightly worse now, but it’s not what killed the original two runs
Your workflow is almost certainly experiencing an Out of Memory (OOM) crash because n8n is trying to hold the execution data for all 1000 leads across every node in RAM simultaneously over a multi-hour run.
You have Save execution progress set to Save.
The Fix: Change this to Default - DO NOT SAVE.
The Reason: When this is enabled, n8n writes the state of every single node to the database continuously during the run. For a 3-to-6 hour execution, this creates a massive bottleneck, bloating both RAM and your PostgreSQL/SQLite database until the Node.js process crashes or the container is killed by the host.
Running 1000 heavy iterations in a single workflow will eventually exhaust Node.js heap memory because the garbage collector cannot clear the data until the entire parent execution finishes.
The Fix: Use an Execute Workflow node to pass the leads to a secondary workflow in smaller batches.
The Reason: When a sub-workflow completes, n8n flushes its execution data from memory, keeping the parent workflow’s RAM footprint small.
You’re right, and my post was wrong on that. Your first post says the third run is the one with the new settings, so save execution progress was not on for runs #1 and #2 and cannot explain them. I read that screenshot as your current state rather than as your changed state. Disregard that part.
What still holds, from your own numbers:
The workflow timeout is still ruled out. Timeout Workflow is off, and the two runs ended at 3h15m and 6h17m rather than at the same mark.
Throughput was steady right to the end both times. 260 leads in 3h15m and 500 in 6h17m is about 45 seconds per lead in both runs. It stopped rather than slowed down first, which is the shape of a process being killed rather than a node throwing or the run degrading.
Separate from the cause, one thing worth checking: run #2 restarted from the beginning and reached 500, while run #1 had already reached 260. If the emails go out inside that loop, roughly the first 260 leads were contacted twice, and nothing flags it because the second run looks clean on its own. A check against an already-contacted field before the send step makes a restart safe to repeat.
Disclosure: I use Claude to help draft these. The timeout point and the throughput arithmetic are read off your screenshots and your stated numbers. I have not checked n8n’s docs or your instance, and I got the settings timeline wrong above, so weigh accordingly.
Both @kjooleng and @ClearStack hit the nail on the head regarding the OOM crash and the danger of duplicate email sends.
When you process 1000 items in a single linear execution or loop, Node.js accumulates all node input/output states in memory. Garbage Collection cannot clean up objects until the entire parent workflow execution completes, leading to host OOM kills.
Here is the 3-step production pattern to solve both the memory leak and the duplicate email risk:
### 1. The Sub-Workflow Batching Pattern (Solves OOM)
Instead of running all 1000 leads in one loop in the parent workflow:
1. Use a **Split In Batches Node** (batch size: 50-100 items).
2. Pass each batch to a secondary worker workflow using the **Execute Workflow Node**.
3. **Why this works:** When each sub-workflow execution finishes, n8n immediately flushes its RAM footprint and triggers the Node.js Garbage Collector, keeping memory usage flat over a 6-hour run.
### 2. Idempotency Check (Prevents Duplicate Email Spam)
As @ClearStack pointed out, if run #1 fails at lead 260 and you restart, leads 1-260 get emailed twice.
* Inside your sub-workflow, right before the Email Send Node, add an **If / Filter Node** that checks `already_contacted === true` (or queries your DB for `email_sent_at IS NOT NULL`).
* If `true` → Skip.
* If `false` → Send email and immediately update the DB status to `contacted`.
### 3. Increase Node.js Heap Limit (If Self-Hosted / Docker)
If you are self-hosting n8n via Docker or PM2, the default Node.js heap memory limit is around 2GB. For long-running batch jobs, increase it in your environment variables:
```bash
NODE_OPTIONS=“–max-old-space-size=4096”
```
*(This allocates up to 4GB RAM to the n8n process).*
Combining **Sub-workflows + Batching + Pre-send Idempotency Checks** is the standard way to run 10k+ lead pipelines without crashing n8n or spamming users.
Hi @AbdullahShah
Going by the Upgrade badges in your settings screenshot you’re on Cloud, where the ceiling is far lower than a default Node process: Trial and Starter get 320MiB, Pro-1 640MiB, Pro-2 1280MiB, and n8n itself takes around 180MiB of that before your run starts. One execution that stays open for six hours across 1000 leads will not fit in what’s left.
Take the whole list out of a single execution. Have the webhook write the 1000 rows into a sheet or table with a status column and finish there, then drive the sending from a Schedule Trigger that pulls the first 25 rows where status is empty, does the work, and writes back sent. Each execution lasts a couple of minutes and releases its memory when it ends, and a crash costs one batch rather than the run.
If you’re weighing whether a bigger plan buys you enough headroom instead, this breaks the tiers down:
ok im now diving my workflow into a sub workflow to see if this resolves the OOM issue, can someone clarify one thing, do i need to publish the subworkflow as well alongside the parent workflow?
my sub workflow:
Sub1 Non-DNC(1).json (526.6 KB)
dont worry about outreaching the same lead twice , as i am integratign this workflow with instantly Ai at the end so instantly Ai automatically never keeps duplicates
Yes, both parent and sub workflow must be published
Just wanted to close the loop on this.
The original workflow was getting force-killed after 3–6 hours with no node error because it kept the entire 1000-lead execution in memory the whole time (classic OOM on n8n Cloud).
I’ve now split it: the parent only does webhook → CSV extract → Google Sheet → DNC filter → Split In Batches (size 1) → Execute Workflow. All the heavy work (Perplexity research + 5× GPT-4 scoring + email generation + Instantly push) lives in two published sub-workflows that run one lead at a time and release memory when each finishes.
I’m currently testing with 500 leads.
Question for the experienced people here: with this architecture, can I safely push a single CSV of 5 000–6 000 leads through the webhook, or should I still break the list into smaller files (e.g. 1 000 at a time) even with the sub-workflow pattern? Any practical ceiling I should still respect on Cloud?
Thanks in advance.
While the child workflows are memory-efficient, the Parent workflow is not. When you use the “Read Binary File” (CSV) and “Extract from File” nodes, n8n converts that CSV into a massive JSON array in the Parent’s memory.
Hence the answer is NO
A 6,000-lead run, processing one lead at a time with heavy AI calls (Perplexity + 5x GPT-4), is going to take a long time (potentially 10–20+ hours depending on latency).
In all, 500 or 1000 leads per CSV/Webhook trigger is a safer choice. Chunking it is the way to go.
Just wanted to close the loop on this.
The original problem was that the workflow was getting force-stopped by n8n after 3–6 hours with no error on any node. It turned out to be a classic Out of Memory (OOM) issue — n8n was holding the entire processed leads data in memory for the full duration of the run.
The fix was to split the workflow into sub-workflows. Everything that came after the loop node was moved into a sub-workflow. This way the parent workflow only keeps the current processing data in memory, and the heavy work (and the already-processed leads data) is released as soon as each sub-workflow finishes.
With this change the automation can now comfortably handle CSVs of 1000–2000 leads, which is a big jump from the previous max of roughly 300–400 leads before hitting OOM.
Huge thanks to everyone in this community who helped diagnose and point me in the right direction. Really grateful for the support .. God bless you all, thank you brothers!
