Handle Failed n8n Workflows Without Losing Data

[Decoure_Ryan]

I’m working on a production n8n setup where workflows depend on external APIs and sometimes process a large amount of data.
I’m trying to design a reliable recovery process for situations where a workflow fails halfway through.
Webhook
↓
Validate
↓
Process Data
↓
External API
↓
Save Result
If the API fails at the last step, I don’t want to restart the entire workflow and repeat work that has already been completed.
I’m considering using checkpoints and a separate recovery workflow: Workflow Failure
↓
Save Failure State
↓
Retry
↓
Still Failing?
↓
Recovery / Manual Review

Describe the problem/error/question

How do you recover partially completed workflows?
Do you use checkpoints, retries, or separate recovery workflows?
How do you prevent a retry from repeating successful steps?
At what point do you stop automatic retries and require manual intervention?

What is the error message (if any)?

Please share your workflow

(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)

Share the output returned by the last node

Information on your n8n setup

• n8n version:
• Database (default: SQLite):
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app):
• Operating system:

[n8n]

Hey @Decoure_Ryan, while you wait for a response, here are some things that might help:

Suggested resources

Automatically matched to your question.

Docs:

• Error Trigger
• Handle errors gracefully

Forum:

• Design Error Handling and Recovery in Large n8n Production Workflows
• N8n Workflow Fails to Resume Safely After Partial Execution (Idempotency & Checkpointing Issue)
• Workflow Timeouts for Long-Running Processes in n8n

@Niffzy - you’ve helped with similar issues before, can you take a look?

Automatically suggested by n8n’s community bot. It’s a pilot - please share feedback here.

[achamm]

@Decoure_Ryan I’m not sure about workflow recovery, but if you have HTTP request nodes, those have settings to retry on fail, to help save the workflow! I don’t use recovery workflows, and I try to prevent errors by adding wait nodes where needed and retry when failed like on external service requests! I haven’t really run into the point of stopping automatic retries, manual intervention takes time, so I build my flows to prevent it!

However, you could build a workflow, so when that workflow fails, it starts another workflow, which just retriggers the first one, I’m not exactly sure how though.

[Emmas]

@Decoure_Ryan

A reliable approach is to make workflows retry-safe and recoverable, rather than simply restarting the entire execution.

Workflow
↓
Process Step
↓
Save Progress
↓
Next Step
↓
Failure?
↓
Retry / Resume

Always Save progress after important steps
Make external actions idempotent so retries don’t create duplicates
Retry temporary failures with backoff
Keep failed executions available for investigation
Move repeatedly failed jobs to a recovery or manual-review process

For example: {
“job_id”: “123”,
“current_step”: 4,
“status”: “failed”,
“retry_count”: 2
}

The main idea is to resume from the last successful step instead of starting everything over.

For production systems, combining checkpoints, idempotency, controlled retries, and a manual recovery path usually gives you much better reliability.

[Niffzy]

I would avoid restarting the whole workflow whenever something fails. Instead, I’d track the progress of important steps and make each step safe to retry.

Start
↓
Process Step 1 → Save
↓
Process Step 2 → Save
↓
Process Step 3 → Failed
↓
Retry Step 3

For temporary issues like API timeouts, I’d use automatic retries with backoff. If the same step keeps failing, I’d move the job into an error/recovery workflow rather than retrying indefinitely.

The important part is making each step idempotent, so retrying it doesn’t create duplicate records, messages, or API actions.

This gives you a system that can recover from failures without unnecessarily repeating work that already succeeded.

[Adam13y]

The state table plus Error Trigger pattern above is the right backbone, so this is one gap rather than a disagreement.

Everything in this thread is triggered by a failure. The Error Trigger is the entry point for the whole recovery path. That leaves one case uncovered, and it is the one that actually loses data quietly: the run that does not fail.

If a step takes a legitimate empty branch, the workflow completes. Your job record gets status = complete, last_step advances, no Error Trigger fires, nothing retries, and the state table now says the job succeeded. It did not. It processed nothing.

Concrete version from one of my own workflows on 31 July: a leftover filter in a Code node cut 8 correct rows to 0. The workflow then took its “no results” branch, which is exactly what that branch is supposed to do when there genuinely are no results, and reported success. Every node green. A resume-from-last_step design would have had nothing to resume, because as far as the state table was concerned the job was done.

Two things that close it, both small additions to what you already have.

Record counts in the job record, not only steps. Write expected_count when you fetch and actual_count when you write. “Completed with 8 of 8” and “completed with 0 of 8” are different outcomes and should not share a status. Most silent data loss shows up as a count mismatch long before anyone notices the missing records.

Reconcile on a schedule, not only at write time. A separate job that compares what the source says it holds against what the destination actually received, running independently of the pipeline. It catches the class where every individual run looked fine and the totals still drifted. Same principle as the manual-review alert above, except it fires on a wrong number rather than on an exception.

Worth being clear that this does not replace the retry architecture. It sits next to it, because retry logic answers “it broke, resume it” and this answers “it did not break and the data is still wrong.”

[anon45958619]

@Decoure_Ryan

I think one gap left . The Error Trigger only fires if n8n is alive to run it. If the container gets OOM killed or restarted mid execution, nothing fires, the job sits at status processing forever, and Adam’s count check misses it too since the row never reached complete.

Fix is a timestamp. Write started_at when you claim the job, then have the same reconciler sweep anything stuck in processing past your max runtime and requeue it.