I have a workflow that calls an external API and then updates a database. Sometimes the API succeeds, but n8n fails before saving the result, so the workflow retries and the API call happens again.
Question
What’s the best production pattern for handling this? Would you use idempotency keys, status checks, database checkpoints, or a combination of them?
Describe the problem/error/question
What is the error message (if any)?
Please share your workflow
(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)
Hi @anon20761130, use a combination, but the external API must enforce the idempotency key. A database checkpoint alone cannot cover the gap between “API succeeded” and “result saved”.
Create and persist one key per intended operation before calling the API, for example order:123:create. Reuse that key and the same payload on every retry, rather than generating a new key per execution. Check how long the provider retains keys too.
Keep a database record with a unique operation key, status and API result. Save the business update and completed status in one database transaction. Completed operations can then be skipped on retry.
If the API has no idempotency support, check the outcome using a stable reference you sent with the original request. When you cannot reliably establish whether it succeeded, leave it as unknown for reconciliation instead of automatically sending it again.
Hi @anon20761130 I would probably combine idempotency keys with a checkpoint in the database. That way, a crash doesn’t automatically mean the API gets called twice.
For example, generate a stable key for the operation:
const orderId = $json.orderId;
return {
...$json,
idempotencyKey: order-${orderId}-payment
};
Send that key with the API request. If n8n crashes after the API succeeds, the retry can check the API using that same key. If it already completed, just save the checkpoint and move on
In a production environment, the most robust architecture is a state-driven workflow. This involves starting with a unique identifier, marking the record in your database as “Processing” to prevent concurrent duplicate runs, executing the API call with the idempotency key, and finally updating the database to “Completed.” This multi-layered approach—combining unique keys, pre-execution checks, and explicit state management—creates a resilient system that can safely recover from a crash at any stage of the execution.
Everyone above has said keys plus a checkpoint, which is right. The part that actually bites is where the key comes from.
I’ve built the receiving side of this, and the most common broken pattern I see is a key derived from something that changes on retry. A uuid generated in a Set node, or $execution.id. Both look stable inside a single run and both regenerate when n8n retries, so the key is decorative and the duplicate goes through anyway. Derive it from the business object, the way Anshul’s order:123:create does, and never from anything n8n hands you at runtime.
Two other things worth checking on whatever API you’re calling.
What it returns on a duplicate. A well-behaved one replays the original response, so your retry just carries on with the result it would have got first time. Some return a 409 instead, which means you have to treat 409 as success and go fetch the record, not as an error to retry again.
How long it remembers the key. If a failed job can sit in a queue for a day before anything retries it, and the provider only dedupes for an hour, you’re back to duplicates with nothing telling you the guarantee lapsed.
@mouadkommir — “decorative key” is exactly the right term, and your three checks (key provenance, duplicate semantics, retention window) are the list I’d hand anyone. Three additions from the sending side:
First, on @kjooleng’s state flow: the “Processing” marker only works if claiming it is atomic. A check-then-update in two steps lets two concurrent retries both see “not started” and both fire. The claim has to be a single insert with a unique constraint on the operation key (or INSERT … ON CONFLICT DO NOTHING) — whoever’s insert fails is the duplicate and stops. The database has to be the bouncer, not the ledger.
Second, on @Anshul_Namdev’s “leave it as unknown for reconciliation”: worth deciding upfront what you’ll reconcile against. If you log the exact request you sent (hash of the payload, timestamp, operation key) at the moment you send it, reconciliation is a join between what you attempted and what the provider says happened. Without that record, “unknown” stays unknown, because all you have is the provider’s side of the story.
Third, the case nothing above covers: APIs with no idempotency key and no status endpoint — which is most of the expensive ones (LLM, TTS, render). There the only safe retry is a byte-identical one, deduped at your own boundary: hash the full outgoing payload, and if it exactly matches something you already sent, replay what you stored instead of calling again. It doesn’t make an unsafe API safe, but it shrinks the unknown class to genuine ambiguity (request sent, response lost, provider silent) instead of “we generated a new key and paid twice.”
@mouadkommir, since you’ve built the receiving side: across providers, how common is the 409-instead-of-replay behavior in your experience? And have you seen anyone document their retention window honestly, or is it always “contact support”?
The key and checkpoint advice above covers the API side. One thing worth checking on the n8n side is how the retry is happening, because n8n’s own retry doesn’t always re-run the API call.
If you retry a failed execution from the Executions list, n8n resumes from the node that failed and reuses the output of the earlier nodes. So if your database node failed, a manual retry won’t call the API again.
The duplicates usually come from something re-running the whole workflow:
Webhook triggers set to respond “When Last Node Finishes”. If the API call is slow, the caller times out and resends the request, and that starts a fresh execution. Setting Respond to “Immediately” (or using a Respond to Webhook node right after you’ve recorded the request) stops the sender from retrying.
Error workflows that re-trigger the main workflow from the top.
Schedule triggers that pick up the same record again because its status was never updated.
On the database node itself, turn on Retry On Fail in the node settings, so a temporary DB error retries only that node. Also set On Error to “Continue (using error output)” and route that output to a fallback that stores the raw API response. That way a successful API result is never lost just because the main write failed.
I’d use a combination, but keep the business operation lifecycle separate from the n8n execution itself.
For example, an invoice payment could move through:
Ready → Claimed → Submitted → Confirmed
with Needs reconciliation for an ambiguous outcome.
The claim should be atomic—such as INSERT … ON CONFLICT DO NOTHING or UPDATE … WHERE state = ‘Ready’ RETURNING id—so only one execution can proceed. n8n then sends a stable operation key to the payment API and records the returned evidence.
If n8n times out after submission, do not immediately mark the payment as failed or retry it with a new key. Move it to Needs reconciliation, check the provider using the same operation reference, and then choose the next valid transition. High-value or unresolved cases can pause for human review.
I’ve used Inistate for this durable state, audit, and review layer alongside n8n, although the same pattern can be implemented with another state-machine approach. It complements rather than replaces atomic claims and API-level idempotency.