Stop duplicate emails and records when an n8n workflow retries or resumes

Tips & Tricks
1

A common production problem: a run fails halfway, you retry it, and the customer gets the same email twice or the CRM gets a duplicate record. Retries are good, but only if every step is safe to run again. Here’s a simple pattern that makes them safe.

1. Give every item a stable key
Build the key from the business data, not from the execution. For example: lead email + form ID, or order ID + action name (“order_1042_welcome_email”). A retried or resumed run produces the same key.

2. Check before you act
Before any side effect (send email, create record, post a message, charge a card), look the key up in a small log table: an n8n Data Table, Postgres, Google Sheets or Redis all work. If it’s already marked done, skip it with an IF node.

3. Mark done right after the action succeeds
Order matters: action first, then write the log. If you write the log first and the action fails, the item is skipped forever without anyone noticing.

4. Prefer upserts over creates
Most CRMs and databases support “create or update” by an external ID. Use it whenever you can, and duplicate records mostly disappear on their own.

5. Treat errors differently

  • Timeouts, 429 and 5xx = recoverable. Use Retry On Fail with a wait between tries, or a loop with backoff.
  • Validation errors = not recoverable. No retry; route the item to a “needs review” list.
  • Anything unexpected = an Error Trigger workflow that alerts you with the execution link.

6. Make re-runs boring
With steps 1 to 3 in place, you can re-run a failed execution from the start. Steps already logged as done are skipped, so you never have to work out where it stopped.

I use the same idea in a reporting workflow: access is re-checked right before sending, each report gets a single send attempt, and every delivery is logged so it can be audited later.

How do you handle this in your workflows? Do you keep the log in a Data Table, or in an external database?

1 個讚
2

Great explanation! I really like the point that retries are only reliable when each step is designed to be safe to execute again.

One important detail I would add is handling the possibility of a failure between the actual action and the log update. For example, an email might be successfully sent, but the workflow could fail before marking the idempotency key as completed. On retry, the email could still be sent twice.

For critical operations, I would consider using an external ID with an upsert operation wherever supported, along with proper status tracking such as Pending, Processing, Completed, and Failed.

I also think n8n Data Tables are a convenient starting point for smaller workflows, while PostgreSQL or Redis can be a better choice when concurrency, scalability, and stronger consistency become important.

Thanks for sharing this pattern. Idempotency is definitely one of the key principles of building reliable production workflows.

3

Curious how you pick the key when the business ID isn’t stable yet (first touch before a CRM id exists). Do you mint one at the start of the run and carry it, or rebuild from email + form id each time?

4

Solid write-up - we hit this exact shape in production, and step 3 has one gap worth closing.

Action first, log after is correct, but between those two steps the action can succeed and the log write can fail (or the container dies). On retry the log says nothing happened, so it sends again. Two ways out:

  1. Two-phase row. Write the key BEFORE the action with status=attempt. Run the action. Update the same row to done. On retry: done means skip; attempt means do not send - alert a human, because from inside n8n you genuinely cannot tell whether the provider accepted it. For email and payments that is the only honest state to be in, and it costs one alert a week, not one a day.

  2. Push idempotency to the provider when it exists. Payment APIs and some mail providers take an Idempotency-Key header, so the retry is safe by contract and the local log becomes reporting rather than safety.

Two n8n-specific traps I would add: Retry On Fail re-runs the whole node, so the key check has to live inside the same sub-workflow that the retry wraps, not in the parent. And when the send sits in an Execute Workflow child, keep Wait For Completion on - a fire-and-forget child plus a parent retry is the most common way to double-send that I see.

5

Thanks all, these are great additions.

Sardar1 and Max001, you’re both right about the gap in step 3. “Action first, log after” still leaves a window where the action succeeds and the log write fails. The two-phase row is the cleaner version: write the key as “attempt” before the action, update it to “done” after, and on retry treat a leftover “attempt” as “don’t resend, alert a human”. For email and payments that’s the honest answer, because n8n can’t know whether the provider actually accepted the request. And where the provider supports an Idempotency-Key header, that’s even better, since the local log then becomes reporting rather than the safety net.

Nachiket, for first touch I’d build the key from data that’s already in the submission (normalized email + form ID, plus the form tool’s own submission ID if it gives you one) rather than minting a random ID at the start. A minted UUID only helps within one run. If the webhook fires twice, you get two different UUIDs and the duplicate slips through. Once the CRM record exists, store its ID against that key and use it for every later step.