Your n8n workflow runs twice and nobody notices: idempotency patterns that fix it

Built with n8n
1

Most n8n bugs are loud. The dangerous ones are silent, and the worst of them is this: the workflow runs twice and both runs do the same side effect.

You find out weeks later, when a customer got two onboarding emails, an invoice was charged twice, or one lead landed in the CRM three times. By then the execution logs are gone.

Here is why it happens, and the patterns that actually stop it.

Why a workflow runs twice

  • Retry On Fail. If a node retries after a timeout, the request may have already succeeded on the other side. The retry sends it again.
  • Webhook re-delivery. Many providers (Stripe, Shopify, GitHub, Twilio) resend a webhook if they do not get a fast 2xx. Your workflow fires once per delivery, not once per event.
  • Overlapping schedules. A run that takes 6 minutes on a 5-minute schedule means two runs overlap and both see the same rows.
  • Manual reruns / partial re-execution. You fix one node and re-run from there, but the earlier nodes already did their work.
  • Multiple workers in queue mode picking up the same job boundary.

The one rule

Idempotency means: running the workflow twice has the same effect as running it once.

You get it by deriving a stable key from the input, and refusing to do the side effect twice for the same key.

The key must be a property of the event, not of the execution. Good sources:

  • an event / order / invoice / message id already in the payload
  • email + campaign id
  • a hash of the meaningful fields (when the event has no id)

Do NOT build the key from things that change on every run: $execution.id, $now, a random uuid, or the item index. Those make every run look brand new, so nothing is ever deduped.

Pattern 1: check, act, then record the key

Keep processed keys somewhere durable (Postgres, Redis, Airtable, Google Sheets, a Processed table).

Flow:

  1. Build key (Code node).
  2. Look the key up in the store.
  3. IF found → stop here (or log a duplicate and exit).
  4. Otherwise do the side effect.
  5. Write the key to the store.

Code node to build the key:

return $input.all().map((item) => {
  const d = item.json;
  const raw = [d.event_id, d.order_id, d.email].filter(Boolean).join('|');
  return { json: { ...d, idem_key: raw.toLowerCase() } };
});

Pattern 2: claim first, then commit

Pattern 1 has a gap: if the side effect succeeds but the workflow dies before the key is written, the retry does it again.

Fix: write the key first with status = pending, then do the side effect, then update status = done.

On a retry:

  • key missing → proceed
  • key = done → skip
  • key = pending → the previous run died mid-flight. Look the effect up on the provider by your key instead of blindly resending.

This single state field turns a silent double-send into a recoverable check.

Pattern 3: let the provider dedupe

Many APIs accept an idempotency key so they do the deduping:

  • send the header Idempotency-Key: on the HTTP Request node (Stripe and many others support this)
  • or use an upsert / unique constraint instead of a plain insert

For databases, INSERT … ON CONFLICT DO NOTHING on a unique idem_key column is usually enough.

This is the safest option, because the dedupe happens on the side that actually knows the truth.

The 2-minute test

Pick one workflow with a side effect. Send the same input twice (re-run the last execution, or replay the trigger). If the recipient sees two messages, you do not have idempotency yet.

Common mistakes

  • The key contains $now or $execution.id, so every run looks new.
  • Keys are stored only in workflow static data or memory, and are lost on restart.
  • Retry On Fail is assumed to be safe for non-GET requests. It is not.
  • The key is written after the side effect with no pending state, so the crash window stays open.

Which side effect in your workflows is still not idempotent? For me the scary one is always whatever sends money or email.

2

One addition to Pattern 2: I’d make the initial claim atomic, rather than a separate lookup followed by an insert. Otherwise two simultaneous deliveries could both see “key missing” and proceed.

With Postgres, I’d use a unique key and INSERT … ON CONFLICT DO NOTHING RETURNING …, then let only the execution that inserted the claim perform the side effect.

I’d also extend the test: send the same event concurrently, then simulate a crash after the provider succeeds but before done is recorded. How would you reconcile that pending state if the provider doesn’t support lookup by your key?

3

Good call on making the claim atomic - the separate lookup-then-insert is exactly where two simultaneous webhook deliveries both read “not seen yet” and both proceed. The lookup and the claim have to be one statement.

n8n-native version of your Postgres idea, in a single Postgres node (Execute Query):

INSERT INTO idem (idem_key, status, claimed_at)
VALUES ($1, ‘pending’, now())
ON CONFLICT (idem_key) DO NOTHING
RETURNING idem_key;

The run that inserts gets a row; the loser gets zero rows, and n8n does not execute downstream nodes on an empty branch - so only the winner reaches the side-effect node. No IF needed, the SQL is the gate.

Then after the side effect:

UPDATE idem SET status=‘done’, done_at=now() WHERE idem_key=$1 AND status=‘pending’;

Keep the “AND status=‘pending’” so a late duplicate cannot flip state backward.

Now the hard part you asked about: reconciling pending when the provider offers no lookup by your key.

  1. Best case: make the pending state rare by putting idempotency on the write side. If the provider accepts a key on write (Stripe’s Idempotency-Key header, or a metadata / reference / external_ref field, or an upsert on a unique natural key), then a blind retry is safe and you never need to read back. That is the only true exactly-once path.

  2. If you can carry your key in the payload, use it for reconciliation. Write idem_key into some reference / note / source field of the created record. Then even when the provider cannot “look up by idempotency key”, you can query its own records for reference == your key. Reconcile = look it up within a time window; found → mark done; not found → safe to retry.

  3. If the provider supports neither (no write key, no reference field, no read by key), exactly-once is impossible - do not blind-retry. Degrade to at-least-once plus review:

    • a scheduled sweeper finds status=‘pending’ AND claimed_at < now() - interval ‘10 minutes’
    • for each, try the natural-key lookup (email + template + time window). Found → done. Not found → you cannot know whether the earlier attempt committed. Move it to status=‘review’ and alert; do not resend.
    • a human, or a second workflow, decides. A silent resend here is exactly how the double email gets created.

Two design notes:

  • keep three states, not two: pending / done / review. pending should be short-lived; a long-lived pending is the thing that should page someone.
  • make the mark-done monotonic (only pending → done), so a duplicated mark-done cannot corrupt state.

On your test idea: yes, and two additions. (a) fire two identical deliveries concurrently and assert exactly one row inserted and exactly one side effect - that tests the claim. (b) inject a failure between the side effect and the mark-done (a Stop / throw node), then run the sweeper and assert it lands in review rather than a resend. The second one is the test that actually finds the crash-window bug.

For the no-lookup case, what is the real provider you are hitting? For me it is generic SMTP sends and a couple of “create lead” endpoints that accept no reference field - there the only safe move is review.

4

Thanks for expanding on this. I wasn’t referring to a specific provider incident; I was asking about the failure boundary where the external action may have completed but the workflow can’t confirm it.

The review state answers that well: uncertainty stays visible rather than becoming another automatic send.

One distinction I’d keep explicit: carrying a key in metadata helps reconciliation, but doesn’t necessarily make the write idempotent. That depends on whether the provider actually enforces uniqueness.

Your SMTP example makes the trade-off concrete. I’d rather surface an uncertain send for review than quietly risk sending it twice.

5

Good distinction, and it is worth stating as a rule: a key carried in metadata is a reconciliation handle, not an idempotency guarantee. Idempotency is a property of the receiving system - the payload only tells the receiver which logical operation it is.

That splits providers into two classes, and I would design differently for each:

  1. Providers that enforce uniqueness server-side (Stripe Idempotency-Key, a DB unique constraint / ON CONFLICT, order APIs with a client-supplied reference). Here a blind retry is safe and you really do get exactly-once for the effect.

  2. Providers where the key is advisory (SMTP, generic webhooks, most append-style APIs). Nothing stops a second execution, so exactly-once is not achievable from the caller side. The honest target is at-least-once plus visibility.

For class 2, surfacing the uncertain send is the entire design. I write the marker before the side effect, so the states are pending (claimed, effect not confirmed), done (confirmed), review (claimed long ago, still unconfirmed). Only review needs a human, and the review record should carry the reconciliation handle plus the execution id so a person can look it up provider-side instead of guessing.

The uncomfortable part: for class 2 the workflow can never distinguish ‘committed but unconfirmed’ from ‘not committed’. It can only refuse to auto-repeat. So I would rather page a human on an uncertain outbound email than retry it silently - the same trade-off you described.

Curious about your setup: when the provider has its own reference field, do you put the handle there, or keep a local ledger keyed by your own id?

6

I’d use both. Keep a local operation record with a stable operation ID, and put that ID in the provider’s reference field when supported. Store the returned provider ID locally once confirmed.

The provider reference helps reconciliation. The local record tracks attempts, uncertain outcomes, and review decisions across executions. Neither makes the write idempotent unless the provider enforces it.

When an uncertain operation waits for reconciliation or human review, it becomes a long-running job. I’d give that review an owner and a deadline, then record the evidence and decision used to resolve it or approve a resend.

One qualification on retries: provider guarantees have limits. Stripe API v1 can prune idempotency keys after at least 24 hours; reusing a pruned key creates a new request. An empty provider lookup can also be inconclusive, so I wouldn’t automatically treat “not found” as permission to resend.

How do you currently assign and resolve operations in review, rather than only alerting someone that they exist?

7

One addition to the replay test: measure the action count, not just whether both executions are green.

I’d try the same ID again later, the same ID with changed action fields, and an older update arriving after a newer one. Those exercise different decisions: duplicate, conflict and stale. Also, pending can mean “still running”, so recovery needs to check ownership/expiry before another worker takes over.

I built a free learning lab on OperantSolo with 16 sequential HTTP cases and recorded the status, decision and mock action count for each:

Disclosure: that’s my site. The receiver uses experimental static data and fictional counters. All 16 sequential checks passed, but I haven’t validated concurrent claims or real crash recovery, so it’s a test fixture to learn from, not a production idempotency solution.

8

Good framing — the long-lived uncertain operation is where this pattern succeeds or fails. A few rules I’d harden into the workflow itself:

  • Owner defaults to the workflow owner. Don’t attribute it to a shared alias or a queue; a single human owner owns every review ticket. Shared aliases diffuse accountability and tickets die there.
  • Deadline is 24 hours, set at creation time. The deadline is a property of the ticket row, not a promise in someone’s head. Anything still pending past now() - 24h is eligible for escalation without human judgement.
  • Every ticket carries an evidence snapshot. Attach the idempotency key, attempt history, and the exact provider state you saw when it went uncertain. Reviewing a ticket without its snapshot is how “not found” gets concluded from a misread dashboard.
  • The resolution must be written back to the ticket. A review decision that lives in chat or a call never happened. Mark-done stays monotonic: pending → done or pending → review, never backwards, and the updater records who decided and on what evidence.
  • Expiry auto-escalates, it doesn’t auto-send. The sweeper upgrades the ticket’s priority or reassigns the owner; it never flips the side effect.

And the critical one you touched on: “not found” is not evidence of absence. An inconclusive lookup only extends the deadline — it never authorizes a resend. If the provider can’t prove the send didn’t happen, the send doesn’t happen. That’s the line between a review queue and a retry loop that doubles your emails.

9

Action count is exactly the right metric — binary pass/fail hides the interesting cases. Your three replay probes map cleanly onto the three branches of the receive handler:

  • Same ID again later → the skip branch. Verifies dedupe actually fires and the receiver does no work. This is the one most green-checkmark tests cover.
  • Same ID with changed action fields → the explicit-conflict branch. This is where designs diverge: first-write-wins silently drops the change, last-write-wins mutates history, and strict-conflict flags it. Whatever your policy is, it must be deliberate — the worst outcome here is a silently overwritten action where two executions “succeed” and one of them is a lie.
  • Older update arriving late → the version-watermark branch. Compare the incoming version/counter against the stored high-water mark; stale input gets recorded (for audit) but never applied.

One thing worth adding for the concurrent case: pending tasks need an atomic claim. Two workers racing the same pending row can both escalate or both mark-done. The fix is a single-flight lock on the ticket — an owner + expiry lease updated with a compare-and-swap: only the holder whose lease hasn’t expired can transition the ticket, and a new claim sets both fields atomically. Without that, every other guarantee above degrades under load, which is precisely when you’d notice it least.

10

atomic claims are critical whenever handling concurrent webhook bursts in n8n. separating state lookups from database inserts creates a race window where duplicate executions slip through before the claim key writes.

merging verification and record creation into a single atomic statement or utilizing a redis lock completely eliminates duplicate payload processing.