Shopify + n8n: 5 tests for duplicate webhooks and failed-run recovery

[Marchik]

The recent orders/create discussion covers duplicate filtering and protecting the downstream action. Here is a test checklist to make the expected recovery behaviour explicit.

Shopify documents that a webhook can arrive more than once and recommends idempotent processing: webhook delivery guidance.

The easy-to-miss case is a failed first attempt:

1. Order 1042 is recorded as “seen”.
2. The workflow fails before creating its CRM order record.
3. A retry arrives, sees the saved ID, and skips the order.

There is no duplicate, but the intended action never happened. “Seen” and “completed” need different meanings.

For an action intended to happen once per order, define a stable operation key, for example shop:orders-create:order-id:create-crm-order. Keep the same key on retries; a different intended action needs its own key.

Run these checks in a test environment with a mock destination:

Test	Expected result
One normal delivery	One destination record, with its ID saved as completion evidence.
Replay after successful completion	No second destination record.
Two simultaneous deliveries	One destination record despite overlapping executions.
Fail after recording the attempt but before the destination write	A controlled recovery completes the missing action; it is not permanently skipped as “already seen”.
Destination write succeeds, but its response or the local completion update is lost	Recovery confirms the existing result or safely retries using destination-supported idempotency; it does not blindly create another record.

n8n’s Remove Duplicates node can compare values with previous executions. That history alone does not establish whether a later business action completed.

For the destination write, use its atomic uniqueness or idempotency mechanism where available. A separate “claim” row is not proof that an external API call succeeded. For uncertain outcomes, reconcile with the destination; if it cannot support a safe retry or reliable lookup, route the case for manual review.

Record the operation key, attempt status, destination record ID and recovery outcome. Judge the tests by destination records, not only by a green n8n execution.

This is a test checklist, not a tested importable workflow or a guarantee across every API. Destination idempotency rules and retention limits still apply.

Which downstream action in your order workflow must happen only once: a CRM record, a fulfilment task, an invoice, or something else?

[Aghassi]

“Seen” and “completed” needing different meanings is the whole thing, and I hit it in the engine rather than in a workflow. A step that was skipped had no state of its own to record, so it was rolled into the run’s own COMPLETED, and 27 branches went unprocessed for weeks while every run reported success. The fix was the same shape as your operation key: give the skipped case a real row with a reason, and derive the run’s counts from those rows so they cannot drift from what happened. The part I would ask about your key: it encodes the intended action, so what happens when the intended action changes: do you version the key, and how do you stop a redeploy rerunning everything under a new one?

[Marchik]

Good question. I wouldn’t include the workflow release in the idempotency key by default. I’d keep the key tied to the business action, for example shop:<shop-id>:order:<order-id>:create-crm-order. If a deploy only changes the mapping but it’s still the same create operation, the key stays the same.

If v2 really is a new operation, I’d track it explicitly in a migration ledger rather than use order.created_at < deploy_time as the cutoff. A delayed webhook for an older order can arrive after the deploy, and its creation time doesn’t prove whether the v1 action completed.

I’d mark a v2 operation as complete during backfill only after verifying that its CRM result already exists. If the outcome is unclear, I’d reconcile it first. A unique constraint or upsert on the Shopify order ID is a useful backstop for the CRM record, but invoices and fulfilment actions need their own idempotency or recovery checks.

[Aghassi]

That answers it, and the delayed webhook is the case I would have got wrong. I would have used created_at as the cutoff. It also made me check my own engine against your checklist, and it fails your second test. A webhook trigger in my tool starts a new run on every POST, with no key at all, so a Shopify retry of order 1042 runs the whole workflow twice. The only thing between that and a duplicate CRM record today is whatever the destination does on its own. So your first two tests are my next job, before anything cleverer.

[Marchik]

Good catch. I’d run test 3 alongside your replay test: send the same order webhook twice concurrently, before either run finishes. A sequential retry can be blocked while two overlapping runs still pass a “not completed” check.

At the engine level, I’d atomically claim a stable business-action key—shop, order, and intended CRM action—before dispatching work, or serialize work by that key. But a second delivery finding pending must not be treated as completed: if the first run dies, the action still needs recovery. Destination uniqueness or documented idempotency, where available, is a backstop when the CRM write succeeds but its response is lost.

Can your ingestion layer claim that key atomically, and what happens to the second delivery while the first is pending?