For webhook receivers in production, how are you handling the atomic claim?
A lookup followed by a write leaves a gap where two executions can both see ‘not seen’ before either saves ‘seen’. In a sequential lab I used a scoped event key, but I haven’t proven concurrent safety there.
Do you use a database unique constraint, Data Tables, or another store for the claim — and how do you handle the conflict path?
To prevent race conditions in high-concurrency environments, you should move away from “check-then-act” logic (lookup followed by a write) and instead use an atomic insert strategy. The most robust method is to use a database like PostgreSQL with a UNIQUE constraint on your event identifier. By attempting to INSERT the event ID as the very first step after receiving the webhook, you allow the database to handle the synchronization; if the insert fails due to a unique violation, the execution has hit a duplicate and can be safely terminated or redirected.
For complex or long-running workflows, you can extend this into a state-driven pattern to handle partial failures. Instead of a simple existence check, you insert a record with a status like processing and update it to completed only after the workflow succeeds. This “intent-based” approach ensures that if a workflow crashes midway, you can distinguish between an event that is currently being handled, one that failed and needs a retry, and one that has already been fully processed.
While n8n provides a built-in Remove Duplicates node for lightweight deduplication, it is generally not recommended for mission-critical production webhooks because it relies on n8n’s internal execution history and may not provide the strict, distributed atomicity required for high-concurrency scenarios. For maximum reliability, using an external database as your “source of truth” for idempotency via unique constraints remains the industry standard.
To add the n8n side and the conflict path to the unique-constraint approach above:
Claim. Make it one statement in a Postgres node (Execute Query), straight after the Webhook node:
INSERT INTO webhook_claims (event_key)
VALUES ($1)
ON CONFLICT (event_key) DO NOTHING
RETURNING event_key;
event_key is the primary key or has a unique constraint. Pass the key through Query Parameters, not by building the SQL string. With ON CONFLICT a duplicate isn’t an error, so you don’t need error branches for the normal case.
Conflict path. The first execution gets a row back; a duplicate gets none. What the node outputs for “no rows” can differ by node version, so test for the value rather than for item count: turn on Always Output Data on the node, then add an IF node checking that {{ $json.event_key }} exists.
Crashes. A plain claim will block a retry if the execution dies after claiming. If you need retries, add status and claimed_at columns and let a stale or failed claim be taken again:
INSERT INTO webhook_claims (event_key, status, claimed_at)
VALUES ($1, 'processing', now())
ON CONFLICT (event_key) DO UPDATE
SET status = 'processing', claimed_at = now()
WHERE webhook_claims.status = 'failed'
OR (webhook_claims.status = 'processing'
AND webhook_claims.claimed_at < now() - interval '15 minutes')
RETURNING event_key;
Set the interval longer than your slowest run, and write status = 'done' as the last step.
Data Tables. I haven’t tested them under concurrent writes, so I wouldn’t use them for this until you’ve run the test below.
To verify: send 20 parallel requests with the same key (for example curl in a loop with &). You should see one row in the table and one execution on the True branch; the other 19 go to False.
Minimolo
Adding to the unique constraint approach: it helps to store a status with the claim row (for example processing and done) plus a timestamp, not only the event ID. A plain “seen” flag has a gap when an execution claims an event and then crashes before finishing, because the retry will be treated as a duplicate and the event is lost.
With a status and timestamp you can let a retry take over a claim that has been in processing for too long, while still rejecting events that are still in progress or already done.
Also, using INSERT … ON CONFLICT DO NOTHING and checking the affected row count gives you the claim result in a single atomic statement, so the conflict path is just an IF branch on that count. And respond to the sender quickly (for example with the Respond to Webhook node) so it does not retry while your workflow is still running.
A PostgreSQL UNIQUE constraint is a solid starting point for the atomic claim.
One edge case I’d test is when a downstream CRM write succeeds, but the workflow crashes before it can mark the event as completed. On retry, the event is still sitting in “processing,” even though the CRM change already happened
For CRM integrations, I’d prefer an upsert using a unique external ID where the API supports it, or reconcile the target record before retrying while guarding against concurrent retries.
A lease or timeout can help recover abandoned claims, but it won’t make downstream API calls idempotent on its own
Are you mainly trying to prevent duplicate writes, or do you also need to recover partially completed executions?
Use a database unique constraint on the event ID and insert first, instead of looking up then writing. If the insert succeeds you own the event, and if it fails with a conflict you stop and return 200 so the sender doesn’t retry. Postgres with ON CONFLICT DO NOTHING works well, and checking whether a row came back tells you which path you’re on.
One gap in the “Remove Items Processed in Previous Executions” route is history size. With “Value Is New”, History Size sets how many items the node stores, and it defaults to 10,000. If the store keeps only the newest IDs, a late retry of an older event will look new and pass. So size the history to your longest retry window, not just your daily volume.
The docs don’t describe behavior under concurrent runs, so I wouldn’t rely on this node for the atomic claim. An insert with a unique constraint in Postgres is the safer pattern for that part. Is your retry window longer than the number of events you store?