n8n webhook idempotency
Test the replay, not the happy path.
A webhook does not become safe because its first delivery worked. The useful invariant is stricter: one source event may produce at most one intended external action, even when delivery, execution, acknowledgement, or receipt storage is repeated.
The business invariant
Exactly-once delivery is not the target.
Assume a valid request can arrive again.
Senders retry after timeouts. Networks lose responses. A worker can stop after an external service accepts a request but before the workflow records success. n8n also documents that a failed execution can be retried with its previous execution data. None of those conditions is automatically a defect; the workflow needs an explicit replay contract.
The design question is therefore not, "How do I prevent a second HTTP request?" It is, "How does every delivery of the same business event converge on the same durable result?"
One stable event key, one external action receipt.
A replay can return the saved result, report that processing is still in flight, or stop for reconciliation. It must not casually create a second email, charge, record, or update.
Start with identity
Choose a key that survives every retry.
The best key is normally created by the system that owns the event: a payment event ID, order ID plus operation, form submission ID, or message ID. Preserve that value at the webhook boundary. Do not replace it with an n8n execution ID, because a retry may create a different execution while representing the same source event. Do not generate a random UUID inside the workflow, because every delivery would receive a new identity and every duplicate would look legitimate.
A payload hash can be a fallback only when the business meaning is understood. Hashes require canonical serialization; otherwise key order, irrelevant timestamps, or formatting can create different hashes. The opposite error matters too: two legitimate orders may have identical amounts and customer fields. A collision that suppresses a real event is not safer than a duplicate action.
Good key properties
- Controlled by the source or defined business operation
- Stable across delivery and manual execution retries
- Small enough to index and log without sensitive payloads
- Scoped to the side effect it protects
Keys that need scrutiny
- A random value created after the webhook arrives
- The current n8n execution ID by itself
- A timestamp or mutable business field
- An undocumented hash of the entire request body
The smallest useful state machine
Claim before acting; commit a receipt after acting.
{
"event_key": "evt_100",
"operation": "create_invoice",
"status": "processing",
"attempt": 1,
"action_id": null
}{
"event_key": "evt_100",
"operation": "create_invoice",
"status": "completed",
"attempt": 1,
"action_id": "inv_456"
}The initial claim must be atomic. In a relational database, that can mean a unique constraint plus an insert or transaction. In a purpose-built key-value store, it can mean a conditional create. A read followed by an unrelated write is not equivalent: two concurrent executions can both read "missing" before either one writes. The invariant belongs in the storage operation, not only in the visual order of n8n nodes.
A record in processing tells later deliveries that an execution already owns the claim. A completed record carries the durable result. A failure that is proven to occur before the external call may become failed_retryable. A failure after the call begins is different: the remote system may have acted even when the workflow did not receive or store a response. That state is ambiguous, and automation should stop until the downstream system is reconciled.
Runnable local proof
A deterministic harness makes the failure boundary visible.
The repository behind this page includes a dependency-free Node.js harness and six automated tests. It does not call n8n, a payment processor, or any external service. Its purpose is narrower: prove the decision contract before adapting it to a real durable store. The harness claims a stable key, counts external-action attempts, records a completed receipt, and refuses to automatically replay an ambiguous post-action failure.
const harness = createWebhookReplayHarness();
await harness.deliver({ eventKey: "evt_100" });
await harness.deliver({ eventKey: "evt_100" });
assert.equal(harness.snapshot().externalActionCount, 1);The concurrent test pauses the first external action after its claim, delivers the same event again, and verifies that the second call receives suppressed_in_flight. Another test fails before the effect and confirms that a retry can complete once. The most important negative test simulates a crash after the effect: the next delivery receives manual_review, and the action count stays at one. That fail-closed result is less convenient than an automatic retry, but it avoids converting uncertainty into a known duplicate.
The local harness is not the production store.
Its in-memory claim is deterministic proof of the state transition, not evidence of durability across processes or hosts. Production implementation still needs a shared atomic store, retention policy, monitoring, and downstream reconciliation path.
Eight required tests
Replay the same key at every meaningful boundary.
| Test | Setup | Required evidence |
|---|---|---|
| First valid delivery | No record exists for the source event key. | Claim the key, perform one action, then store a completed receipt. |
| Immediate duplicate | The same key arrives while the first run is still processing. | Return an in-flight response. Do not start another action. |
| Completed replay | The same key arrives after a completed receipt exists. | Return the existing outcome. External action count remains one. |
| Failure before action | Validation or preparation fails before the external call begins. | Mark retryable. A later attempt may claim the same key safely. |
| Crash after action | The external call finishes, but receipt storage does not. | Mark ambiguous and require reconciliation; never auto-repeat blindly. |
| Manual execution retry | A person retries failed execution data in n8n. | The same stable key reaches the same claim boundary and cannot act twice. |
| Different event, same payload | Two legitimate source IDs contain identical business fields. | Treat them as distinct events unless the business invariant says otherwise. |
| Missing event key | The sender provides no stable source-controlled identifier. | Fail closed or route to review. Do not invent a random key per delivery. |
Count the external action directly. A green workflow execution is not sufficient evidence, because an execution can succeed after suppressing a replay or fail after a remote service already acted. Record the source key, claim outcome, action attempt, downstream identifier, and final receipt status. Redact secrets and personal data; the evidence needs identifiers and state transitions, not a copy of every payload.
Map it into n8n
Keep the credentialed node downstream of the claim.
- 01
Receive and authenticate
Use the production webhook URL and the authentication supported by the sender. Reject malformed or unauthenticated requests before they can create a claim record.
- 02
Normalize the operation key
Preserve the source ID and combine it with the exact business action when one event can legitimately trigger several independent effects.
- 03
Perform an atomic claim
Use a database or service that can enforce uniqueness in one operation. Route completed, in-flight, ambiguous, and new outcomes separately.
- 04
Call the external system once
Pass the same idempotency key downstream when its API supports one. Keep credentials here, not in the public webhook or validation branch.
- 05
Commit and acknowledge
Save the downstream identifier and completed receipt, then return the acknowledgement. If receipt storage fails after the action, record ambiguity instead of assuming failure.
n8n's Remove Duplicates node can compare current items with data from previous executions, and that can be a useful filter. Treat it as one layer rather than the entire contract. Confirm what fields it compares, how long history is retained, whether all action paths pass through it, and what happens under concurrent execution. A durable unique claim close to the side effect is easier to reason about than a visual deduplication step far upstream.
The dangerous middle
A timeout does not prove the action failed.
Imagine an HTTP Request node sends an invoice request. The remote service creates the invoice, but the connection closes before n8n receives the response. Retrying the node may create another invoice. Marking the run failed is operationally correct but does not answer the business question. The system needs a way to query by the stable key, inspect the remote system, or hand the event to a reviewer before another action is attempted.
This is why retry policy must be classified by failure location. Schema validation, authentication rejection, and a database connection failure before claiming are different from a socket timeout after sending the request body. A generic "retry three times" setting erases that distinction. Bounded retries are useful only for states known to be safe to repeat.
Usually retryable
- Failure before the external action begins
- Rejected validation with corrected new input
- Downstream API with a verified idempotency contract
Reconcile before retrying
- Timeout after request transmission begins
- Worker crash after the remote response
- Receipt-store failure after a successful action
Evidence that survives deployment
Log decisions, not sensitive payloads.
A replay-safe workflow should make its boundary inspectable. For each delivery, record a correlation ID, a redacted or opaque event key, the operation, claim result, current state, attempt number, downstream action ID when available, and the final route. Include enough timing to reconstruct order without treating wall-clock time as the identity. Protect those records using the same privacy and access rules as the underlying business event.
Alerts should distinguish a suppressed replay from an ambiguous event. A suppressed completed replay is often healthy evidence that the design worked. An ambiguous event is actionable because a person or reconciliation process must determine whether the remote system acted. Counting both as generic failures makes the dashboard louder and the important state harder to see.
| Field | Why retain it | Avoid |
|---|---|---|
| Event key | Correlates every delivery of the same source event. | Raw tokens, passwords, or full sensitive payloads. |
| Claim outcome | Shows new, in-flight, completed, retryable, or ambiguous. | A single success/failure flag that hides the boundary. |
| Action ID | Supports downstream lookup and reconciliation. | Invented IDs that the external system cannot query. |
| Attempt | Explains bounded retry and replay behavior. | Unbounded counters with no stop or review rule. |
Production checklist
Do not ship until the negative cases are repeatable.
Contract
Document the stable key, protected operation, retention period, and exact state transitions.
Concurrency
Deliver the same key simultaneously and prove the store grants one claim rather than two read-then-write approvals.
Ambiguity
Interrupt the path after the effect and prove the next delivery stops for reconciliation.
Operations
Alert on ambiguous records, rehearse the lookup procedure, and assign a human owner before launch.
Run this matrix in a non-production environment using synthetic events and a harmless test destination. Do not intentionally duplicate real charges, messages, or customer records to prove the control. If a provider offers a sandbox and official idempotency key, verify its documented retention and response behavior rather than assuming all services implement the same semantics.
Official n8n references
Platform behavior used by this test plan.
n8n documents separate test and production webhook URLs and supported authentication options for the Webhook node. Its Remove Duplicates node documentation describes comparing items with data from previous executions. The execution documentation describes retrying a failed workflow with current or original workflow data, and the error-handling guide describes Error Trigger workflows. Those capabilities support the implementation, but none removes the need to define and test the business-side idempotency invariant.
Need a second set of eyes?
Get five prioritized workflow risks for $25.
Send one sanitized n8n export and two or three sanitized run examples. Receive a manually checked, node-level reliability report within 24 hours after complete intake. No account access, credentials, production changes, or claim of guaranteed uptime.
Boundaries and questions
What this pattern does and does not establish.
Is the n8n Remove Duplicates node an idempotency guarantee?
No. It can compare items with earlier execution data, which is useful, but a complete guarantee also depends on the selected key, history retention, concurrency, the workflow path, and the external system. Test the final side effect, not only the node output.
Should a failed webhook always be retried?
Only when evidence shows the external action did not begin or the downstream API provides a trustworthy idempotency contract. If the action may have succeeded but the receipt was lost, reconcile or review it instead of automatically repeating it.
Can I use a hash of the payload as the event key?
Sometimes, but it changes the business rule. Two legitimate events may contain identical payloads, while irrelevant field ordering or timestamps may change a hash. Prefer a stable source event ID. If a derived key is necessary, document its canonicalization and collision boundary.
Does this article prove a production deployment?
No. This is a self-directed, deterministic test pattern with a runnable local harness. Production safety still depends on the real sender, durable store, deployment topology, downstream API, retention policy, and observed failure behavior.