What is Idempotency?
The property that performing an operation more than once has the same effect as performing it once, which makes retries safe.
Definition
An operation is idempotent if repeating it leaves the system in the same state as running it once. Setting a user's email to a value is idempotent; adding 10 to a balance is not. In distributed systems this matters because failures are ambiguous: when a request times out, the client cannot tell whether the server never received it, processed it and crashed, or processed it and lost the response. Retrying is the only practical recovery, and retries are only safe if the operation is idempotent. HTTP defines idempotency per method in RFC 9110: GET, HEAD, OPTIONS, and TRACE are safe and therefore idempotent, and PUT and DELETE are idempotent because repeating them yields the same resource state. POST and PATCH are not guaranteed to be idempotent. Note that idempotency concerns server state, not the response: a second DELETE may return 404 instead of 204 and still be idempotent. For non-idempotent operations such as creating a payment, APIs add idempotency keys. The client generates a unique key (typically a UUID) per logical operation and sends it in an Idempotency-Key header; the server stores the key with the result and returns the stored result for any retry with the same key. Stripe popularized this pattern, keeping keys for at least 24 hours and rejecting reuse with different parameters. An IETF Internet-Draft (draft-ietf-httpapi-idempotency-key-header, not yet an RFC) describes the header and suggests 400 for a missing key, 409 for a retry while the original is still in flight, and 422 for reuse with a different payload. In message queues with at-least-once delivery, consumers achieve the same effect by recording processed message IDs or using upserts.
When to Use Idempotency
Design for idempotency anywhere a client, proxy, queue, or job scheduler might retry: payment and order creation, webhook handlers, message consumers, and scheduled jobs. Prefer naturally idempotent designs first, such as PUT to a client-chosen ID, upserts keyed on a business identifier, or a unique constraint that rejects duplicates. Add idempotency keys when an operation must stay a POST with side effects. Store keys atomically with the side effect, ideally in the same database transaction, or a crash between the two reintroduces duplicates. Decide and document how long keys are kept and what happens on reuse. Idempotency does not provide ordering or exactly-once delivery by itself; it makes duplicate delivery harmless.
Idempotency in Practice
- Creating a charge with an idempotency key; retrying the same command after a timeout returns the original charge instead of creating a second one. curl https://api.example.com/v1/charges \ -H "Idempotency-Key: 6f1c2a1e-6b8e-4d6c-9e0a-1f2b3c4d5e6f" \ -d amount=2000 -d currency=usd
- A Postgres upsert makes a webhook handler idempotent: replaying the same event updates the same row instead of inserting a duplicate. INSERT INTO orders (external_id, status) VALUES ($1, $2) ON CONFLICT (external_id) DO UPDATE SET status = EXCLUDED.status;
- A queue consumer inserts each message ID into a processed_messages table with a unique constraint inside the same transaction as its work; a redelivered message hits the constraint and is acknowledged without being processed again.
- A mobile checkout generates one key when the user taps Pay and reuses it for every automatic retry, so a flaky connection cannot produce two orders.
Common Mistakes
- Generating a new idempotency key on every retry. The key must identify the logical operation; create it once and reuse it for all retries.
- Checking for the key, doing the work, then saving the key in separate steps. Concurrent retries can both pass the check; use a unique constraint or lock and store the key in the same transaction.
- Assuming PUT or DELETE handlers are idempotent just because of the HTTP method. Side effects like sending an email or incrementing a counter in the handler break that guarantee.
- Accepting a reused key with a different request body. Store a hash of the original parameters and reject mismatches with an error.
- Treating idempotency as exactly-once delivery. Messages can still arrive twice or out of order; idempotency only makes duplicates harmless.