Idempotency & Exactly-Once

In one line: Because you can't tell "succeeded" from "lost reply", every reliable distributed system retries — and retries mean operations will run more than once, so the only way to be correct is to make operations idempotent (safe to apply repeatedly); "exactly-once delivery" is impossible, but "exactly-once effect" is achievable, and that's what you actually want.

In plain English

This is the property that ties the whole chapter together. Networks lose messages and replies, so reliable systems retry; queues redeliver; sagas re-run steps; users double-click. The result is that the same operation arrives more than once, and if running it twice does damage — two charges, two emails, two shipments — you have a bug that only shows up in production, intermittently, around money. Idempotency is the cure: design the operation so that running it five times has the same effect as running it once. People wish for "exactly-once delivery" (the message is delivered precisely once, ever) — but that's provably impossible over an unreliable network. What you can build is "exactly-once effect": the message might be delivered several times, but you process it such that the outcome is as if it happened once. The trick is almost always the same: a stable idempotency key you record and check.

Why "exactly-once delivery" is a myth

Walk the cases from the fallacies page. A sender delivers a message and waits for an ack. The ack doesn't come. Two possibilities the sender can't distinguish: (a) the message was lost — resend it; (b) the message arrived and was processed, but the ack was lost — resending duplicates it. The sender must choose:

• At-most-once: never retry. No duplicates, but messages can be lost. (Fine for, say, a metrics sample.)
• At-least-once: retry until acked. No losses, but duplicates happen. (The default for anything that matters.)
• Exactly-once delivery: neither loss nor duplication. Impossible in the general case — you'd need to know the unknowable (did the other side get it?).

Since at-least-once is the only safe choice for important work, duplicates are inevitable — so the receiver must be built to tolerate them. That's idempotency. "Exactly-once" systems (like Kafka's exactly-once semantics) achieve the effect by combining at-least-once delivery with idempotent processing/deduplication under the hood — not by magically delivering once.

Naturally vs non-naturally idempotent operations

Some operations are idempotent for free; others need help:

Operation	Idempotent?	Why
SET balance = 100	✅ Yes	Applying it again gives the same result
DELETE WHERE id = 5	✅ Yes	Already deleted → still deleted
HTTP GET, PUT, DELETE	✅ By spec	(PUT replaces; DELETE removes)
balance = balance + 10	❌ No	Each run adds again → wrong total
INSERT a new order	❌ No	Each run creates another order
HTTP POST (create)	❌ Typically not	Each call makes a new resource

The dangerous ones are the increments and creates — exactly the operations involving money and orders. You make them idempotent with a key.

The idempotency key pattern

Attach a stable, unique idempotency key to each logical operation (generated by the client or derived from the operation), record it the first time you act, and on any repeat, recognize the key and return the original result instead of acting again.

// Charge a card at-most-once-in-effect, even if the request arrives many times.
async function charge(idempotencyKey, userId, amountCents) {
  // Atomically claim the key. The UNIQUE constraint is the linchpin:
  // a concurrent or later duplicate fails to insert and is recognized.
  const claimed = await db.query(
    `INSERT INTO idempotency_keys (key, status) VALUES ($1, 'pending')
     ON CONFLICT (key) DO NOTHING RETURNING key`,
    [idempotencyKey]
  );

  if (claimed.rowCount === 0) {
    // Key already exists → this is a duplicate. Return the stored result.
    const prior = await db.query(
      `SELECT result FROM idempotency_keys WHERE key = $1`, [idempotencyKey]
    );
    return prior.rows[0].result; // same answer as the first time; no second charge
  }

  const result = await paymentProvider.charge(userId, amountCents); // the real side effect
  await db.query(`UPDATE idempotency_keys SET status='done', result=$2 WHERE key=$1`,
    [idempotencyKey, result]);
  return result;
}

This is exactly how Stripe's API works — you pass an Idempotency-Key header and Stripe guarantees that retrying the same request won't charge twice. The pattern generalizes to message consumers (dedupe on a message ID), saga steps, and webhook handlers (which providers retry aggressively).

Highlight: idempotency is the keystone of distributed reliability

Notice how many earlier pages quietly depend on this one:

• Retries with backoff are only safe if the operation is idempotent — otherwise every retry risks a duplicate effect.
• At-least-once queues and serverless redeliver; consumers must dedupe.
• Saga compensations re-run on failure; they must be idempotent.
• The transactional outbox relays events at-least-once; consumers must be idempotent.
• A client double-click or a flaky network resubmits; the endpoint must be idempotent.

Idempotency is the single property that makes "retry until it works" — the fundamental coping mechanism for an unreliable network — correct instead of dangerous. If there's one habit to internalize from this entire chapter, it's: any operation with a side effect that can be retried needs an idempotency strategy, decided before it ships, not after the first duplicate.

Worked example: the webhook that fired ten times

A team's payment provider sends a webhook on successful payment, which their handler uses to grant the user access and send a receipt. The provider's delivery is — correctly — at-least-once, and one day a network blip means the provider doesn't get the team's 200 OK, so it retries the webhook... ten times over an hour. The handler runs ten times: ten access grants (harmless-ish) and ten receipt emails (the user is annoyed and files a complaint). The provider did nothing wrong — at-least-once is the only safe delivery model. The bug is the handler assuming it'd be called once. The one-line fix: record the webhook's event ID and make the handler a no-op if that ID was already processed. Every webhook integration needs this, because every reputable provider retries. "Process each webhook exactly once" isn't something the provider can give you — it's something you build with an idempotency check.

Common mistakes

Where people commonly trip up

• Assuming exactly-once delivery exists. It doesn't over an unreliable network. Build at-least-once + idempotent processing to get exactly-once effect.
• Retrying non-idempotent operations. Backoff/retry on a charge or INSERT without a key duplicates the effect. Add an idempotency key before enabling retries.
• No idempotency on webhook handlers. Providers retry aggressively; un-deduped handlers double-grant, double-email, double-ship. Dedupe on the event ID.
• Using a non-stable or wrong-scope key. A key that changes per attempt (e.g. a fresh UUID each retry) defeats the purpose; the key must identify the logical operation, generated once and reused across retries.
• Race conditions in the dedupe check. "Check then act" without atomicity lets two concurrent duplicates both pass the check. Use an atomic claim (unique constraint / conditional insert), as in the example.
• Deciding idempotency after the first incident. It's a design property. Decide the key and storage for every side-effecting, retryable operation before it ships.

Page checkpoint

Checkpoint Quiz

Did idempotency stick?

Required

What's next

→ Continue to Messaging patterns — the delivery semantics, deduplication, and ordering guarantees of the message systems that connect distributed services.