Idempotency Key

Problem

Networks fail. Clients time out. Load balancers drop connections. When a client sends a request and never receives a response, it faces a dilemma: did the server process the request and the response was lost, or did the server never see the request at all? The safe-but-wrong answer is "assume failure and retry." Without additional safeguards, that retry may execute a duplicate operation - charging a credit card twice, creating two user accounts with the same email, or shipping an order twice.

This problem is especially acute in payment processing, order management, and anywhere an API drives a real-world side effect that is expensive or impossible to reverse. A "try again" UX is only safe if the server can detect and absorb the retry.

Forces

• Retries are unavoidable. Mobile clients have intermittent connectivity. Microservices call each other across flaky internal networks. Any robust client must retry on timeout; any robust server must be prepared to absorb that retry.
• Non-idempotent operations are the default. HTTP POST (create), charging an amount, and sending a notification are all naturally non-idempotent. Making them idempotent requires explicit server-side bookkeeping.
• Exact-once is a client-server contract. The client must signal its intent ("this is a retry, not a new request"), and the server must honor that signal by tracking past requests and returning the cached result.
• Deduplication window matters. A key stored forever is a storage leak. A key stored for too short a time lets legitimate retries slip through. Most payment APIs use a 24-hour window.

Solution

The client generates a unique identifier - the idempotency key - before sending the request. This key is included in a request header (conventionally Idempotency-Key or X-Idempotency-Key). The server stores the key alongside the result of the first successful execution. On subsequent requests with the same key, the server short-circuits execution and returns the stored result without re-running the operation.

Client side:

async function chargeCard(amount: number, token: string): Promise<ChargeResult> {
  const idempotencyKey = crypto.randomUUID(); // generated once per logical operation
  
  return fetch('/api/charges', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Idempotency-Key': idempotencyKey,
    },
    body: JSON.stringify({ amount, token }),
  }).then(r => r.json());
}

The key must be generated before the first attempt and persisted by the client for the lifetime of the retry loop. Generating a new key on every retry defeats the purpose.

Server side:

async function handleCharge(req: Request): Promise<Response> {
  const key = req.headers.get('Idempotency-Key');
  
  if (key) {
    const cached = await db.query(
      'SELECT result FROM idempotency_keys WHERE key = $1 AND expires_at > now()',
      [key]
    );
    if (cached.rows[0]) {
      return Response.json(cached.rows[0].result, { status: 200 });
    }
  }
 
  // Execute the actual operation
  const result = await stripe.charges.create({ amount: req.body.amount, source: req.body.token });
 
  if (key) {
    await db.query(
      'INSERT INTO idempotency_keys (key, result, expires_at) VALUES ($1, $2, now() + interval \'24 hours\')',
      [key, JSON.stringify(result)]
    );
  }
 
  return Response.json(result, { status: 201 });
}

For extra safety, acquire a row-level lock on the key before executing: if two concurrent retries arrive simultaneously, only one should execute while the other waits and gets the cached result.

Key design guidelines:

• Use UUID v4 (random) rather than deterministic keys derived from request content - content-based keys can collide across different users sending identical payloads.
• Store the full response, not just a boolean. Clients need the original result on replay.
• Expire keys after a reasonable window (24h is standard for payment APIs).
• Return HTTP 200 (not 201) for replays - the resource was already created.

When NOT to Use

• Pure read operations. GET requests are already idempotent by HTTP semantics. There is no value in adding key tracking to reads.
• Idempotent writes by nature. Upsert operations (INSERT OR REPLACE, PUT with a client-supplied ID) are naturally idempotent when the payload is deterministic. An explicit key mechanism adds overhead without benefit.
• Internal batch jobs. Background workers that process a queue item once and mark it consumed already have their own idempotency mechanism (the queue's visibility lock). Layering idempotency keys on top is redundant.
• Very high throughput endpoints with short TTLs. If you're processing millions of requests per second, the database write per request for key storage can become a bottleneck. Consider in-memory deduplication with a Redis SET at the cost of losing the guarantee across restarts.

Related Patterns

Outbox and Idempotency Key are complementary: the Outbox pattern guarantees at-least-once event delivery from producer to broker, and Idempotency Key ensures that the consumer of those events can safely absorb duplicates. Together they cover both ends of the reliability contract. The Saga pattern relies on both - saga steps need reliable event delivery (Outbox) and safe compensation on retry (Idempotency Key).

References

• Stripe API documentation. "Idempotent requests." stripe.com/docs/api/idempotent_requests
• Kleppmann, Martin. Designing Data-Intensive Applications. O'Reilly, 2017. Chapter 9 (Consistency and Consensus).
• Helland, Pat. "Idempotence Is Not a Medical Condition." ACM Queue, 2012.
• RFC 9110 - HTTP Semantics. Section 9.2.2 (Idempotent Methods).