Stop Duplicate Charges: 3 Storage Choices for Idempotent API Design

Idempotent API design makes retries safe: repeated calls produce the same observable effect as one. For inherently unsafe methods like POST and PATCH, the fix is an Idempotency-Key header bound to method, path, principal, and payload, so a network retry or client timeout never doubles a payment, ships two orders, or duplicates a record. GET, PUT, DELETE, HEAD, and OPTIONS get this safety for free under the HTTP specification; everything else needs deliberate design.

TL;DR:

• Idempotency keys must be generated with UUID or ULID standards and bound to method, path, principal, and request body to prevent collisions.
• Safeguards like locks or polling should manage concurrent long-running requests to maintain idempotency under high load.
• The server should respond with appropriate status codes, such as 409 for conflicts or 202 for long operations, to clearly communicate retry conditions.
• Keys shorter than 16 characters or not scoped to the correct request scope increase collision risk and compromise retry safety.
• Using a combination of durable storage and fast in-memory checks helps ensure response consistency and quick duplicate detection.

Table of Contents

• What makes an API idempotent under HTTP semantics?
• How does the Idempotency-Key header work?
• Where should idempotency keys be stored?
• Handling concurrency and long-running operations
• Which status codes signal idempotency behaviour correctly?
• Proving idempotency holds under real failure conditions
• Checklist and mistakes that break idempotency in practice
• Operational perspective: idempotency inside a living integration
• A different route to fewer contract bugs across your stack
• Sources
• FAQ

What makes an API idempotent under HTTP semantics?

An HTTP method is idempotent when firing it once has the same effect on the server as firing it ten times. That's the definition RFC 7231 sets out, and it's the bedrock every other decision in this article rests on. It's not about whether the response looks identical, it's about whether the state change is identical.

Five methods get idempotency built in by their own semantics:

• GET and HEAD: pure reads, no state change at all.
• PUT: replaces a resource wholesale, so applying the same replacement twice leaves the same end state.
• DELETE: the resource is gone after the first call; calling it again just confirms it's still gone (even if the status code shifts from 200 to 404).
• OPTIONS: metadata only, never mutates anything.

POST and PATCH don't get this guarantee. POST typically creates a new resource, so two identical calls usually mean two resources, whether that's two customer records or two charged invoices. PATCH applies a partial modification, and depending on how that modification is expressed (an increment rather than a set value, say), replaying it can compound rather than repeat. A checkout retry after a dropped connection is the textbook failure: the client never saw a response, assumes it failed, retries, and the customer is billed twice.

This is exactly why native idempotent methods let clients and load balancers retry automatically without asking permission. Anything outside that set needs its own retry contract, which is what the Idempotency-Key header exists to provide.

How does the Idempotency-Key header work?

The header gives POST and PATCH the retry safety that PUT and DELETE get natively. A client generates a unique key, attaches it to the request, and the server treats every subsequent request carrying that same key as a replay rather than a new operation. This isn't an informal convention any more: an IETF draft standard now defines the header's semantics, and MDN's documentation covers the practical client and server behaviour developers actually need.

On the first request with a given key, the server processes it normally and stores the outcome. On any duplicate, it skips reprocessing and returns the stored response verbatim, keeping the client's view of the world consistent.

Format matters more than developers often assume:

• Use UUIDv4, UUIDv7, or ULID rather than sequential integers or predictable strings.
• Reject keys under roughly 16 characters. The community idempotency standard flags short keys as a collision and abuse risk, not just a style nit.
• Never embed personal or account data inside the key itself. It should be opaque, a reference, not a payload.

A quick check on scope: the IETF draft's own recommendation is to bind each key to a composite of method, canonical path, authenticated principal, and a hash of the request body, not just to store it as a bare string. Skip that binding and the same key becomes reusable across completely unrelated endpoints, which is a bug waiting to surface in production.

The server contract needs documenting up front: which endpoints require the header, what happens if it's missing, how long a key stays valid, and exactly what a client sees if it reuses a key with a different payload. Leaving any of that undocumented is how "idempotent" APIs still get RFC-compliant retries that produce inconsistent results.

Where should idempotency keys be stored?

Three decisions drive the storage design: what fields the record needs, whether to use a relational database, Redis, or both, and how the fingerprint that scopes the key gets computed.

A minimal idempotency store needs to answer one question fast: has this exact key been seen before, and if so, what did the server return? That means each record needs the key itself, a fingerprint of the bound scope, the response status and body to replay, a processing state (in progress, completed, failed), and a creation timestamp for expiry logic.

On storage choice, the trade-off is speed versus durability:

• Relational database with a unique constraint on the idempotency key gives strong durability and a natural way to reject concurrent duplicates at the database layer, but adds latency on every write.
• Redis offers sub-millisecond lookups and is well suited to short-lived locks, though it needs persistence configured (AOF or RDB) or a completed transaction can vanish on restart.
• Hybrid: use Redis for the fast "is this key already being processed" check and a durable relational store, with a unique constraint, for the final response. This is the pattern Google Cloud's guidance and most engineering teams converge on for anything touching money or inventory.

The fingerprint itself should hash method, canonical path, authenticated principal, and the request body together, precisely the composite the IETF draft recommends. Without the principal in that mix, one tenant's key could theoretically collide with another's.

TTL is a business-risk decision, not a technical default. A payment endpoint might retain keys for 24 to 48 hours to cover realistic client retry windows; a low-stakes preference update might expire in an hour. Retain too long and storage costs creep; retain too short and a legitimate late retry gets treated as brand new.

Handling concurrency and long-running operations

Two requests carrying the same idempotency key can arrive within milliseconds of each other, particularly when a client's retry logic fires before the first attempt has even returned. The server needs a clear policy for who "wins."

A single-writer lock on the fingerprint is the standard approach: the first request acquires the lock and proceeds, the second checks the store, finds an in-progress record, and has three sensible options.

• Wait briefly for the first request to finish, then return its stored response. Fine for fast operations.
• Return 409 Conflict immediately, telling the client to retry later rather than block. Better for anything with unpredictable latency.
• Replay the stored response once it's available, which is really the wait strategy's endpoint, not a separate one.

For operations that genuinely take time, blocking a second request isn't practical anyway. The better pattern is to treat the operation as long-running: return 202 Accepted with a Location header pointing to a status resource, let the client poll it, and update the idempotency store to the terminal response only once the work actually completes. This keeps the idempotency guarantee intact without forcing a client connection to sit open for a batch job or an async payment settlement.

The trade-off is UX versus scale. Blocking feels simpler for the client but doesn't survive load; asynchronous polling scales cleanly but pushes more design work onto the client side, including knowing when to stop polling.

Which status codes signal idempotency behaviour correctly?

Status codes are how the server communicates why a request was treated a particular way, and getting them wrong is one of the fastest ways to confuse client retry logic.

• 400 Bad Request when an endpoint requires an Idempotency-Key and the client didn't send one.
• 409 Conflict when a request with the same key is currently being processed elsewhere.
• 422 Unprocessable Entity when the key matches a prior request but the payload doesn't, a mismatch, not a duplicate.
• 202 Accepted for long-running operations, paired with a Location header for status polling.
• 5xx responses should never be written to the idempotency store as terminal outcomes. A transient failure isn't the same as a completed operation, and treating it as one traps the client in a replay loop that repeats the original error forever.

Wrap error bodies in RFC 9457 Problem Details formatting, with a type field linking to documentation, so a client library can act on the error programmatically rather than parsing prose.

Proving idempotency holds under real failure conditions

Idempotency claims are easy to write in documentation and surprisingly easy to break in production, usually under exactly the conditions that made retries necessary in the first place: network partitions, slow databases, and clients retrying more aggressively than anyone tested for.

1. Fire identical concurrent requests with the same key and confirm only one side effect occurs, not two records or two charges.
2. Kill the process after the database commit but before the response is sent, then replay the same key and confirm the client gets the correct stored response rather than a reprocessed one.
3. Compare replay responses byte-for-byte against the original to catch silent drift in response shape.
4. Run chaos experiments, injecting latency and dropped connections specifically around the commit boundary, which is where most idempotency bugs actually live according to engineering guides on the topic.

Track replay rate, key collision events, fingerprint mismatches (422s), TTL expiries hitting legitimate retries, and idempotency-store lookup latency on a dashboard. A sudden spike in mismatches usually means a client bug, reusing a key across different requests, rather than a server fault.

Pro Tip: Run your kill-after-commit test against a queue-based side effect too, not just the primary database write. Idempotency bugs love to hide in the second system that gets updated after the "real" transaction succeeds.

Checklist and mistakes that break idempotency in practice

A short checklist catches most of what goes wrong:

• Require and validate the Idempotency-Key header on every unsafe write endpoint.
• Bind the key to method, path, principal, and payload hash, never just the raw string.
• Set TTLs based on business risk, not a copy-pasted default.
• Store and replay the exact original response, not a freshly recomputed one.
• Document expiry, required endpoints, and error semantics in your API spec, ideally as an OpenAPI header parameter with examples of 409 and 422 responses.

Operational perspective: idempotency inside a living integration

I've come to think idempotency keys and domain uniqueness constraints solve different problems, and conflating them is where teams get into trouble. A unique constraint on an order number stops true duplicates at the database. An idempotency key stops the client's retry from ever reaching that far in the first place. You often need both.

What I find more interesting is how often idempotency actually breaks, not because of poor concurrency handling, but because a provider quietly changed a field and the fingerprint logic hashed something different on the retry than it did on the original call. A comprehension-first view of your integrations catches that kind of contract drift before it becomes a 422 nobody can explain. Document your error semantics generously. Future engineers, including you in six months, will thank you.

— Aaron Gammon

A different route to fewer contract bugs across your stack

Idempotency keys solve the retry problem at a single endpoint. They don't solve the harder problem beneath it: an upstream provider silently renames a field, your fingerprint hash changes shape, and a perfectly correct idempotency implementation starts rejecting legitimate replays as mismatches. That's a comprehension problem, not a retry problem, and there are platforms built to address it.

Certain platforms build a live, inferred model of how your systems actually relate to each other, rather than relying on documentation that goes stale the day a vendor ships an update. When a schema shifts, self-healing integration catches the drift before it corrupts a fingerprint or breaks a downstream contract your idempotency logic depends on. For teams managing dozens of integrated APIs, that means fewer mystery 422s and fewer 3am pages caused by a field that quietly changed meaning upstream.

If your integrations are complex enough that "just add an Idempotency-Key" only solves half the problem, start in a free Development environment and see what Inferrex infers about your own stack.

Sources

• The Idempotency-Key HTTP request header field (IETF draft)
• RFC 7231: HTTP/1.1 Semantics and Content
• Idempotency-Key header - HTTP | MDN
• What is Idempotency? A guide to API reliability | Google Cloud

FAQ

What is idempotent in API design?

An idempotent API call produces the same result whether it's made once or repeated multiple times with identical parameters. Under RFC 7231, GET, PUT, DELETE, HEAD, and OPTIONS are idempotent by definition, while POST and PATCH need an Idempotency-Key to gain the same guarantee.

How do I design an idempotent API?

Use HTTP methods that are natively idempotent wherever the operation allows it, and for POST or PATCH endpoints, require an Idempotency-Key header bound to method, path, principal, and payload hash. Store the first response and replay it on any duplicate.

What's the difference between atomic and idempotent?

Atomic means an operation completes entirely or not at all, with no partial state left behind. Idempotent means repeating that same operation any number of times leaves the system in the same state as running it once. An operation can be atomic without being idempotent, and vice versa.

What does idempotent mean in plain terms?

It means repetition is harmless. Pressing a lift button five times doesn't send the lift up five floors, it just confirms the request once. That's the behaviour idempotent API calls aim to replicate for retries.

Does Inferrex help with idempotency across integrated systems?

Inferrex doesn't implement idempotency keys for you, but its schema comprehension reduces the contract drift, unexpected field changes, renamed properties, that commonly causes fingerprint mismatches and broken idempotency checks in multi-system integrations.

Recommended

• The Hidden Cost of Brittle Integrations
• What’s Inside an API Schema Corpus
• Inferring API Structure — How Schema Inference Works

Made with BabyLoveGrowth to create SEO content