Retries are only safe when duplicates are harmless. An Idempotency-Key header makes a non-streaming request exactly-once: the first request under a key executes and its response is stored; any later request with the same key replays that stored response instead of running again. This is what lets the SDK retry a request that may already have reached the gateway without risking a double charge. This page is about the Idempotency-Key header and its exactly-once semantics. For the SDK’s retry, backoff, and timeout behavior — and the error envelope — see Errors & retries.

​
How it works

  1. First request — the gateway atomically reserves the key (first-writer-wins), executes the request, and stores the response under the key.
  2. Replay — a later request carrying the same key returns the stored status and body without re-executing and without re-metering. Usage stays attributed to the original request, so the finalizer never runs twice and you are never double-charged. A replayed response carries an Idempotency-Replayed: true header.
  3. Conflict — reusing a key with a different request body fails with idempotency_error / idempotency_conflict (HTTP 409). The key is matched to the exact body it was first used with (via a SHA-256 body hash), so a differing body is detected rather than silently run.
The stored record is keyed on (project, key, Idempotency-Key) — not on the body — which is precisely why a differing body is a conflict instead of a fresh execution.

​
Semantics and limits

  • Window — a key is remembered for 24 hours. After that the same key is treated as new.
  • Storage — records live in Redis, scoped per project and per key.
  • Response size cap — responses larger than 1 MB are not stored; the reservation is released so a subsequent retry re-executes normally (you lose only the replay optimization, not correctness).
  • Only successes are cached — if the request errors or the client disconnects, the reservation is released so a retry may legitimately re-run.
  • The key is optional — omit the header and the request simply runs normally. There is no required format or length; any string works.

​
Concurrent requests with the same key

If a second request arrives while the first is still in flight, it waits for the first to finish (polling briefly, up to ~10 s) and then replays its response — it does not execute a duplicate. If the first writer is stuck past that window, the gateway fails open: the second request executes without a replay guarantee (and logs a warning) rather than blocking your customer. In that rare case you could see the request run twice.

​
In the SDK

The SDK attaches this automatically. For non-streaming create calls it generates a stable ULID Idempotency-Key and reuses the same key across every retry attempt, so a request that was retried after already reaching the gateway is de-duplicated:
Streaming requests never carry an Idempotency-Key — a live stream cannot be replayed. (The SDK also only retries before the first byte arrives.)

​
Sending your own key

Over raw HTTP (or any client), send the header yourself — for example to make a client-side retry of a chat request safe:
Reuse the same key when retrying the identical request; use a fresh key for a genuinely new request. Reusing a key with a changed body returns 409 idempotency_conflict.
The exactly-once replay store applies to non-streaming chat requests today (native POST /v1/chat and the OpenAI-compatible POST /v1/chat/completions). Feedback is made idempotent by a separate mechanism — a natural-key upsert on (request_id, end_user) — so submitting the same feedback twice is harmless regardless of the header.