Rate limits, quotas & budgets

Roadie enforces three kinds of usage control in the gateway, against Redis, before your provider key is ever charged. Enforcement adds at most a couple of pipelined Redis round-trips per request.

The three controls

Rate limits — requests per second / minute / day, and max concurrency. Token-bucket counters allow short bursts (a burst_multiplier) while bounding the sustained rate.
Quotas — a count per fixed calendar window (day or month) on one of these metrics: requests, input_tokens, output_tokens, total_tokens, or cost_usd. A quota is pre-checked with an estimate, then corrected with actuals by the metering pipeline.
Budgets — a spend cap in USD, enforced with a reserve/settle scheme so many concurrent requests cannot collectively blow past the cap.

Scopes and precedence

A policy can be attached at several scopes. For each dimension the gateway evaluates the most specific policy that applies, in this order:

end-user plan policy → end-user default → key → environment → project → platform plan default

Then it enforces all applicable levels — a request must pass the project, environment, key, and end-user checks. The first limit that trips rejects the request. Platform-plan defaults fill in only when no explicit policy applies at any scope.

Soft vs hard limits (the kill-switch)

Every threshold is either soft or hard:

Behavior	What happens
Soft limit	Never blocks. Crossing a threshold sends an email alert to the account owner, once per window per threshold. Use it as an early warning.
Hard limit	Rejects the request: `429` `rate_limit_error` or `quota_error`, or `402` `budget_error`, with `Retry-After` where meaningful. This is the kill-switch.
Grace	A `grace_pct` on a hard quota admits N% overage before blocking — for teams who want a soft landing rather than a hard stop.

Windows are fixed and keyed by UTC date; counters self-expire, so there is no reset job.

Alerts are delivered by email to the account owner. Roadie does not send outbound webhooks for limit or budget events.

Rate-limit response headers

Every data-plane response carries the most-constrained window’s state:

X-RateLimit-Limit
X-RateLimit-Remaining
X-RateLimit-Reset (epoch seconds)
Retry-After (on 429)

The SDK surfaces these as result.rateLimit and, on errors, err.retryAfter.

Handling limit errors

import { RateLimitError, QuotaError, BudgetError } from '@roadie/sdk';

try {
  await roadie.chat.create({ model: 'openai/gpt-4o', messages });
} catch (err) {
  if (err instanceof RateLimitError) {
    // transient — wait err.retryAfter seconds, then retry
  } else if (err instanceof QuotaError) {
    // a usage window is exhausted (429 quota_exceeded) — surface a "limit reached" state
  } else if (err instanceof BudgetError) {
    // spend cap hit (402 budget_exceeded) — alert the account owner
  }
}

The SDK automatically retries 429 responses (honoring Retry-After) up to maxRetries — and because both rate_limit_error and quota_error are 429, the SDK retries both; either one reaches your catch only after those retries are exhausted. A quota retry can’t succeed until the window resets, so a QuotaError still surfaces (just after a few short waits). budget_error is 402, so it is not retried — it surfaces immediately.

On the OpenAI-compatible surface, both quota and budget rejections are reported as insufficient_quota at 429, because the OpenAI error schema has no 402. The native surface keeps them distinct (quota_error at 429, budget_error at 402).

When Redis is unavailable

If Redis is unreachable, the gateway fails open with a degraded local limiter (conservative per-instance buckets) and pages on-call, rather than failing your customers’ requests. Budgets are temporarily unenforced for the outage window. Availability of your app outranks strict enforcement for minutes-long incidents; the event is visible in the audit log.

Reconciliation

Redis counters are the fast path; the durable truth is the metering pipeline. A reconciliation loop periodically rewrites Redis from the recorded rollups, bounding any drift. Budget/quota accounting is therefore approximately right in real time and exactly right after settlement. For a task-oriented walkthrough, see the usage-controls guide and the per-user limits tutorial.

Get started

Core concepts

Guides

SDKs

Rate limits, quotas & budgets

The three controls

Scopes and precedence

Soft vs hard limits (the kill-switch)

Rate-limit response headers

Handling limit errors

When Redis is unavailable

Reconciliation

​The three controls

​Scopes and precedence

​Soft vs hard limits (the kill-switch)

​Rate-limit response headers

​Handling limit errors

​When Redis is unavailable

​Reconciliation

The three controls

Scopes and precedence

Soft vs hard limits (the kill-switch)

Rate-limit response headers

Handling limit errors

When Redis is unavailable

Reconciliation