The three parts
Control plane
— the management API behind the dashboard. Creates orgs, projects,
environments, keys, provider credentials, and policies, and writes them to Postgres.
Data plane (gateway)
— the hot path. Authenticates the request, enforces policy, calls the
provider with your BYOK key, and streams the response.
Metering pipeline
A background worker that consumes usage events off a queue, applies privacy, and writes usage
records, request logs, and rollups.
How they stay decoupled
The control plane and the gateway never call each other synchronously. Instead:- The control plane writes configuration to Postgres and publishes a config-dirty notification on Redis.
- The gateway serves requests from an in-memory config snapshot per environment, refreshed on a short TTL (~30 s) and invalidated eagerly by that config-dirty signal. A warm environment answers entirely from memory with zero Postgres access; a cold key triggers a single bootstrap read; and if Postgres is briefly unavailable, the gateway keeps serving the last good snapshot.
The request lifecycle
A data-plane request flows through the gateway pipeline in order, deny-by-default at every step:- Authenticate — the bearer credential (secret key, publishable key, or client token) is resolved to a tenant context: org, project, environment, key scopes, and the end user. See Authentication.
- Authorize — key restrictions, then environment and project policy, then plan entitlements. Any denial is a typed error.
- Resolve the model — an alias expands to its ordered list of provider targets, or a direct
provider/modelis used. See Models & fallback. - Enforce limits — rate limits, quotas, and budgets are checked against Redis before any provider call. See Rate limits, quotas & budgets.
- Call the provider — the gateway decrypts your BYOK credential in memory and calls OpenAI or Anthropic through a unified adapter, with retries and cross-target fallback.
- Finalize — usage is metered in exactly one place, so it is recorded no matter how the
request ends. Every response carries an
X-Request-Id, tokenusage, and an estimatedcost.
The metering pipeline never blocks a request
When a request finishes, the gateway produces exactly one usage record and fans it out to two queues — one for metering, one for request logs — by appending to an in-memory buffer and returning immediately. A background flush batches and sends those events fire-and-forget. It never blocks or fails the request: a queue outage degrades to metering that catches up later, never a failed customer request. The worker consumes those events and does the durable writes:- Usage → usage records and hourly/daily rollups that power dashboards, quotas, and budgets.
- Request logs → the prompt/response log, with the privacy mode applied here, before anything is persisted, so excluded content is never written.
- Billing → metered usage rolled up for your Roadie subscription.
State stores
When Redis is unavailable, the gateway fails open
with a degraded local limiter rather than failing your customers’ requests — availability of your
app outranks strict enforcement for short incidents.
Providers
Roadie is BYOK: you bring OpenAI and Anthropic keys, and the gateway calls the providers on your behalf through a unified chat and embeddings schema. Provider-specific quirks live in per-provider adapters behind a common interface, so a single request shape works across providers and a request can fall back from one target to another when a provider fails.Where to go next
- Tenancy & isolation — the org / project / environment model the whole system is scoped by.
- Authentication & API keys — the credentials that enter the pipeline.
- Streaming — how the response half of the hot path works.
- Errors & retries — the uniform failure contract across both planes.