Cross-provider fallback

An alias is a project-level model name (like smart) that maps to an ordered list of targets across providers. Request the alias and the gateway tries the targets in order: on a retryable provider fault it retries, then fails over to the next target — so an OpenAI overload transparently lands on Anthropic without your code changing. This tutorial defines a two-provider alias, routes through it, and verifies the failover from the response metadata.

Fallback is static and ordered — the gateway tries targets in the configured order. There is no dynamic or latency-based routing, and a direct provider/model request (a single target) has no fallback at all.

Prerequisites

Validated OpenAI and Anthropic provider credentials in the same environment (fallback across providers needs both — see Providers & BYOK).
A secret key for the environment.

1. Define the alias

An alias resolves to its ordered targets:

{
  "alias": "smart",
  "targets": [
    { "provider": "openai", "model": "gpt-4o" },
    { "provider": "anthropic", "model": "claude-sonnet-5" }
  ]
}

Requesting smart tries openai/gpt-4o first and falls over to anthropic/claude-sonnet-5 on a retryable OpenAI fault.

Aliases are config-only today: they live in your project’s gateway configuration. There is no model-alias management API and no dashboard editor for them yet, so the alias itself is set in your project config rather than through a self-serve endpoint. Everything else in this tutorial — how it resolves, retries, and reports — is exactly what the gateway does at request time.

2. Request the alias

Send the alias as the model to the data plane at /v1/chat — exactly as a direct provider/model id would be sent. Nothing else about the call changes.

import { createClient } from './client.js';

export async function chatViaAlias(): Promise<{ text: string; fallbackUsed: boolean }> {
  const roadie = createClient();

  const res = await roadie.chat.create({
    model: 'smart',
    messages: [{ role: 'user', content: 'Say hello via my routing alias.' }],
  });

  return {
    text: res.message.content.map((part) => part.text).join(''),
    fallbackUsed: res.gateway?.fallback_used ?? false,
  };
}

curl https://gateway.roadie.paroaria.ai/v1/chat \
  -H "Authorization: Bearer $ROADIE_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "model": "smart", "messages": [{ "role": "user", "content": "Hello via alias." }] }'

3. Verify the failover from the gateway block

Every response carries a gateway block that tells you what actually happened:

{ "gateway": { "fallback_used": true, "retries": 1, "latency_ms": 1840 } }

fallback_used — whether a target other than the first served the request. true means OpenAI faulted and Anthropic answered.
retries — how many retry attempts were made before success.
latency_ms — total gateway-observed latency.

With the SDK, read res.gateway.fallback_used; over raw HTTP it is in the JSON body. This is your proof the alias failed over — no logs required.

4. What triggers a failover (and what does not)

The distinction is retryability:

Retryable faults — provider overload, 5xx, timeout. The gateway retries the current target with exponential backoff and jitter (up to 2 retries, capped at ~2 s per wait), then fails over to the next target.
Non-retryable faults — an invalid request, authentication, permission, or a content-filter block. These are terminal and do not trigger failover: a bad request would fail identically on the next provider, so there is nothing to gain.

So a smart request survives an OpenAI outage, but a malformed prompt fails fast on the first target.

5. The circuit breaker

Each (project, provider, model) target has a Redis-backed circuit breaker shared across all gateway instances. When a target’s error rate is sustained — at least 50% failures over at least 20 requests in a rolling 30-second window — the breaker opens and the gateway skips straight to the next fallback target instead of hammering a failing provider. Every 15 seconds one probe request is allowed through: a success closes the breaker, a failure resets the clock. The breaker is deliberately approximate, so a trip may land a request or two early or late under load.

6. Ineligible targets are skipped

When the alias expands, the gateway keeps only eligible targets. A target is skipped when it is not in the environment catalog, is excluded by the key’s model allowlist, lacks a capability the request needs (tools, JSON schema, vision), or has no configured provider credential. The alias still routes as long as one eligible target remains; if none do, the request fails 404 not_found_error (no_eligible_target). This is why fallback across providers needs both BYOK credentials in place — an Anthropic target with no Anthropic key is silently skipped, leaving no failover.

What you built

You have a single stable model name, smart, that routes to OpenAI first and fails over to Anthropic on a retryable fault — with retries, a shared circuit breaker, and no code that knows about providers. Your app reads gateway.fallback_used to confirm resilience, and the alias decouples your code from any one provider’s availability.

Next steps

Models, aliases & fallback

The full resolution, retry, and circuit-breaker model.

Routing & fallback guide

The task-oriented reference for configuring routes.

Observability

Find which requests failed over in your logs.

Errors & retries

Which faults are retryable, and how the SDK layers its own retries.

​Prerequisites

​1. Define the alias

​2. Request the alias

​3. Verify the failover from the gateway block

​4. What triggers a failover (and what does not)

​5. The circuit breaker

​6. Ineligible targets are skipped

​What you built

​Next steps

Models, aliases & fallback

Routing & fallback guide

Observability

Errors & retries

Prerequisites

1. Define the alias

2. Request the alias

3. Verify the failover from the gateway block

4. What triggers a failover (and what does not)

5. The circuit breaker

6. Ineligible targets are skipped

What you built

Next steps