Build a Node chat backend

This tutorial builds the shape of a real SaaS backend that proxies AI through Roadie (journey J2): your secret key stays on the server, every call is attributed to one of your end users, and you get typed errors, automatic retries, usage, cost, and a gateway requestId on every response. It uses the first-party @roadie/sdk. The finished code mirrors the runnable examples/node-chat reference app, which is exercised against a live gateway in the repo’s CI smoke gate — so what you copy here is proven against the real wire contract.

Prefer to keep your existing OpenAI code? See the OpenAI drop-in tutorial instead. This page adopts the Roadie SDK for its typed errors, retries, and normalized responses.

Prerequisites

In the dashboard:

Create a project (its development and production environments are created for you).
Add and validate a provider credential (OpenAI or Anthropic — BYOK). See Providers & BYOK.
Create a secret key (rd_sk_…) for the environment you want to call.

You will need Node 20+.

1. Scaffold the project

mkdir roadie-chat && cd roadie-chat
npm init -y
npm pkg set type=module
npm install @roadie/sdk
npm install -D tsx typescript

Put your key in a local .env (never commit it):

.env

ROADIE_KEY=rd_sk_prod_…
# Optional: point at a self-hosted or regional gateway. Omit for the hosted default.
# ROADIE_BASE_URL=https://gateway.roadie.paroaria.ai

A secret key grants full access to your project’s data plane. Keep it on the server. The SDK throws if it detects an rd_sk_… key in a browser-like environment — client apps use client tokens via a tokenProvider.

2. Construct the client once

Build the client from the environment and reuse it. Providing baseUrl is optional — omit it for the default hosted gateway ().

src/roadie.ts

import { Roadie } from '@roadie/sdk';

/** Builds a client from `ROADIE_KEY` (+ optional `ROADIE_BASE_URL`). */
export function createRoadie(): Roadie {
  return new Roadie({
    apiKey: requireEnv('ROADIE_KEY'),
    ...(process.env.ROADIE_BASE_URL !== undefined && {
      baseUrl: process.env.ROADIE_BASE_URL,
    }),
  });
}

function requireEnv(name: string): string {
  const value = process.env[name];
  if (value === undefined || value === '') {
    throw new Error(`Missing required environment variable: ${name}`);
  }
  return value;
}

3. A non-streaming completion

chat.create returns the normalized response: the assistant message, finish_reason, token usage, an estimated cost, and the gateway requestId. Attribute the call to one of your end users with user so per-user limits, quotas, and analytics apply.

src/chat.ts

import type { Roadie } from '@roadie/sdk';

const CHAT_MODEL = 'openai/gpt-4o';

export async function chatOnce(roadie: Roadie) {
  const res = await roadie.chat.create({
    model: CHAT_MODEL,
    messages: [
      { role: 'system', content: 'You are a concise assistant.' },
      { role: 'user', content: 'In one sentence, what is an AI gateway?' },
    ],
    user: { id: 'user_42', plan: 'pro' }, // your end user; optional for server-key calls
  });

  return {
    text: res.message.content.map((part) => part.text).join(''),
    usage: res.usage,
    costUsd: res.cost?.estimated_usd,
    requestId: res.requestId,
  };
}

model accepts an explicit provider/model id (like openai/gpt-4o) or a project alias (like smart) that fails over across providers — see Models, aliases & fallback.

4. Stream tokens to the caller

Iterate the async-iterable stream with for await. content_delta events carry incremental text; the terminal message_end event carries the final usage. The onDelta callback is where you forward each chunk — to stdout, a WebSocket, or an HTTP ReadableStream.

src/stream.ts

import type { Roadie, Usage } from '@roadie/sdk';

export async function streamChat(
  roadie: Roadie,
  onDelta: (delta: string) => void,
): Promise<{ text: string; usage: Usage | undefined }> {
  let text = '';
  let usage: Usage | undefined;

  for await (const event of roadie.chat.stream(
    {
      model: 'openai/gpt-4o',
      messages: [{ role: 'user', content: 'Stream a friendly one-line greeting.' }],
    },
    { timeoutMs: 60_000 },
  )) {
    if (event.type === 'content_delta') {
      text += event.delta;
      onDelta(event.delta);
    } else if (event.type === 'message_end') {
      usage = event.usage;
    }
  }

  return { text, usage };
}

Every call — streaming or not — accepts { signal, timeoutMs }; pass an AbortSignal to cancel. See Streaming.

5. Create embeddings

embeddings.create accepts a string or an array and returns one vector per input, in order.

src/embeddings.ts

import type { Roadie } from '@roadie/sdk';

export async function embed(roadie: Roadie): Promise<number[][]> {
  const res = await roadie.embeddings.create({
    model: 'openai/text-embedding-3-small',
    input: ['The quick brown fox.', 'A second document to embed.'],
  });
  return res.data.map((entry) => entry.embedding);
}

6. Handle failures by type

Every SDK failure is a RoadieError subclass carrying a stable code, the requestId, the HTTP status, and — for rate limits — retryAfter. Branch on the class and react appropriately.

src/errors.ts

import {
  AuthenticationError,
  BudgetError,
  isRoadieError,
  QuotaError,
  RateLimitError,
  type Roadie,
} from '@roadie/sdk';

export function classifyError(err: unknown): string {
  if (err instanceof RateLimitError) return 'rate_limited';      // 429 — back off, retry
  if (err instanceof QuotaError) return 'quota_exhausted';       // 429 — surface a limit state
  if (err instanceof BudgetError) return 'budget_exceeded';      // 402 — alert the account owner
  if (err instanceof AuthenticationError) return 'bad_key';      // 401 — key missing/revoked
  if (isRoadieError(err)) return `${err.type} (${err.code ?? 'unknown'})`;
  return 'unknown';
}

export async function chatSafely(roadie: Roadie) {
  try {
    const res = await roadie.chat.create({
      model: 'openai/gpt-4o',
      messages: [{ role: 'user', content: 'Hello!' }],
    });
    return { ok: true as const, text: res.message.content.map((p) => p.text).join('') };
  } catch (err) {
    return { ok: false as const, kind: classifyError(err) };
  }
}

The SDK already retries transient failures (network errors, 408/429/5xx) before they reach you, honoring Retry-After. So a RateLimitError in your catch means retries were already exhausted. See Errors & retries and the full error reference.

7. Collect feedback on a response

Use the requestId from a completion to record a thumbs-up/down — the same id shows up in your request logs, so feedback and the request line up.

src/feedback.ts

import type { Roadie } from '@roadie/sdk';

export async function thumbsUp(roadie: Roadie, requestId: string): Promise<void> {
  await roadie.feedback.submit({ requestId, rating: 'up', comment: 'Great answer.' });
}

8. Wire it into a server route

The functions above are transport-agnostic. In a Next.js App Router route you bridge the SDK’s stream onto a ReadableStream so the browser reads tokens as they arrive, while the secret key never leaves the server:

app/api/chat/route.ts

import { Roadie } from '@roadie/sdk';

export const runtime = 'nodejs';

let client: Roadie | undefined;
const roadie = () => (client ??= new Roadie({ apiKey: process.env.ROADIE_KEY! }));

export async function POST(request: Request): Promise<Response> {
  const { messages, userId } = await request.json();
  const stream = roadie().chat.stream({
    model: 'openai/gpt-4o',
    messages,
    ...(userId && { user: { id: userId } }),
  });

  const encoder = new TextEncoder();
  const body = new ReadableStream<Uint8Array>({
    async start(controller) {
      try {
        for await (const event of stream) {
          if (event.type === 'content_delta') controller.enqueue(encoder.encode(event.delta));
        }
      } catch (error) {
        controller.error(error);
        return;
      }
      controller.close();
    },
  });

  return new Response(body, { headers: { 'content-type': 'text/plain; charset=utf-8' } });
}

See the Next.js quickstart for the matching client page.

What you built

You built a server-side AI backend on @roadie/sdk: a reusable client, non-streaming and streaming chat, embeddings, feedback, and error handling that branches on typed classes — with every call attributed to one of your end users. Your secret key stays server-side, retries and idempotency are automatic, and each response carries the usage, cost, and requestId you need for billing and support.

Next steps

Per-user limits

Turn user: { id } attribution into per-user rate limits and quotas.

Cross-provider fallback

Route model: 'smart' through an alias that fails over across providers.

Observability

Read the usage, cost, and logs these calls produce.

TypeScript SDK reference

The full client surface: options, resources, streaming, and token providers.

​Prerequisites

​1. Scaffold the project

​2. Construct the client once

​3. A non-streaming completion

​4. Stream tokens to the caller

​5. Create embeddings

​6. Handle failures by type

​7. Collect feedback on a response

​8. Wire it into a server route

​What you built

​Next steps

Per-user limits

Cross-provider fallback

Observability

TypeScript SDK reference

Prerequisites

1. Scaffold the project

2. Construct the client once

3. A non-streaming completion

4. Stream tokens to the caller

5. Create embeddings

6. Handle failures by type

7. Collect feedback on a response

8. Wire it into a server route

What you built

Next steps