Call other providers

Never worry about provider outages again. The AI Gateway automatically routes your requests to the best available provider, with instant failover when things go wrong.

The Problem

The Solution

Provider routing gives you access to models across multiple providers through a single OpenAI-compatible API. When you hit rate limits or a provider has issues, the gateway automatically retries with exponential backoff and can fall back to alternative models you configure. All without setup or code changes.

Using Provider Routing

Zero configuration required for OpenAI models. Just request a model:

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello!" }]
});

That's it. The gateway automatically:

• Routes to the correct provider based on the model prefix
• Retries transient failures with exponential backoff
• Falls back to alternative models if you configure a fallback chain

Your request succeeds even when providers fail.

How It Works

The gateway routes requests based on model name prefixes. Models without a prefix are routed to OpenAI. Models with a provider prefix (e.g., groq/llama-3.3-70b-versatile) are routed to the corresponding provider.

Routing Priority:

1. Your provider keys (BYOK) if configured
2. CoreValue's managed keys (credits) - automatic fallback at 0% markup

Retries: The gateway automatically retries transient failures (429, 500, 502, 503, 504, 529) with exponential backoff and jitter. Retry behavior is configured via environment variables on the gateway, not per-request headers.

Failover: Use fallback chains (Cova-Fallbacks header) to fail over to different models when the primary model returns a retryable error.

Advanced: Customizing Routing

The default routing handles most use cases. Customize only if you need specific control:

Route to a Specific Provider

Force requests to a specific provider by prefixing the model name with the provider prefix:

model: "groq/llama-3.3-70b-versatile"  // Route through Groq

When to use: You want to use a specific provider for cost, latency, or feature reasons.

What happens: The gateway strips the prefix and forwards the request to that provider's base URL. The provider is determined by the prefix — see the Provider Prefixes table below.

Bring Your Own Keys (BYOK)

Add your provider API keys in Provider keys:

What happens: Your keys are always tried first, then CoreValue's managed keys as fallback. This gives you control over provider accounts while maintaining reliability.

Benefits: Use provider credits, meet compliance requirements, or maintain direct provider relationships while still getting automatic failover.

Retry Triggers

The gateway automatically retries when encountering these transient errors:

Retries use exponential backoff with jitter and respect the provider's Retry-After header. The X-Cova-Retry-Count response header indicates how many retries were attempted.

Provider Prefixes

The gateway recognizes 19 model prefixes. When a request model starts with one of these prefixes, the prefix is stripped and the request is forwarded to the corresponding provider's base URL.

Vercel AI Gateway

If you already use Vercel AI Gateway, put CoreValue in front of it with your Vercel key; you do not need an account with each model provider.

1. Add your Vercel AI Gateway API key in Provider keys as Vercel AI Gateway.
2. Point your client at https://gateway.corevalue.dev/gateway/v1 and prefix Vercel's model id with vercel/:

const client = new OpenAI({
  apiKey: process.env.COVA_API_KEY,
  baseURL: "https://gateway.corevalue.dev/gateway/v1",
});

const response = await client.chat.completions.create({
  model: "vercel/openai/gpt-4o-mini", // Vercel's id is openai/gpt-4o-mini
  messages: [{ role: "user", content: "Hello!" }],
});

Your Vercel account is billed as before. CoreValue records each request at the model's list price on its author's own API, the price Vercel states it charges, so vercel/openai/gpt-4o-mini costs what gpt-4o-mini costs on OpenAI. Budgets, key caps and attribution apply as for any provider. Requests through vercel/ use your own Vercel key only.

Models the gateway cannot price

Every spend control — budget gates, per-key caps, reservations, escrow — runs on an estimate of what the request will cost, and every cost you see is what the gateway priced the response at. A model id that nothing prices would pass all of them having reserved nothing and be recorded at $0.

The gateway refuses such a request instead, before it is sent, with HTTP 400 in your provider's error format:

Model "not-a-model" on groq cannot be priced by this gateway and was not sent.
List priceable models at GET /v1/models.

Every model GET /v1/models returns is one the gateway prices. Provider prefixes count as routing, so openrouter/anthropic/claude-3.5-sonnet is priced under OpenRouter's rates for that id.

In a fallback chain, a candidate that cannot be priced is skipped rather than attempted, and the chain continues with the next one.

Operators can run this gate in a shadow mode that logs what it would refuse and admits the request; see UNPRICED_MODEL_GATE_MODE in the gateway environment reference.

Fallback Chains

To fail over to another model, or to another provider by its prefix, use fallback chains with the Cova-Fallbacks header.

Real World Examples

Scenario: Provider Outage

Your production app uses a Groq model. Groq goes down at 3am.

// Your code doesn't change — the gateway retries automatically
const response = await client.chat.completions.create({
  model: "groq/llama-3.3-70b-versatile",
  messages: [{ role: "user", content: "Process this customer request" }]
});

What happens: The gateway retries with exponential backoff. If you configured a fallback chain, it falls back to the next model. Your app stays online, customers never notice.

Scenario: Using Fallback Chains

You want to try a fast Groq model first, then fall back to GPT-4o if Groq fails:

import { buildHeaders } from "@cova/gateway";

const response = await client.chat.completions.create({
  model: "groq/llama-3.3-70b-versatile",
  messages: messages,
  extraHeaders: buildHeaders({
    fallbacks: ["gpt-4o", "anthropic/claude-sonnet-4-5-20250929"],
  }),
});

What happens: Tries groq/llama-3.3-70b-versatile first, then falls back to gpt-4o, then anthropic/claude-sonnet-4-5-20250929 if each preceding model returns a retryable error.

Scenario: Using BYOK with Credits Fallback

You have your own OpenAI API key but want credits as a safety net.

// Just request the model — BYOK is tried first, then PTB credits
const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Analyze this data" }]
});

What happens: The gateway tries your BYOK key first. If your key fails or hits rate limits, it falls back to CoreValue-managed credits (PTB) at 0% markup.

Next Steps