Fallbacks

When your primary model fails, the gateway can send the request to the next model in a list you specify, until one succeeds or the list runs out.

How It Works

Set the Cova-Fallbacks request header to a JSON array of model names, ordered from first fallback to last. When the primary model still fails after its retries, with status 401, 402, 403, 429, 500, 502, 503, 504 or 529, the gateway:

1. Rewrites the model to the next entry in the fallback chain
2. Re-resolves the provider key for the new model (BYOK first, then PTB)
3. Re-runs the wallet check if the new model will be served by a PTB key
4. Forwards the retried request upstream

This repeats for each entry in the chain until one succeeds or the chain is exhausted.

Header Format

Cova-Fallbacks is a JSON array of model name strings.

Example

Cova-Fallbacks: ["gpt-4o","gpt-4o-mini"]

Using the SDK

The @cova/gateway SDK exposes a buildHeaders() helper that constructs and validates Cova-* headers for you. Invalid input throws rather than being silently dropped.

import { buildHeaders } from "@cova/gateway";

const headers = buildHeaders({
  fallbacks: ["gpt-4o", "gpt-4o-mini"],
});

const response = await fetch("https://gateway.corevalue.dev/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.COVA_API_KEY}`,
    "Content-Type": "application/json",
    ...headers,
  },
  body: JSON.stringify({
    model: "groq/llama-3.3-70b-versatile",
    messages: [{ role: "user", content: "Hello!" }],
  }),
});

from cova_gateway import build_headers

headers = build_headers(
    fallbacks=["gpt-4o", "gpt-4o-mini"],
)

response = requests.post(
    "https://gateway.corevalue.dev/v1/chat/completions",
    headers={
        "Authorization": f"Bearer {os.environ['COVA_API_KEY']}",
        "Content-Type": "application/json",
        **headers,
    },
    json={
        "model": "groq/llama-3.3-70b-versatile",
        "messages": [{"role": "user", "content": "Hello!"}],
    },
)

Example Scenario

Your primary model is a fast, cheap Groq model. If Groq is down or rate-limited, fall back to Claude Sonnet 4.5, then to Llama on Together. Prefixed models need the /gateway/v1 base URL:

POST /gateway/v1/chat/completions HTTP/1.1
Host: gateway.corevalue.dev
Authorization: Bearer $COVA_API_KEY
Content-Type: application/json
Cova-Fallbacks: ["anthropic/claude-sonnet-4-5-20250929","together/meta-llama/Llama-3.3-70B-Instruct-Turbo"]

{
  "model": "groq/llama-3.3-70b-versatile",
  "messages": [{ "role": "user", "content": "Summarize this article." }]
}

What happens: The gateway tries groq/llama-3.3-70b-versatile first, with its retries. If it still fails, it rewrites the model to anthropic/claude-sonnet-4-5-20250929, resolves your Anthropic key, and sends it there; then the Together model. The Cova-Model response header reflects whichever model actually served the request.

Response Headers

After a fallback succeeds, the response headers tell you which model ended up serving the request:

Related