Fallback Chains

When your primary model fails with a retryable status code, the gateway can automatically retry the request against a list of fallback models you specify. This is distinct from provider routing — fallback chains let you switch to a completely different model, not just a different provider for the same model.

How It Works

Set the Cova-Fallbacks request header to a JSON array of model names, ordered from first fallback to last. When the primary model returns a retryable status code (429, 401, 408, 500+), the gateway:

1. Rewrites the model to the next entry in the fallback chain
2. Re-resolves the provider key for the new model (BYOK first, then PTB)
3. Re-runs the wallet check if the new model will be served by a PTB key
4. Forwards the retried request upstream

This repeats for each entry in the chain until one succeeds or the chain is exhausted.

Header Format

Cova-Fallbacks is a JSON array of model name strings.

Example

Cova-Fallbacks: ["gpt-4o","claude-sonnet-4"]

Using the SDK

The @cova/gateway SDK exposes a buildHeaders() helper that constructs and validates Cova-* headers for you. Invalid input throws rather than being silently dropped.

import { buildHeaders } from "@cova/gateway";

const headers = buildHeaders({
  fallbacks: ["gpt-4o", "claude-sonnet-4"],
});

const response = await fetch("https://gateway.corevalue.dev/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.COVA_API_KEY}`,
    "Content-Type": "application/json",
    ...headers,
  },
  body: JSON.stringify({
    model: "groq/llama-3.1-70b",
    messages: [{ role: "user", content: "Hello!" }],
  }),
});

from cova_gateway import build_headers

headers = build_headers(
    fallbacks=["gpt-4o", "claude-sonnet-4"],
)

response = requests.post(
    "https://gateway.corevalue.dev/v1/chat/completions",
    headers={
        "Authorization": f"Bearer {os.environ['COVA_API_KEY']}",
        "Content-Type": "application/json",
        **headers,
    },
    json={
        "model": "groq/llama-3.1-70b",
        "messages": [{"role": "user", "content": "Hello!"}],
    },
)

Example Scenario

Your primary model is a fast, cheap Groq model. If Groq is down or rate-limited, fall back to GPT-4o, then to Claude Sonnet 4:

POST /v1/chat/completions HTTP/1.1
Host: gateway.corevalue.dev
Authorization: Bearer $COVA_API_KEY
Content-Type: application/json
Cova-Fallbacks: ["gpt-4o","claude-sonnet-4"]

{
  "model": "groq/llama-3.1-70b",
  "messages": [{ "role": "user", "content": "Summarize this article." }]
}

What happens: The gateway tries groq/llama-3.1-70b first. On a retryable error it rewrites the model to gpt-4o, re-resolves a key, and retries. If that also fails it tries claude-sonnet-4. The Cova-Model response header reflects whichever model actually served the request.

Response Headers

After a fallback succeeds, the response headers tell you which model ended up serving the request:

Related