Retries

The AI Gateway automatically retries requests that fail with transient HTTP status codes. Retry behavior (max attempts, backoff factor, min/max delay) is configured via gateway environment variables and applies to all requests — there are no per-request retry headers.

Retry Triggers

The gateway retries requests that fail with these status codes:

• 429 — Rate limit exceeded
• 500 — Internal server error
• 502 — Bad gateway
• 503 — Service unavailable
• 504 — Gateway timeout
• 529 — Provider overload

Client errors (4xx except 429) are not retried — they typically indicate a problem with the request itself.

How It Works

The gateway uses exponential backoff:

• Starts with a short delay (configurable via env)
• Multiplies the wait time by a backoff factor after each failed attempt
• Caps the maximum wait time (configurable via env)
• Prevents overwhelming the upstream provider while maximizing success chances

Observability: X-Cova-Retry-Count

The gateway exposes the number of retry attempts on every response via the X-Cova-Retry-Count response header:

Reading the retry count

curl -i https://gateway.corevalue.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-cova-XXXXXXXXXXXXXXXX" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hi"}]}'

# Response headers include:
# X-Cova-Retry-Count: 2

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.corevalue.dev/v1",
    api_key="sk-cova-XXXXXXXXXXXXXXXX",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hi"}],
    with_response=True,
)
# response.response.headers.get("x-cova-retry-count")

import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.corevalue.dev/v1",
  apiKey: "sk-cova-XXXXXXXXXXXXXXXX",
});

const { data, response } = await client.chat.completions
  .create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Hi" }],
  })
  .withResponse();

// response.headers.get("x-cova-retry-count")

Legacy Retry Request Headers

See the Header Directory for the full list of unsupported headers.