` that signs in to CoreValue
also signs you in to the server in the same step: the consent screen says
*You will also sign in at* the provider's host. If that sign-in fails or you
cancel it, the client is still connected to CoreValue, and you connect the
server later from a link.
The connect page, **Connect *server* as *your email***, names the host you
will sign in at. Click **Continue**, sign in there and grant access. CoreValue
stores the access it grants, encrypted, and uses it only for your calls to
that server. When the page says the server is connected, return to your MCP
client and try again.
A connect link works once, for ten minutes, and only for the CoreValue account
it was made for. If it has expired or was already used, ask your MCP client
for a new one or reconnect from **Connected accounts**. If you are signed in
as someone else, the page says the link belongs to another account.
## Connected accounts [#connected-accounts]
**Gateway › MCP servers › Connected accounts** shows, under **Your accounts**,
each server you have connected: where you signed in, the scopes granted, when
you connected and last used it, and its **Status**.
* **Needs reconnecting** means the provider stopped accepting the connection —
for example, you removed CoreValue's access there. Click **Reconnect** to
sign in again. CoreValue renews access in the background otherwise, so you
do not reconnect when a token expires.
* **Disconnect** stops your calls to that server at once. Where the provider
supports it, CoreValue also asks the provider to revoke its access. Where it
does not — GitHub is one — the dialog says so: also remove CoreValue's
access in your account settings at the provider. On GitHub, that is the
OAuth app in your authorized applications.
Owners and admins also see **All connected accounts in *organisation***, with
each member's server connections, and can disconnect any of them.
### When connections end on their own [#when-connections-end-on-their-own]
* **The server changes.** Changing a server's URL, authorization server,
client or authentication choice ends every member's connection to it,
because their access was granted for the old settings. The dialog warns you
before you save. Members connect again
the next time they use the server. Changing only the scopes, the name or the
allowed tools keeps connections.
* **The server is removed.** Its connections end with it.
* **A member leaves the organisation.** Their connections can no longer be
used.
## Related [#related]
* [Connect MCP clients](/features/mcp/connect-clients): API keys, client sign-in and
Connected clients.
* [Add MCP servers](/features/mcp/servers)
* [Tool-call logging and cost](/features/mcp/tool-calls)
# Caching (https://docs.corevalue.dev/features/routing/caching)
# Response Caching [#response-caching]
Cache responses to reduce cost and latency for repeated requests. Cache is opt-in per request via `Cova-Cache-*` headers.
**Tier:**
Available on all tiers.
**Semantic cache deferred:** The exact-match cache is shipped. Semantic cache
(vector similarity) is planned behind `SEMANTIC_CACHE_ENABLED` — the flag
exists in `env.ts` but is not yet wired into the cache middleware.
## How It Works [#how-it-works]
When cache is enabled, the gateway:
1. Computes a cache key from the request body (excluding ignored keys)
2. Checks the cache bucket for a matching response
3. On hit: returns the cached response with `Cova-Cache: HIT` header
4. On miss: forwards to upstream, saves the response to the cache bucket
## Cache Request Headers [#cache-request-headers]
| Header | Purpose | Default |
| ---------------------------- | -------------------------------- | ---------- |
| `Cova-Cache-Enabled` | Enable read + write | — |
| `Cova-Cache-Save` | Enable write only | — |
| `Cova-Cache-Read` | Enable read only | — |
| `Cova-Cache-Bucket-Max-Size` | Max cached responses per key | 1 (max 20) |
| `Cova-Cache-Seed` | Cache isolation seed | — |
| `Cova-Cache-Ignore-Keys` | Body keys to ignore in cache key | — |
## Usage [#usage]
### Read + Write (Default) [#read--write-default]
```bash
curl https://gateway.corevalue.dev/v1/chat/completions \
-H "Authorization: Bearer sk-cova-..." \
-H "Content-Type: application/json" \
-H "Cova-Cache-Enabled: true" \
-d '{"model":"gpt-4o","messages":[{"role":"user","content":"What is 2+2?"}]}'
```
### Write Only (Seed Cache) [#write-only-seed-cache]
```bash
curl https://gateway.corevalue.dev/v1/chat/completions \
-H "Authorization: Bearer sk-cova-..." \
-H "Cova-Cache-Save: true" \
-d '{"model":"gpt-4o","messages":[...]}'
```
### Read Only (No Saving) [#read-only-no-saving]
```bash
curl https://gateway.corevalue.dev/v1/chat/completions \
-H "Authorization: Bearer sk-cova-..." \
-H "Cova-Cache-Read: true" \
-d '{"model":"gpt-4o","messages":[...]}'
```
## Cache Response Headers [#cache-response-headers]
On a cache hit, the gateway sets:
| Header | Purpose |
| ----------------------- | ---------------------------- |
| `Cova-Cache` | "HIT" |
| `Cova-Cache-Bucket-Idx` | Cache bucket index |
| `Cova-Cache-Latency` | Cached response latency (ms) |
## Escrow Interaction [#escrow-interaction]
When `ESCROW_ENABLED` is true, the gateway still consults the request cache after reserving escrow. On a cache hit, the escrow hold is **cancelled** (a cached response incurs no PTB cost), so the user is not charged for a cached answer. The request cache itself is never bypassed by escrow.
**Related:** Escrow for reserve-before-execute
behavior. [Cova-\* Headers Guide](/api-reference/headers) for
all headers.
# Fallbacks (https://docs.corevalue.dev/features/routing/fallbacks)
When your primary model fails, the gateway can send the request to the next model in a list you specify, until one succeeds or the list runs out.
## How It Works [#how-it-works]
Set the `Cova-Fallbacks` request header to a JSON array of model names, ordered from first fallback to last. When the primary model still fails after its retries, with status 401, 402, 403, 429, 500, 502, 503, 504 or 529, the gateway:
1. **Rewrites the model** to the next entry in the fallback chain
2. **Re-resolves the provider key** for the new model (BYOK first, then PTB)
3. **Re-runs the wallet check** if the new model will be served by a PTB key
4. **Forwards the retried request** upstream
This repeats for each entry in the chain until one succeeds or the chain is exhausted.
To fall back to a **different provider**, use the base URL `https://gateway.corevalue.dev/gateway/v1` and prefixed model names, for example `["anthropic/claude-sonnet-4-5-20250929", "groq/llama-3.3-70b-versatile"]`. On a provider's own endpoint, such as `/v1/messages`, every fallback model goes to that same provider.
## Header Format [#header-format]
`Cova-Fallbacks` is a JSON array of model name strings.
| Rule | Detail |
| -------------- | ------------------------------------------------------------------------------ |
| **Type** | Valid JSON array of strings |
| **Max length** | 5 models |
| **Non-empty** | Array must contain at least one entry |
| **Each entry** | A non-empty model name string |
| **Invalid** | Invalid JSON, empty array, or > 5 entries → header ignored (no fallback chain) |
### Example [#example]
```http
Cova-Fallbacks: ["gpt-4o","gpt-4o-mini"]
```
## Using the SDK [#using-the-sdk]
The `@cova/gateway` SDK exposes a `buildHeaders()` helper that constructs and validates `Cova-*` headers for you. Invalid input throws rather than being silently dropped.
```typescript
import { buildHeaders } from "@cova/gateway";
const headers = buildHeaders({
fallbacks: ["gpt-4o", "gpt-4o-mini"],
});
const response = await fetch("https://gateway.corevalue.dev/v1/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.COVA_API_KEY}`,
"Content-Type": "application/json",
...headers,
},
body: JSON.stringify({
model: "groq/llama-3.3-70b-versatile",
messages: [{ role: "user", content: "Hello!" }],
}),
});
```
```python
from cova_gateway import build_headers
headers = build_headers(
fallbacks=["gpt-4o", "gpt-4o-mini"],
)
response = requests.post(
"https://gateway.corevalue.dev/v1/chat/completions",
headers={
"Authorization": f"Bearer {os.environ['COVA_API_KEY']}",
"Content-Type": "application/json",
**headers,
},
json={
"model": "groq/llama-3.3-70b-versatile",
"messages": [{"role": "user", "content": "Hello!"}],
},
)
```
## Example Scenario [#example-scenario]
Your primary model is a fast, cheap Groq model. If Groq is down or rate-limited, fall back to Claude Sonnet 4.5, then to Llama on Together. Prefixed models need the `/gateway/v1` base URL:
```http
POST /gateway/v1/chat/completions HTTP/1.1
Host: gateway.corevalue.dev
Authorization: Bearer $COVA_API_KEY
Content-Type: application/json
Cova-Fallbacks: ["anthropic/claude-sonnet-4-5-20250929","together/meta-llama/Llama-3.3-70B-Instruct-Turbo"]
{
"model": "groq/llama-3.3-70b-versatile",
"messages": [{ "role": "user", "content": "Summarize this article." }]
}
```
**What happens:** The gateway tries `groq/llama-3.3-70b-versatile` first, with its retries. If it still fails, it rewrites the model to `anthropic/claude-sonnet-4-5-20250929`, resolves your Anthropic key, and sends it there; then the Together model. The `Cova-Model` response header reflects whichever model actually served the request.
## Response Headers [#response-headers]
After a fallback succeeds, the response headers tell you which model ended up serving the request:
| Header | Meaning |
| --------------- | -------------------------------------------------------------------------------- |
| `Cova-Model` | The model that actually served the request (may differ from the requested model) |
| `Cova-Provider` | The provider that served the request |
| `Cova-Status` | `success` or `error` |
## Related [#related]
Fail over across providers for the same model
Where fallback resolution fits in the 12-step pipeline
# Several keys for one provider (https://docs.corevalue.dev/features/routing/key-pool)
You can add more than one key for the same provider under **Gateway → Provider keys**. Today the gateway uses the first enabled key for every request; spreading requests across several keys, and moving to another key when one is rate-limited, is not built yet.
To keep requests working when a provider or key fails, use [retries and fallbacks](/guides/provider-failover).
# Route by model name (https://docs.corevalue.dev/features/routing/model-routing)
With the OpenAI SDK pointed at `https://gateway.corevalue.dev/v1`, a request names only a model, so it would normally go to OpenAI. **Routing by model name** lets the same request reach the provider that actually serves that model: send `claude-sonnet-4-5-20250929` to `/v1/chat/completions`, and if your organisation holds an Anthropic or Bedrock key, CoreValue sends it there and returns an OpenAI-format reply.
It is on for every organisation. There is nothing to configure.
```python
import os
from openai import OpenAI
client = OpenAI(
base_url="https://gateway.corevalue.dev/v1",
api_key=os.environ["COVA_API_KEY"],
)
# No prefix, no second SDK: routed to a provider you hold a key for.
response = client.chat.completions.create(
model="claude-sonnet-4-5-20250929",
messages=[{"role": "user", "content": "Hello"}],
)
```
## How the provider is chosen [#how-the-provider-is-chosen]
For each eligible request, CoreValue:
1. Looks the model up in its [price registry](/additional/supported-models#the-price-registry). It accepts the registry name (as [`GET /v1/models`](/api-reference/list-models) lists it) or the provider's own id for the model.
2. Keeps the providers that serve that model **and** that your organisation can reach: you have added a key for it on **Gateway** → **Provider keys**.
3. Picks the one with the lowest list price for that model.
If the chosen provider is OpenAI, nothing changes. Otherwise the request is rewritten as if you had sent it to `/gateway/v1` with that provider's prefix, for example `anthropic/claude-sonnet-4-5-20250929`, and is translated and mapped back exactly as a prefixed request would be.
The request is left on the OpenAI path, unchanged, when:
* the registry does not know the model,
* no provider you hold a key for serves it, or
* the cheapest one you hold a key for is one this route cannot address (Vertex AI).
A request left on the OpenAI path is sent to OpenAI. Without an OpenAI key, it fails with *No openai provider key available*.
## What it applies to [#what-it-applies-to]
Only `POST /v1/chat/completions` in OpenAI format. These are never rerouted:
* the Responses API (`/v1/responses`), embeddings, images, audio and the rest of the OpenAI API;
* requests to `/gateway/v1`, where you have already named the provider with a prefix;
* requests to a provider's own endpoint, such as `/v1/messages` (Anthropic), `:generateContent` (Gemini, Vertex AI) or `/model/{id}/invoke` (Bedrock).
## It is not failover [#it-is-not-failover]
Routing chooses **one** provider before the request is sent. If that provider then fails, CoreValue retries the same provider and does not try another one that serves the model. To fall over to another provider or model, add a [fallback chain](/features/routing/fallbacks) with `Cova-Fallbacks`, naming each fallback with its provider prefix, for example `["groq/llama-3.3-70b-versatile"]`.
## See where a request went [#see-where-a-request-went]
The response says which provider served it:
| Header | Routed request |
| ------------------- | --------------------------------------------------------------------------------------------------------------- |
| `Cova-Provider` | The provider chosen, for example `anthropic` or `bedrock` |
| `Cova-Model` | The model as sent to the provider, with the provider prefix, for example `anthropic/claude-sonnet-4-5-20250929` |
| `Cova-Gateway-Mode` | `translated` when the request went to Anthropic, Gemini or Bedrock |
See [Response headers](/api-reference/response-headers). On **Requests**, the request is recorded under the provider that served it.
## With key scopes and budgets [#with-key-scopes-and-budgets]
* A key's **Models** [scope](/features/access/key-scopes) is checked against the model you asked for, before routing. Its **Providers** scope is checked against the provider routing chose, so a key limited to Anthropic is refused if the request would be routed to Bedrock.
* [Budgets](/features/budgets/budgets-and-limits) and key caps are charged at the price of the provider that served the request.
## Related [#related]
* [Supported providers and models](/additional/supported-models) — every provider, prefix and native endpoint
* [Call other providers](/features/routing/other-providers) — naming the provider yourself with a prefix
* [Fallbacks](/features/routing/fallbacks) — trying another model when one fails
# Call other providers (https://docs.corevalue.dev/features/routing/other-providers)
Never worry about provider outages again. The AI Gateway automatically routes your requests to the best available provider, with instant failover when things go wrong.
## The Problem [#the-problem]
Provider downtime breaks your app and frustrates users
Hit provider quotas and block your users from accessing your service
Limited availability in certain regions reduces your global reach
Tied to one provider prevents cost optimization and flexibility
## The Solution [#the-solution]
Provider routing gives you access to models across multiple providers through a single OpenAI-compatible API. When you hit rate limits or a provider has issues, the gateway automatically retries with exponential backoff and can fall back to alternative models you configure. All without setup or code changes.
## Using Provider Routing [#using-provider-routing]
Zero configuration required for OpenAI models. Just request a model:
```typescript
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Hello!" }]
});
```
That's it. The gateway automatically:
* Routes to the correct provider based on the model prefix
* Retries transient failures with exponential backoff
* Falls back to alternative models if you configure a fallback chain
Your request succeeds even when providers fail.
## How It Works [#how-it-works]
The gateway routes requests based on model name prefixes. Models without a prefix are routed to OpenAI. Models with a provider prefix (e.g., `groq/llama-3.3-70b-versatile`) are routed to the corresponding provider.
**Routing Priority:**
1. Your provider keys (BYOK) if configured
2. CoreValue's managed keys (credits) - automatic fallback at 0% markup
**Retries:** The gateway automatically retries transient failures (429, 500, 502, 503, 504, 529) with exponential backoff and jitter. Retry behavior is configured via environment variables on the gateway, not per-request headers.
**Failover:** Use [fallback chains](/features/routing/fallbacks) (`Cova-Fallbacks` header) to fail over to different models when the primary model returns a retryable error.
Credits let you access LLM providers without signing up for each one. Add funds to your CoreValue account and we manage all the provider API keys for you. You pay exactly what providers charge (0% markup) and avoid provider rate limits. [Learn more about credits](https://corevalue.dev/credits).
## Advanced: Customizing Routing [#advanced-customizing-routing]
The default routing handles most use cases. Customize only if you need specific control:
### Route to a Specific Provider [#route-to-a-specific-provider]
Force requests to a specific provider by prefixing the model name with the provider prefix:
```typescript
model: "groq/llama-3.3-70b-versatile" // Route through Groq
```
**When to use:** You want to use a specific provider for cost, latency, or feature reasons.
**What happens:** The gateway strips the prefix and forwards the request to that provider's base URL. The provider is determined by the prefix — see the [Provider Prefixes](#provider-prefixes) table below.
### Bring Your Own Keys (BYOK) [#bring-your-own-keys-byok]
Add your provider API keys in [Provider keys](https://app.corevalue.dev/gateway?tab=provider-keys):
**What happens:** Your keys are always tried first, then CoreValue's managed keys as fallback. This gives you control over provider accounts while maintaining reliability.
**Benefits:** Use provider credits, meet compliance requirements, or maintain direct provider relationships while still getting automatic failover.
The gateway forwards **any** model/provider combination, even models not yet in our registry. Unknown models only route through your BYOK keys.
## Retry Triggers [#retry-triggers]
The gateway automatically retries when encountering these transient errors:
| Error | Description |
| ----- | ---------------------- |
| 429 | Rate limit errors |
| 500 | Internal server errors |
| 502 | Bad gateway |
| 503 | Service unavailable |
| 504 | Gateway timeout |
| 529 | Overloaded |
Retries use exponential backoff with jitter and respect the provider's `Retry-After` header. The `X-Cova-Retry-Count` response header indicates how many retries were attempted.
## Provider Prefixes [#provider-prefixes]
The gateway recognizes 19 model prefixes. When a request model starts with one of these prefixes, the prefix is stripped and the request is forwarded to the corresponding provider's base URL.
Prefixes are honoured only on the `/gateway/v1` base URL: `https://gateway.corevalue.dev/gateway/v1`. On the plain `/v1` base URL, a prefixed model such as `groq/llama-3.3-70b-versatile` is not routed by its prefix; the request goes to OpenAI.
| Prefix | Provider |
| ------------- | --------------------------------------------------- |
| `together/` | together |
| `groq/` | groq |
| `openrouter/` | openrouter |
| `fireworks/` | fireworks |
| `mistral/` | mistral |
| `perplexity/` | perplexity |
| `deepseek/` | deepseek |
| `cerebras/` | cerebras |
| `deepinfra/` | deepinfra |
| `novita/` | novita |
| `nebius/` | nebius |
| `baseten/` | baseten |
| `chutes/` | chutes |
| `xai/` | xai |
| `canopywave/` | canopywave |
| `vercel/` | Vercel AI Gateway (see [below](#vercel-ai-gateway)) |
| `anthropic/` | anthropic (native — cross-provider translation) |
| `google/` | google (native — cross-provider translation) |
| `bedrock/` | bedrock (native — cross-provider translation) |
The `anthropic/`, `google/`, and `bedrock/` prefixes route to native provider formats. When cross-provider translation is enabled, the OpenAI-format request body is translated to the native format before forwarding.
Requests that don't match a known provider prefix are routed to OpenAI (for OpenAI-format requests) or the matching native handler (for Anthropic, Google, or Bedrock-format requests). The `GATEWAY_TARGET` env var is intentionally not used as a fallback — this prevents SSRF attacks.
## Vercel AI Gateway [#vercel-ai-gateway]
If you already use [Vercel AI Gateway](https://vercel.com/docs/ai-gateway), put CoreValue in front of it with your Vercel key; you do not need an account with each model provider.
1. Add your Vercel AI Gateway API key in [Provider keys](https://app.corevalue.dev/gateway?tab=provider-keys) as **Vercel AI Gateway**.
2. Point your client at `https://gateway.corevalue.dev/gateway/v1` and prefix Vercel's model id with `vercel/`:
```typescript
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/gateway/v1",
});
const response = await client.chat.completions.create({
model: "vercel/openai/gpt-4o-mini", // Vercel's id is openai/gpt-4o-mini
messages: [{ role: "user", content: "Hello!" }],
});
```
Your Vercel account is billed as before. CoreValue records each request at the model's list price on its author's own API, the price Vercel states it charges, so `vercel/openai/gpt-4o-mini` costs what `gpt-4o-mini` costs on OpenAI. Budgets, key caps and attribution apply as for any provider. Requests through `vercel/` use your own Vercel key only.
## Models the gateway cannot price [#models-the-gateway-cannot-price]
Every spend control — budget gates, per-key caps, reservations, escrow — runs on
an estimate of what the request will cost, and every cost you see is what the
gateway priced the response at. A model id that nothing prices would pass all
of them having reserved nothing and be recorded at $0.
The gateway refuses such a request instead, before it is sent, with **HTTP
400** in your provider's error format:
```
Model "not-a-model" on groq cannot be priced by this gateway and was not sent.
List priceable models at GET /v1/models.
```
Every model `GET /v1/models` returns is one the gateway prices. Provider
prefixes count as routing, so `openrouter/anthropic/claude-3.5-sonnet` is
priced under OpenRouter's rates for that id.
In a fallback chain, a candidate that cannot be priced is skipped rather than
attempted, and the chain continues with the next one.
Operators can run this gate in a shadow mode that logs what it would refuse
and admits the request; see `UNPRICED_MODEL_GATE_MODE` in the gateway
environment reference.
## Fallback Chains [#fallback-chains]
To fail over to another model, or to another provider by its prefix, use [fallback chains](/features/routing/fallbacks) with the `Cova-Fallbacks` header.
## Real World Examples [#real-world-examples]
### Scenario: Provider Outage [#scenario-provider-outage]
Your production app uses a Groq model. Groq goes down at 3am.
```typescript
// Your code doesn't change — the gateway retries automatically
const response = await client.chat.completions.create({
model: "groq/llama-3.3-70b-versatile",
messages: [{ role: "user", content: "Process this customer request" }]
});
```
**What happens:** The gateway retries with exponential backoff. If you configured a [fallback chain](/features/routing/fallbacks), it falls back to the next model. Your app stays online, customers never notice.
### Scenario: Using Fallback Chains [#scenario-using-fallback-chains]
You want to try a fast Groq model first, then fall back to GPT-4o if Groq fails:
```typescript
import { buildHeaders } from "@cova/gateway";
const response = await client.chat.completions.create({
model: "groq/llama-3.3-70b-versatile",
messages: messages,
extraHeaders: buildHeaders({
fallbacks: ["gpt-4o", "anthropic/claude-sonnet-4-5-20250929"],
}),
});
```
**What happens:** Tries `groq/llama-3.3-70b-versatile` first, then falls back to `gpt-4o`, then `anthropic/claude-sonnet-4-5-20250929` if each preceding model returns a retryable error.
### Scenario: Using BYOK with Credits Fallback [#scenario-using-byok-with-credits-fallback]
You have your own OpenAI API key but want credits as a safety net.
```typescript
// Just request the model — BYOK is tried first, then PTB credits
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Analyze this data" }]
});
```
**What happens:** The gateway tries your BYOK key first. If your key fails or hits rate limits, it falls back to CoreValue-managed credits (PTB) at 0% markup.
## Next Steps [#next-steps]
Explore all available models and providers
Connect your provider accounts
Combine routing with managed prompts
# Streaming (https://docs.corevalue.dev/features/routing/streaming)
When you make a streaming request through the CoreValue AI Gateway, the response cannot carry the `Cova-Request-Cost` header: headers are sent before the first chunk, and the cost is only known once the stream ends. The gateway reports it in one of two places instead.
* **SSE streams** (`text/event-stream`): the gateway appends a final Server-Sent Events frame containing per-call FinOps metadata — the actual cost, the provider that served the request, your remaining credit balance, and whether cross-provider translation was applied. The `/meta` endpoint is the fallback when this frame is missing.
* **Every other stream format** — Bedrock's `application/vnd.amazon.eventstream` and Google's streaming JSON: no frame is appended, because appending text would corrupt a binary event stream or a JSON document. Read the cost from the `/meta` endpoint, using the `Cova-Id` response header.
## Final SSE Frame Format [#final-sse-frame-format]
For SSE streams only. After the upstream stream closes (including after the OpenAI-format `data: [DONE]` frame), the gateway appends one final SSE event:
```
data: {"cova":{"requestId":"...","provider":"...","model":"...","status":"success","requestCost":0.0015,"creditsRemaining":48.77,"gatewayMode":"translated","translationWarning":"logprobs"}}
```
The frame is a single SSE `data:` line terminated by `\n\n` (two newlines). The JSON payload is nested under a `"cova"` key so the SDK can detect it by checking for `"cova" in lastChunk`.
The `requestCost` field is only present when `usage.costUsd > 0`. If no usage was extracted (e.g. the upstream errored before completion), the field is **absent** and you should fall back to the `/meta` endpoint to retrieve the cost.
## CovaGatewayMeta Fields [#covagatewaymeta-fields]
The `cova` object in the final frame contains the following fields:
| Field | Type | Description |
| -------------------- | -------------------------------- | -------------------------------------------------------------------- |
| `requestId` | `string` | Gateway-generated request UUID (same as `Cova-Id` header) |
| `provider` | `string` | Provider that served the request (after fallback resolution) |
| `model` | `string` | Model that served the request (after fallback/budget-gate downgrade) |
| `status` | `"success" \| "error"` | Gateway-level status |
| `requestCost` | `number?` | Actual cost in USD (absent if no usage extracted) |
| `estimatedCost` | `number?` | Upfront worst-case cost estimate (PTB only) |
| `creditsRemaining` | `number?` | Post-request credit balance in USD (PTB only) |
| `gatewayMode` | `"passthrough" \| "translated"?` | Whether cross-provider translation was applied |
| `translationWarning` | `string?` | Comma-separated list of dropped fields (only when translated) |
## Using the SDK [#using-the-sdk]
The `@cova/gateway` SDK's `streamChat()` wraps the OpenAI streaming call and yields chunks unchanged. After `[DONE]`, it yields one final `{ cova: CovaGatewayMeta }` value with the actual cost.
```typescript
import OpenAI from "openai";
import { streamChat } from "@cova/gateway";
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/v1",
});
const stream = streamChat(client, {
model: "groq/llama-3.3-70b-versatile",
messages: [{ role: "user", content: "Hello!" }],
stream: true,
});
for await (const chunk of stream) {
if ("cova" in chunk) {
// Final frame — per-call FinOps metadata
console.log(chunk.cova.requestCost);
console.log(chunk.cova.provider);
console.log(chunk.cova.creditsRemaining);
} else {
// Normal streaming chunk — process as usual
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
}
```
```python
from openai import OpenAI
from cova_gateway import stream_chat
client = OpenAI(
api_key=os.environ["COVA_API_KEY"],
base_url="https://gateway.corevalue.dev/v1",
)
stream = stream_chat(client, model="groq/llama-3.3-70b-versatile",
messages=[{"role": "user", "content": "Hello!"}],
stream=True,
)
for chunk in stream:
if "cova" in chunk:
# Final frame — per-call FinOps metadata
print(chunk["cova"]["requestCost"])
print(chunk["cova"]["provider"])
else:
print(chunk.choices[0].delta.content or "", end="")
```
If no `cova` frame is present (e.g. the upstream errored before completion), `streamChat()` yields an empty meta. In that case, fall back to the `/meta` endpoint using the `Cova-Id` from the response headers.
## Upfront Cost Estimate (M4) [#upfront-cost-estimate-m4]
Before the stream even begins, the gateway may set the `Cova-Request-Cost-Estimate` response header (M4 milestone 1.5a). This is the upfront worst-case cost estimate — the escrow reservation amount from the wallet check.
| Header | When set |
| ---------------------------- | --------------------------------------------------------------- |
| `Cova-Request-Cost-Estimate` | Only when escrow is enabled and a PTB key may serve the request |
`Cova-Request-Cost-Estimate` is a **UX hint only** — it is not the source of truth. The actual charge is the `requestCost` field in the final SSE `cova` frame (or `Cova-Request-Cost` header for non-streaming).
## Fallback: /meta Endpoint [#fallback-meta-endpoint]
For Bedrock and Google streaming JSON responses this is the only source of the cost. For SSE streams, use it when the final frame is missing or the `requestCost` field is absent. Call it after the stream completes, with the `Cova-Id` response header as the request id:
```bash
curl https://gateway.corevalue.dev/gateway/v1/meta/550e8400-e29b-41d4-a716-446655440000 \
-H "Authorization: Bearer $COVA_API_KEY"
```
See the [Meta Endpoint](/api-reference/request-cost-lookup) page for full details on response codes and the `CovaGatewayMeta` shape.
## Related [#related]
Fallback lookup for per-request FinOps metadata
How cost is computed during async post-processing
The gatewayMode and translationWarning fields
# OpenAI Codex (https://docs.corevalue.dev/sdks/integrations/codex)
This integration uses the [AI Gateway](/), which provides a unified API for multiple LLM providers. The AI Gateway is currently in beta.
## CLI Integration [#cli-integration]
Update your `$CODEX_HOME/.codex/config.toml` file to include the CoreValue provider configuration:
`$CODEX_HOME` is typically `~/.codex` on Mac or Linux.
```plaintext config.toml
model_provider = "corevalue"
[model_providers.corevalue]
name = "CoreValue"
base_url = "https://gateway.corevalue.dev/v1"
env_key = "COVA_API_KEY"
wire_api = "chat"
```
Set the `COVA_API_KEY` environment variable:
```bash
export COVA_API_KEY=
```
Use Codex as normal. Your requests will automatically be logged to CoreValue:
```bash
# If you set model_provider in config.toml
codex "What files are in the current directory?"
# Or specify the provider explicitly
codex -c model_provider="corevalue" "What files are in the current directory?"
```
## SDK Integration [#sdk-integration]
```bash
npm install @openai/codex-sdk
```
Initialize the Codex SDK with the AI Gateway base URL:
```typescript
import { Codex } from "@openai/codex-sdk";
const codex = new Codex({
baseUrl: "https://gateway.corevalue.dev/v1",
apiKey: process.env.COVA_API_KEY,
});
const thread = codex.startThread({
model: "gpt-5" // many models supported
});
const turn = await thread.run("What files are in the current directory?");
console.log(turn.finalResponse);
console.log(turn.items);
```
The Codex SDK doesn't currently support specifying the wire API, so it will use the Responses API by default. This works with the AI Gateway with limited model and provider support. See the [Responses API documentation](/features/routing/provider-features/responses-api) for more details.
## Additional Features [#additional-features]
Once integrated with CoreValue AI Gateway, you can take advantage of:
* **Unified Observability**: Monitor all your Codex usage alongside other LLM providers
* **Cost Tracking**: Track costs across different models and providers
* **Custom Properties**: Add metadata to your requests for better organization
* **Rate Limiting**: Control usage and prevent abuse
## {strings.relatedGuides} [#stringsrelatedguides]
Learn more about CoreValue's AI Gateway and its features
Use the OpenAI Responses API format through CoreValue AI Gateway
Configure automatic routing and fallbacks for reliability
Add metadata to your requests for better tracking and organization
# DSPy (https://docs.corevalue.dev/sdks/integrations/dpsy)
## Introduction [#introduction]
[DSPy](https://dspy.ai) is a declarative framework for building modular AI software with structured code instead of brittle prompts, offering algorithms that compile AI programs into effective prompts and weights for language models across classifiers, RAG pipelines, and agent loops.
## Integration Steps [#integration-steps]
Create a `.env` file in your project.
```bash
COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX
```
{strings.installRequiredDependencies}
```bash Python
pip install dspy
```
{strings.viewRequestsInDashboard}
```python Python
import dspy
import os
from dotenv import load_dotenv
load_dotenv()
# Configure DSPy to use CoreValue AI Gateway
lm = dspy.LM(
'gpt-4o-mini', # or any other model from the CoreValue model registry
api_key=os.getenv('COVA_API_KEY'),
api_base='https://gateway.corevalue.dev/v1'
)
dspy.configure(lm=lm)
print(lm("Hello, world!"))
```
## Complete Working Examples [#complete-working-examples]
### Basic Chain of Thought [#basic-chain-of-thought]
```python Python
import dspy
import os
from dotenv import load_dotenv
load_dotenv()
# Configure CoreValue AI Gateway
lm = dspy.LM(
'gpt-4o-mini',
api_key=os.getenv('COVA_API_KEY'),
api_base='https://gateway.corevalue.dev/v1'
)
dspy.configure(lm=lm)
# Define a module
qa = dspy.ChainOfThought('question -> answer')
# Run inference
response = qa(question="How many floors are in the castle David Gregory inherited?")
print('Answer:', response.answer)
print('Reasoning:', response.reasoning)
```
### Custom Generation Configuration [#custom-generation-configuration]
Configure temperature, max\_tokens, and other parameters:
```python Python
import dspy
import os
from dotenv import load_dotenv
load_dotenv()
# Configure with custom generation parameters
lm = dspy.LM(
'gpt-4o-mini',
api_key=os.getenv('COVA_API_KEY'),
api_base='https://gateway.corevalue.dev/v1',
temperature=0.9,
max_tokens=2000
)
dspy.configure(lm=lm)
# Use with any DSPy module
predict = dspy.Predict("question -> creative_answer")
response = predict(question="Write a creative story about AI")
print(response.creative_answer)
```
### Tracking with Custom Properties [#tracking-with-custom-properties]
Add custom properties to track and filter your requests in the CoreValue dashboard:
```python Python
import dspy
import os
from dotenv import load_dotenv
load_dotenv()
# Configure with custom CoreValue headers
lm = dspy.LM(
'gpt-4o-mini',
api_key=os.getenv('COVA_API_KEY'),
api_base='https://gateway.corevalue.dev/v1',
extra_headers={
# Session tracking
'Cova-Property-Session-Id': 'dspy-example-session',
'Cova-Property-Session-Name': 'Question Answering',
# User tracking
'Cova-User-Id': 'user-123',
# Custom properties for filtering
'Cova-Property-Environment': 'production',
'Cova-Property-Module': 'chain-of-thought',
'Cova-Property-Version': '1.0.0'
}
)
dspy.configure(lm=lm)
# Use normally
qa = dspy.ChainOfThought('question -> answer')
response = qa(question="What is DSPy?")
print(response.answer)
```
## CoreValue Prompts Integration [#corevalue-prompts-integration]
Use CoreValue Prompts for centralized prompt management with DSPy signatures:
```python Python
import dspy
import os
from dotenv import load_dotenv
load_dotenv()
# Configure with prompt parameters
lm = dspy.LM(
'gpt-4o-mini',
api_key=os.getenv('COVA_API_KEY'),
api_base='https://gateway.corevalue.dev/v1',
extra_body={
'prompt_id': 'customer-support-prompt-id',
'version_id': 'version-uuid',
'environment': 'production',
'inputs': {
'customer_name': 'Sarah',
'issue_type': 'technical'
}
}
)
dspy.configure(lm=lm)
```
Learn more about [Prompts with AI Gateway](/features/routing/provider-features/prompt-caching).
## Advanced Features [#advanced-features]
### Rate Limiting [#rate-limiting]
Configure rate limits for your DSPy applications:
```python Python
lm = dspy.LM(
'gpt-4o-mini',
api_key=os.getenv('COVA_API_KEY'),
api_base='https://gateway.corevalue.dev/v1',
extra_headers={
'Cova-RateLimit-Policy': 'basic-100'
}
)
```
### Caching [#caching]
Enable intelligent caching to reduce costs:
```python Python
lm = dspy.LM(
'gpt-4o-mini',
api_key=os.getenv('COVA_API_KEY'),
api_base='https://gateway.corevalue.dev/v1',
cache=True # DSPy's built-in caching works with CoreValue
)
```
### Session Tracking for Multi-Turn Conversations [#session-tracking-for-multi-turn-conversations]
Track entire conversation flows in DSPy programs:
```python Python
import uuid
session_id = str(uuid.uuid4())
lm = dspy.LM(
'gpt-4o-mini',
api_key=os.getenv('COVA_API_KEY'),
api_base='https://gateway.corevalue.dev/v1',
extra_headers={
'Cova-Property-Session-Id': session_id,
'Cova-Property-Session-Name': 'Customer Support',
'Cova-Property-Session-Path': '/support/chat'
}
)
dspy.configure(lm=lm)
# All calls in this session will be grouped together
qa = dspy.ChainOfThought('question -> answer')
# Multiple turns
response1 = qa(question="What is your return policy?")
response2 = qa(question="How long does shipping take?")
response3 = qa(question="Do you ship internationally?")
# View the full conversation in CoreValue Sessions
```
## Related Documentation [#related-documentation]
Learn about CoreValue's AI Gateway features and capabilities
Configure intelligent routing and automatic failover
Browse all available models and providers
Version and manage prompts with CoreValue Prompts
Add metadata to track and filter your requests
Track multi-turn conversations and user sessions
Configure rate limits for your applications
Reduce costs and latency with intelligent caching
# LangChain (https://docs.corevalue.dev/sdks/integrations/langchain)
## Introduction [#introduction]
[LangChain](https://www.langchain.com/) is a popular open-source framework for building applications with large language models across Python, TypeScript, and other languages. By integrating CoreValue AI Gateway with LangChain, you can:
* **Route to different models & providers** with automatic failover through a single endpoint
* **Unified billing** with pass-through billing or bring your own keys
* **Monitor all requests** with automatic cost tracking in one dashboard
* **Stream responses** with full observability for real-time applications
This integration requires only **two changes** to your existing LangChain code - updating the base URL and API key.
## Integration Steps [#integration-steps]
Sign up at [corevalue.dev](https://www.corevalue.dev) and generate an [API key](https://app.corevalue.dev/governance?tab=keys).
You'll also need to configure your provider API keys (OpenAI, Anthropic, etc.) at [CoreValue Providers](https://app.corevalue.dev/gateway?tab=provider-keys) for BYOK (Bring Your Own Keys).
```bash
# Your CoreValue API key
export COVA_API_KEY=
```
Create a `.env` file in your project:
```bash
COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX
```
```bash TypeScript
npm install @langchain/openai @langchain/core dotenv
# or
yarn add @langchain/openai @langchain/core dotenv
```
```bash Python
pip install langchain-openai langchain-core python-dotenv
```
```typescript TypeScript
import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage, SystemMessage } from "@langchain/core/messages";
import dotenv from 'dotenv';
dotenv.config();
// Initialize ChatOpenAI with CoreValue AI Gateway
const chat = new ChatOpenAI({
model: 'gpt-4.1-mini', // many models supported
apiKey: process.env.COVA_API_KEY,
configuration: {
baseURL: "https://gateway.corevalue.dev/v1",
defaultHeaders: {
// Optional: Add custom tracking headers
// Session tracking via custom properties:
"Cova-Property-Session-Id": "my-session",
"Cova-User-Id": "user-123",
"Cova-Property-Environment": "production",
},
},
});
```
```python Python
import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from dotenv import load_dotenv
load_dotenv()
# Initialize ChatOpenAI with CoreValue AI Gateway
chat = ChatOpenAI(
model='gpt-4.1-mini', # many models supported
api_key=os.getenv('COVA_API_KEY'),
base_url="https://gateway.corevalue.dev/v1",
default_headers={
# Optional: Add custom tracking headers
'Cova-Property-Session-Id': 'my-session',
'Cova-User-Id': 'user-123',
'Cova-Property-Environment': 'production',
},
)
```
The **only changes** from a standard LangChain setup are the `apiKey`, `baseURL` (or `base_url` in Python), and optional tracking headers. Everything else stays the same!
Your existing LangChain code continues to work without any changes:
```typescript TypeScript
// Simple completion
const response = await chat.invoke([
new SystemMessage("You are a helpful assistant."),
new HumanMessage("What is the capital of France?"),
]);
console.log(response.content);
```
```python Python
# Simple completion
messages = [
SystemMessage(content="You are a helpful assistant."),
HumanMessage(content="What is the capital of France?"),
]
response = chat.invoke(messages)
print(response.content)
```
* Request/response bodies
* Latency metrics
* Token usage and costs
* Model performance analytics
* Error tracking
* Session tracking
## Migration Example [#migration-example]
Here's what migrating an existing LangChain application looks like:
### Before (Direct OpenAI) [#before-direct-openai]
```typescript
import { ChatOpenAI } from "@langchain/openai";
const chat = new ChatOpenAI({
model: 'gpt-4o-mini',
apiKey: process.env.OPENAI_API_KEY,
});
```
```python
from langchain_openai import ChatOpenAI
chat = ChatOpenAI(
model='gpt-4o-mini',
api_key=os.getenv('OPENAI_API_KEY'),
)
```
### After (CoreValue AI Gateway) [#after-corevalue-ai-gateway]
```typescript
import { ChatOpenAI } from "@langchain/openai";
const chat = new ChatOpenAI({
model: 'gpt-4.1-mini', // many models supported
apiKey: process.env.COVA_API_KEY, // Your CoreValue API key
configuration: {
baseURL: "https://gateway.corevalue.dev/v1" // Add this!
},
});
```
```python
from langchain_openai import ChatOpenAI
chat = ChatOpenAI(
model='gpt-4.1-mini', # many models supported
api_key=os.getenv('COVA_API_KEY'), # Your CoreValue API key
base_url="https://gateway.corevalue.dev/v1" # Add this!
)
```
That's it! Just two changes and you're routing through CoreValue's AI Gateway.
## Complete Working Examples [#complete-working-examples]
### Basic Example [#basic-example]
```typescript
import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage, SystemMessage } from "@langchain/core/messages";
import dotenv from 'dotenv';
dotenv.config();
const chat = new ChatOpenAI({
model: 'gpt-4.1-mini', // many models supported
apiKey: process.env.COVA_API_KEY,
configuration: {
baseURL: "https://gateway.corevalue.dev/v1",
defaultHeaders: {
// Session tracking via custom properties:
"Cova-Property-Session-Id": "langchain-example",
"Cova-User-Id": "demo-user",
},
},
});
async function main() {
console.log('🦜 Starting LangChain + CoreValue AI Gateway example...\n');
const response = await chat.invoke([
new SystemMessage("You are a helpful assistant."),
new HumanMessage("Tell me a joke about programming."),
]);
console.log('🤖 Assistant response:');
console.log(response.content);
console.log('\n✅ Completed successfully!');
}
main().catch(console.error);
```
```python
import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from dotenv import load_dotenv
load_dotenv()
chat = ChatOpenAI(
model='gpt-4.1-mini', # many models supported
api_key=os.getenv('COVA_API_KEY'),
base_url="https://gateway.corevalue.dev/v1",
default_headers={
'Cova-Property-Session-Id': 'langchain-example',
'Cova-User-Id': 'demo-user',
},
)
def main():
print('🐍 Starting LangChain + CoreValue AI Gateway example...\n')
messages = [
SystemMessage(content="You are a helpful assistant."),
HumanMessage(content="Tell me a joke about Python programming."),
]
response = chat.invoke(messages)
print('🤖 Assistant response:')
print(response.content)
print('\n✅ Completed successfully!')
if __name__ == "__main__":
main()
```
### Streaming Example [#streaming-example]
```typescript
async function streamingExample() {
console.log('\n🌊 Streaming example...\n');
const stream = await chat.stream([
new SystemMessage("You are a helpful assistant."),
new HumanMessage("Write a short story about a robot learning to code."),
]);
console.log('🤖 Assistant (streaming):');
for await (const chunk of stream) {
process.stdout.write(chunk.content as string);
}
console.log('\n\n✅ Streaming completed!');
}
streamingExample().catch(console.error);
```
```python
def streaming_example():
print('\n🌊 Streaming example...\n')
messages = [
SystemMessage(content="You are a helpful assistant."),
HumanMessage(content="Write a short story about a robot learning to code."),
]
print('🤖 Assistant (streaming):')
for chunk in chat.stream(messages):
print(chunk.content, end='', flush=True)
print('\n\n✅ Streaming completed!')
streaming_example()
```
### Multiple Models Example [#multiple-models-example]
```typescript
async function testMultipleModels() {
console.log('🚀 Testing multiple models through CoreValue AI Gateway\n');
const models = [
{ id: 'gpt-4.1-mini', name: 'OpenAI GPT-4.1 Mini' },
{ id: 'anthropic/claude-opus-4-1-20250805', name: 'Anthropic Claude Opus 4.1' },
{ id: 'google/gemini-2.5-flash-lite', name: 'Google Gemini 2.5 Flash Lite' },
];
for (const model of models) {
try {
const chat = new ChatOpenAI({
model: model.id,
apiKey: process.env.COVA_API_KEY,
configuration: {
baseURL: "https://gateway.corevalue.dev/v1",
},
});
console.log(`🤖 Testing ${model.name}... `);
const response = await chat.invoke([
new HumanMessage("Say hello in one sentence."),
]);
console.log(` Response: ${response.content}\n`);
} catch (error) {
console.error(` Error: ${error}\n`);
}
}
console.log('✅ All models tested!');
console.log('🔍 Check your dashboard: https://app.corevalue.dev/dashboard');
}
testMultipleModels().catch(console.error);
```
```python
def test_multiple_models():
print('🚀 Testing multiple models through CoreValue AI Gateway\n')
models = [
{'id': 'gpt-4.1-mini', 'name': 'OpenAI GPT-4.1 Mini'},
{'id': 'anthropic/claude-opus-4-1-20250805', 'name': 'Anthropic Claude Opus 4.1'},
{'id': 'google/gemini-2.5-flash-lite', 'name': 'Google Gemini 2.5 Flash Lite'},
]
for model in models:
try:
chat = ChatOpenAI(
model=model['id'],
api_key=os.getenv('COVA_API_KEY'),
base_url="https://gateway.corevalue.dev/v1",
)
print(f"🤖 Testing {model['name']}... ")
response = chat.invoke([
HumanMessage(content="Say hello in one sentence."),
])
print(f" Response: {response.content}\n")
except Exception as error:
print(f" Error: {error}\n")
print('✅ All models tested!')
print('🔍 Check your dashboard: https://app.corevalue.dev/dashboard')
test_multiple_models()
```
### Batch Processing Example (Python) [#batch-processing-example-python]
```python Python
def batch_example():
print('\n📦 Batch processing example...\n')
message_batches = [
[HumanMessage(content="What is Python?")],
[HumanMessage(content="What is JavaScript?")],
[HumanMessage(content="What is TypeScript?")],
]
responses = chat.batch(message_batches)
print('🤖 Batch responses:')
for i, response in enumerate(responses, 1):
print(f'\nResponse {i}: {response.content}')
print('\n✅ Batch processing completed!')
batch_example()
```
## CoreValue Prompts Integration [#corevalue-prompts-integration]
You can use CoreValue Prompts for centralized prompt management and versioning by passing parameters through `modelKwargs`:
```typescript
const chat = new ChatOpenAI({
model: 'gpt-4.1-mini',
apiKey: process.env.COVA_API_KEY,
modelKwargs: {
prompt_id: 'customer-support-prompt',
version_id: 'version-uuid',
environment: 'production',
inputs: { customer_name: 'John', issue_type: 'billing' },
},
configuration: {
baseURL: "https://gateway.corevalue.dev/v1",
},
});
```
```python
chat = ChatOpenAI(
model='gpt-4.1-mini',
api_key=os.getenv('COVA_API_KEY'),
base_url="https://gateway.corevalue.dev/v1",
model_kwargs={
'prompt_id': 'customer-support-prompt',
'version_id': 'version-uuid',
'environment': 'production',
'inputs': {'customer_name': 'John', 'issue_type': 'billing'},
},
)
```
All prompt parameters (`prompt_id`, `version_id`, `environment`, `inputs`) are optional. Learn more about [Prompts with AI Gateway](/features/routing/provider-features/prompt-caching).
## Custom Headers and Properties [#custom-headers-and-properties]
You can add custom properties to track and filter your requests:
```typescript
const chat = new ChatOpenAI({
model: 'gpt-4.1-mini',
apiKey: process.env.COVA_API_KEY,
configuration: {
baseURL: "https://gateway.corevalue.dev/v1",
defaultHeaders: {
// Session tracking
// Session tracking via custom properties:
"Cova-Property-Session-Id": "session-abc-123",
"Cova-Property-Session-Name": "Customer Support Chat",
"Cova-Property-Session-Path": "/support/chat/456",
// User tracking
"Cova-User-Id": "user-789",
// Custom properties for filtering
"Cova-Property-Environment": "production",
"Cova-Property-App-Version": "2.1.0",
"Cova-Property-Feature": "customer-support",
// Rate limiting (optional)
"Cova-RateLimit-Policy": "basic-100",
},
},
});
```
```python
chat = ChatOpenAI(
model='gpt-4.1-mini',
api_key=os.getenv('COVA_API_KEY'),
base_url="https://gateway.corevalue.dev/v1",
default_headers={
# Session tracking
'Cova-Property-Session-Id': 'session-abc-123',
'Cova-Property-Session-Name': 'Customer Support Chat',
'Cova-Property-Session-Path': '/support/chat/456',
# User tracking
'Cova-User-Id': 'user-789',
# Custom properties for filtering
'Cova-Property-Environment': 'production',
'Cova-Property-App-Version': '2.1.0',
'Cova-Property-Feature': 'customer-support',
# Rate limiting (optional)
'Cova-RateLimit-Policy': 'basic-100',
},
)
```
## Related Documentation [#related-documentation]
Learn about CoreValue's AI Gateway features and capabilities
Configure intelligent routing and automatic failover
Browse all available models and providers
Version and manage prompts with CoreValue Prompts
Add metadata to track and filter your requests
Track multi-turn conversations and user sessions
Configure rate limits for your applications
# Langfuse (https://docs.corevalue.dev/sdks/integrations/langfuse)
## Introduction [#introduction]
[Langfuse](https://langfuse.com/) is an open-source LLM observability and analytics platform that provides tracing, monitoring, and analytics for LLM applications.
This integration requires only **two changes** to your existing Langfuse code - updating the base URL and API key.
## Integration Steps [#integration-steps]
Create a `.env` file in your project:
```bash
COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX
```
```bash
pip install langfuse python-dotenv
```
Use Langfuse's OpenAI client wrapper with CoreValue's base URL:
```python
import os
from dotenv import load_dotenv
from langfuse.openai import openai
# Load environment variables
load_dotenv()
# Create an OpenAI client with CoreValue's base URL
client = openai.OpenAI(
api_key=os.getenv("COVA_API_KEY"),
base_url="https://gateway.corevalue.dev/v1"
)
```
Your existing Langfuse code continues to work without any changes:
```python
# Make a chat completion request
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me a fun fact about space."}
],
name="fun-fact-request" # Optional: Name of the generation in Langfuse
)
# Print the assistant's reply
print(response.choices[0].message.content)
```
* Request/response bodies
* Latency metrics
* Token usage and costs
* Model performance analytics
* Error tracking
* LLM traces and spans in Langfuse
* Session tracking
## Complete Working Example [#complete-working-example]
```python
#!/usr/bin/env python3
import os
from dotenv import load_dotenv
from langfuse.openai import openai
# Load environment variables
load_dotenv()
# Create an OpenAI client with CoreValue's base URL
client = openai.OpenAI(
api_key=os.getenv("COVA_API_KEY"),
base_url="https://gateway.corevalue.dev/v1"
)
# Make a chat completion request
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me a fun fact about space."}
],
name="fun-fact-request" # Optional: Name of the generation in Langfuse
)
# Print the assistant's reply
print(response.choices[0].message.content)
```
### Streaming Responses [#streaming-responses]
Langfuse supports streaming responses with full observability:
```python
# Streaming example
stream = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "user", "content": "Write a short story about a robot learning to code."}
],
stream=True,
name="streaming-story"
)
print("🤖 Assistant (streaming):")
for chunk in stream:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="", flush=True)
print("\n")
```
### Nested Example [#nested-example]
```python
import os
from dotenv import load_dotenv
from langfuse import observe
from langfuse.openai import openai
load_dotenv()
client = openai.OpenAI(
base_url="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY"),
)
@observe() # This decorator enables tracing of the function
def analyze_text(text: str):
# First LLM call: Summarize the text
summary_response = summarize_text(text)
summary = summary_response.choices[0].message.content
# Second LLM call: Analyze the sentiment of the summary
sentiment_response = analyze_sentiment(summary)
sentiment = sentiment_response.choices[0].message.content
return {
"summary": summary,
"sentiment": sentiment
}
@observe() # Nested function to be traced
def summarize_text(text: str):
return client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "You summarize texts in a concise manner."},
{"role": "user", "content": f"Summarize the following text:\n{text}"}
],
name="summarize-text"
)
@observe() # Nested function to be traced
def analyze_sentiment(summary: str):
return client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "You analyze the sentiment of texts."},
{"role": "user", "content": f"Analyze the sentiment of the following summary:\n{summary}"}
],
name="analyze-sentiment"
)
# Example usage
text_to_analyze = "OpenAI's GPT-4 model has significantly advanced the field of AI, setting new standards for language generation."
analyze_text(text_to_analyze)
```
## Related Documentation [#related-documentation]
Learn about CoreValue's AI Gateway features and capabilities
Configure intelligent routing and automatic failover
Browse all available models and providers
Add metadata to track and filter your requests
Track multi-turn conversations and user sessions
Configure rate limits for your applications
# LangGraph (https://docs.corevalue.dev/sdks/integrations/langgraph)
## Introduction [#introduction]
[LangGraph](https://www.langchain.com/langgraph) is a framework for building stateful, multi-agent applications with LLMs. The integration with CoreValue AI Gateway is nearly identical to the [LangChain integration](/sdks/integrations/langchain), with the addition of agent-specific features.
This integration requires only **two changes** to your existing LangGraph code - updating the base URL and API key. See the [LangChain AI Gateway docs](/sdks/integrations/langchain) for full feature details.
## Quick Start [#quick-start]
Follow the same setup as [LangChain AI Gateway integration](/sdks/integrations/langchain), then create your agent:
```typescript
import { ChatOpenAI } from "@langchain/openai";
import { createReactAgent } from "@langchain/langgraph/prebuilt";
import { MemorySaver } from "@langchain/langgraph";
const model = new ChatOpenAI({
model: 'gpt-4.1-mini',
apiKey: process.env.COVA_API_KEY,
configuration: {
baseURL: "https://gateway.corevalue.dev/v1",
},
});
const agent = createReactAgent({
llm: model,
tools: yourTools,
checkpointer: new MemorySaver(),
});
```
```python
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
from langgraph.checkpoint.memory import MemorySaver
model = ChatOpenAI(
model='gpt-4.1-mini',
api_key=os.getenv('COVA_API_KEY'),
base_url="https://gateway.corevalue.dev/v1",
)
agent = create_react_agent(
model,
tools=your_tools,
checkpointer=MemorySaver(),
)
```
## Migration Example [#migration-example]
### Before (Direct Provider) [#before-direct-provider]
```typescript
import { ChatOpenAI } from "@langchain/openai";
import { createReactAgent } from "@langchain/langgraph/prebuilt";
const model = new ChatOpenAI({
model: 'gpt-4o-mini',
apiKey: process.env.OPENAI_API_KEY,
});
const agent = createReactAgent({
llm: model,
tools: myTools,
});
```
```python
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
model = ChatOpenAI(
model='gpt-4o-mini',
api_key=os.getenv('OPENAI_API_KEY'),
)
agent = create_react_agent(model, tools=my_tools)
```
### After (CoreValue AI Gateway) [#after-corevalue-ai-gateway]
```typescript
import { ChatOpenAI } from "@langchain/openai";
import { createReactAgent } from "@langchain/langgraph/prebuilt";
const model = new ChatOpenAI({
model: 'gpt-4.1-mini', // many models supported
apiKey: process.env.COVA_API_KEY, // Your CoreValue API key
configuration: {
baseURL: "https://gateway.corevalue.dev/v1" // Add this!
},
});
const agent = createReactAgent({
llm: model,
tools: myTools,
});
```
```python
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
model = ChatOpenAI(
model='gpt-4.1-mini', # many models supported
api_key=os.getenv('COVA_API_KEY'), # Your CoreValue API key
base_url="https://gateway.corevalue.dev/v1" # Add this!
)
agent = create_react_agent(model, tools=my_tools)
```
## Adding Custom Headers to Agent Invocations [#adding-custom-headers-to-agent-invocations]
You can add custom properties when calling your agent with `invoke()`:
```typescript
import { HumanMessage } from "@langchain/core/messages";
import { v4 as uuidv4 } from 'uuid';
const result = await agent.invoke(
{ messages: [new HumanMessage("What is the weather in San Francisco?")] },
{
options: {
headers: {
// Session tracking via custom properties:
"Cova-Property-Session-Id": uuidv4(),
"Cova-Property-Session-Path": "/weather/query",
"Cova-Property-Query-Type": "weather",
},
},
}
);
```
```python
from langchain_core.messages import HumanMessage
import uuid
result = agent.invoke(
{"messages": [HumanMessage(content="What is the weather in San Francisco?")]},
{
"configurable": {
"headers": {
// Session tracking via custom properties:
"Cova-Property-Session-Id": str(uuid.uuid4()),
"Cova-Property-Session-Path": "/weather/query",
"Cova-Property-Query-Type": "weather",
}
}
}
)
```
## Related Documentation [#related-documentation]
Learn about CoreValue's AI Gateway features and capabilities
Configure intelligent routing and automatic failover
Browse all available models and providers
Full AI Gateway feature documentation
Track multi-turn conversations and agent workflows
Add metadata to track and filter your requests
# LiteLLM (https://docs.corevalue.dev/sdks/integrations/litellm)
## Introduction [#introduction]
[LiteLLM](https://www.litellm.ai/) is a unified interface for calling LLM APIs.
## Integration Steps [#integration-steps]
```bash
COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX
```
{strings.installRequiredDependencies}
```bash
pip install litellm python-dotenv
```
Point LiteLLM at the Cova AI Gateway by setting `api_base` to the gateway URL and using the `openai/` prefix for models:
```python
import os
from litellm import completion
from dotenv import load_dotenv
load_dotenv()
# Route through Cova AI Gateway
response = completion(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "What is the capital of France?"}],
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY")
)
print(response.choices[0].message.content)
```
## Complete Working Examples [#complete-working-examples]
### Basic Completion [#basic-completion]
```python
import os
from litellm import completion
from dotenv import load_dotenv
load_dotenv()
# Simple completion
response = completion(
model="openai/gpt-4o-mini",
messages=[{"role": "user", "content": "Tell me a fun fact about space"}],
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY")
)
print(response.choices[0].message.content)
```
### Streaming Responses [#streaming-responses]
```python
import os
from litellm import completion
from dotenv import load_dotenv
load_dotenv()
# Streaming example
response = completion(
model="openai/anthropic/claude-sonnet-4-5-20250929",
messages=[{"role": "user", "content": "Write a short story about a robot learning to paint"}],
stream=True,
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY")
)
print("🤖 Assistant (streaming):")
for chunk in response:
if hasattr(chunk.choices[0].delta, 'content') and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
print("\n")
```
### Custom Properties and Session Tracking [#custom-properties-and-session-tracking]
Add metadata to track and filter your requests:
```python
import os
from litellm import completion
from dotenv import load_dotenv
load_dotenv()
response = completion(
model="openai/gpt-4o-mini",
messages=[{"role": "user", "content": "What's the weather like?"}],
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY"),
extra_headers={
"Cova-User-Id": "user-789",
"Cova-Property-Environment": "production",
"Cova-Property-App-Version": "2.1.0",
"Cova-Property-Feature": "weather-query"
}
)
print(response.choices[0].message.content)
```
## Provider Selection and Fallback [#provider-selection-and-fallback]
Cova's AI Gateway supports automatic failover between providers. Use the `provider/model` format to select a specific provider:
```python
import os
from litellm import completion
from dotenv import load_dotenv
load_dotenv()
# Automatic routing (cheapest provider)
response = completion(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "Hello!"}],
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY")
)
# Manual provider selection (anthropic/)
response = completion(
model="openai/anthropic/claude-sonnet-4-5-20250929",
messages=[{"role": "user", "content": "Hello!"}],
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY")
)
```
For fallback chains, use the `Cova-Fallbacks` header:
```python
response = completion(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "Hello!"}],
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY"),
extra_headers={
"Cova-Fallbacks": '["anthropic/claude-sonnet-4-5-20250929"]'
}
)
```
## Advanced Features [#advanced-features]
### Caching [#caching]
Enable caching to reduce costs and latency for repeated requests:
```python
import os
from litellm import completion
from dotenv import load_dotenv
load_dotenv()
# Enable caching for this request
response = completion(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "What is 2+2?"}],
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY"),
extra_headers={
"Cova-Cache-Enabled": "true"
}
)
print(response.choices[0].message.content)
# Subsequent identical requests will be served from cache
response2 = completion(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "What is 2+2?"}],
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY"),
extra_headers={
"Cova-Cache-Enabled": "true"
}
)
print(response2.choices[0].message.content)
```
### Rate Limiting [#rate-limiting]
Apply rate limiting policies to control request rates:
```python
import os
from litellm import completion
from dotenv import load_dotenv
load_dotenv()
response = completion(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "Hello"}],
api_base="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY"),
extra_headers={
"Cova-RateLimit-Policy": "10000;w=3600"
}
)
print(response.choices[0].message.content)
```
## Related Documentation [#related-documentation]
Learn about Cova's AI Gateway features and capabilities
Configure intelligent routing and automatic failover
Browse all available models and providers
Add metadata to track and filter your requests
Track multi-turn conversations and user sessions
Configure rate limits for your applications
Reduce costs and latency with intelligent caching
Official LiteLLM documentation
# LlamaIndex (https://docs.corevalue.dev/sdks/integrations/llamaindex)
## Introduction [#introduction]
The CoreValue LLM for LlamaIndex lets you send OpenAI‑compatible requests through the CoreValue AI Gateway — no provider keys needed. Gain centralized routing, observability, and control across many models and providers.
This integration uses a dedicated LlamaIndex package: llama-index-llms-corevalue.
## Install [#install]
```bash
pip install llama-index-llms-corevalue
```
## Usage [#usage]
```python
from llama_index.llms.corevalue import CoreValue
from llama_index.llms.openai_like.base import ChatMessage
llm = CoreValue(
api_key="",
model="gpt-4o-mini", # works across providers
is_chat_model=True,
)
message: ChatMessage = ChatMessage(role="user", content="Hello world!")
response = llm.chat(messages=[message])
print(str(response))
```
### Parameters [#parameters]
* model: OpenAI‑compatible model name routed via CoreValue. See the
model registry.
* api\_base (optional): Base URL for CoreValue AI Gateway (defaults to the package’s `DEFAULT_API_BASE`). Can also be set via `COVA_API_BASE`.
* api\_key: Your CoreValue API key. You can set via constructor or `COVA_API_KEY`.
* default\_headers (optional): Add additional headers; the `Authorization: Bearer ` header is set automatically.
## Environment Variables [#environment-variables]
```bash
export COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX
# Optional override
export COVA_API_BASE=https://gateway.corevalue.dev/v1
```
## Advanced Configuration [#advanced-configuration]
```python
from llama_index.llms.corevalue import CoreValue
llm = CoreValue(
model="gpt-4.1-mini",
api_key="",
api_base="https://gateway.corevalue.dev/v1",
default_headers={
"Cova-Session-Id": "demo-session",
"Cova-User-Id": "user-123",
"Cova-Property-Environment": "production",
},
temperature=0.2,
max_tokens=256,
)
```
## Notes [#notes]
* Authentication uses your CoreValue API key; provider keys are not required when using the AI Gateway.
* All requests appear in the CoreValue dashboard with full request/response visibility and cost tracking.
* Learn more about routing and model coverage:
*
Provider routing
*
Model registry
# n8n (https://docs.corevalue.dev/sdks/integrations/n8n)
## Introduction [#introduction]
The CoreValue Chat Model is a community node for [n8n](https://n8n.io/) that provides a LangChain-compatible interface for AI workflows. Route requests to any LLM provider through the CoreValue AI Gateway.
This is an n8n community node that integrates seamlessly with n8n's AI chain functionality.
## Prerequisites [#prerequisites]
* An n8n account (see [n8n installation docs](https://docs.n8n.io/hosting/) for setup options)
* A CoreValue API key ([get one here](https://app.corevalue.dev/governance?tab=keys))
## Integration Steps [#integration-steps]
From your n8n interface:
1. Click the **user menu** (bottom left corner)
2. Select **Settings**
3. Go to **Community Nodes**
4. Click **Install a community node**
5. Enter the package name: `n8n-nodes-corevalue`
6. Click **Install**
Wait \~30 seconds for installation. The node will appear in your nodes panel.
Learn more about installing community nodes in the [n8n documentation](https://docs.n8n.io/integrations/community-nodes/installation/).
Add your CoreValue API key to n8n:
1. Go to **Settings** → **Credentials**
2. Click **Add Credential**
3. Search for "CoreValue" and select **CoreValue LLM Observability**
4. Enter your CoreValue API key
5. Click **Save**
1. Create a new workflow or open an existing one
2. Click "+" to add a node
3. Search for "CoreValue Chat Model"
4. Configure the node:
* **Credentials**: Select your saved CoreValue credentials
* **Model**: Choose any model from the [model registry](https://corevalue.dev/models) (e.g., `gpt-4.1-mini`, `anthropic/claude-3-opus-20240229`)
* **Options**: Configure temperature, max tokens, and other model parameters
The CoreValue Chat Model node outputs a LangChain-compatible model that can be used with other AI nodes in n8n.
The CoreValue Chat Model node is designed to work with n8n's AI chain functionality:
1. Connect the node to other AI nodes that accept `ai_languageModel` inputs
2. Build complex AI workflows with Chat nodes, Chain nodes, and other AI processing nodes
3. All requests are automatically logged to CoreValue
Example workflow:
Chat Input → CoreValue Chat Model → Chat Output
Open your [CoreValue dashboard](https://app.corevalue.dev/dashboard) to see:
* All workflow requests logged automatically
* Token usage and costs per request
* Response time metrics
* Full request/response bodies
* Session tracking for multi-turn conversations
* Custom properties for filtering and analysis
## Node Configuration [#node-configuration]
### Required Parameters [#required-parameters]
* **Model**: Any model supported by CoreValue AI Gateway.
Examples: `gpt-4.1-mini`, `anthropic/claude-opus-4-1-20250805`, `google/gemini-2.5-flash-lite`.
See all models in the [CoreValue's model registry](https://corevalue.dev/models)
### Model Options [#model-options]
* **Temperature** (0-2): Controls randomness in responses
* **Max Tokens**: Maximum tokens to generate
* **Top P** (0-1): Nucleus sampling parameter
* **Frequency Penalty** (-2 to 2): Reduces repetition
* **Presence Penalty** (-2 to 2): Encourages new topics
* **Response Format**: Text or JSON
* **Timeout**: Request timeout in milliseconds
* **Max Retries**: Number of retry attempts on failure
## Example Workflows [#example-workflows]
### Basic Chat Workflow [#basic-chat-workflow]
```
[Chat Input] → [CoreValue Chat Model] → [Chat Output]
```
1. Add a **Chat Input** node (triggers on user message)
2. Add the **CoreValue Chat Model** node
* Model: `gpt-4.1-mini`
* Temperature: 0.7
3. Add a **Chat Output** node to display the response
### Multi-Step AI Chain [#multi-step-ai-chain]
```
[Webhook] → [CoreValue Chat Model] → [Extract Data] → [CoreValue Chat Model] → [Response]
```
1. Receive data via webhook
2. First CoreValue Chat Model analyzes the input
3. Extract structured data
4. Second CoreValue Chat Model generates a response
5. Both requests appear in CoreValue dashboard with session tracking
### Workflow with Custom Properties [#workflow-with-custom-properties]
Configure the node with custom properties to track workflow metadata:
1. Open the **CoreValue Chat Model** node
2. Expand **CoreValue Options** → **Custom Properties**
3. Add a JSON object:
```json
{
"workflow_name": "customer-onboarding",
"environment": "production",
"version": "2.1.0"
}
```
All requests from this node will include these properties in CoreValue.
## Troubleshooting [#troubleshooting]
### Node Installation Issues [#node-installation-issues]
* **Node not appearing**: Wait 30 seconds after installation, then refresh n8n
* **Installation failed**: Check your n8n instance has internet access
* **Version conflicts**: Ensure you're running a compatible n8n version (>= 1.0)
### Authentication Errors [#authentication-errors]
* **Invalid API key**: Verify your CoreValue API key starts with `sk-cova-`
* **403 Forbidden**: Ensure your API key has write access enabled
* **Provider not configured**: Check the name of the model is exactly the [model ID expected by the gateway](https://corevalue.dev/models). If you've added your own provider keys, make sure they are correctly set in [your CoreValue dashboard](https://app.corevalue.dev/gateway?tab=provider-keys)
### Model Errors [#model-errors]
* **Model not found**: Check the exact model name at [CoreValue's model registry](https://corevalue.dev/models)
* **Model unavailable**: Verify provider access in your CoreValue account
* **Different naming**: Providers use different conventions (e.g., OpenAI uses `gpt-4o-mini`, while the gateway uses `gpt-4.1-mini`)
### Getting Help [#getting-help]
* [n8n Community Forum](https://community.n8n.io/)
* [CoreValue Documentation](https://docs.corevalue.dev)
* [GitHub Repository](https://github.com/CoreValue/n8n-nodes-corevalue)
## Related Documentation [#related-documentation]
Learn about CoreValue's AI Gateway features and capabilities
Configure intelligent routing and automatic failover
Browse all available models and providers
Explore the 12-step gateway request pipeline
Add metadata to track and filter your requests
Track multi-turn conversations and user sessions
# OpenAI Agents (https://docs.corevalue.dev/sdks/integrations/openai-agents)
## Introduction [#introduction]
[OpenAI Agents SDK](https://github.com/openai/agents) is a framework for building AI agents with tool calling, multi-step reasoning, and structured outputs.
## {strings.howToIntegrate} [#stringshowtointegrate]
{strings.generateKeyInstructions}
```js
COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX
```
```bash
npm install @openai/agents openai
# or
pip install openai-agents
```
```typescript TypeScript
import { Agent, setDefaultOpenAIClient } from "@openai/agents";
import OpenAI from "openai";
import dotenv from "dotenv";
dotenv.config();
const client = new OpenAI({
baseURL: "https://gateway.corevalue.dev/v1",
apiKey: process.env.COVA_API_KEY
});
// Set the client globally for all agents
setDefaultOpenAIClient(client);
```
```python Python
import os
from agents import set_default_openai_client
from openai import OpenAI
client = OpenAI(
base_url="https://gateway.corevalue.dev/v1",
api_key=os.getenv("COVA_API_KEY")
)
# Set the client globally for all agents
set_default_openai_client(client)
```
Your existing OpenAI Agents code continues to work without any changes:
```typescript TypeScript
import { Agent, run, tool } from "@openai/agents";
import { z } from "zod";
// Define tools
const calculator = tool({
name: "calculator",
description: "Perform basic arithmetic operations",
parameters: z.object({
operation: z.enum(["add", "subtract", "multiply", "divide"]),
a: z.number(),
b: z.number()
}),
async execute({ operation, a, b }) {
switch (operation) {
case "add":
return a + b;
case "subtract":
return a - b;
case "multiply":
return a * b;
case "divide":
if (b === 0) return "Error: Division by zero";
return a / b;
}
}
});
// Create an agent with tools
const agent = new Agent({
name: "Assistant",
instructions: "You are a helpful assistant.",
tools: [calculator],
model: "gpt-4o-mini",
});
// Run the agent
const result = await run(agent, "Multiply 2 by 2");
console.log(result.finalOutput);
```
```python Python
from agents import Agent, Runner, tool
from typing import Literal
# Define tools
@tool
def calculator(operation: Literal["add", "subtract", "multiply", "divide"], a: float, b: float) -> float | str:
"""Perform basic arithmetic operations."""
if operation == "add":
return a + b
elif operation == "subtract":
return a - b
elif operation == "multiply":
return a * b
elif operation == "divide":
if b == 0:
return "Error: Division by zero"
return a / b
# Create an agent with tools
agent = Agent(
name="Assistant",
instructions="You are a helpful assistant.",
tools=[calculator],
model="gpt-4o-mini"
)
# Run the agent
result = Runner.run_sync(agent, "Multiply 2 by 2")
print(result.final_output)
```
* Request/response bodies
* Latency metrics
* Token usage and costs
* Model performance analytics
* Tool usage tracking
* Agent reasoning steps
* Error tracking
* Session tracking
## Related Documentation [#related-documentation]
Learn about CoreValue's AI Gateway features and capabilities
Configure intelligent routing and automatic failover
Browse all available models and providers
Version and manage prompts with CoreValue Prompts
Add metadata to track and filter your requests
Track multi-turn conversations and user sessions
Configure rate limits for your applications
Monitor tool calls and function usage in your agents
# All integrations (https://docs.corevalue.dev/sdks/integrations/overview)
CoreValue AI Gateway integrates with most AI frameworks and development tools to give you access to LLM providers across 18 provider prefixes and top tier observability.
## Why Use CoreValue Integrations? [#why-use-corevalue-integrations]
Most integrations require only updating your base URL and API key - your existing code stays the same
Access GPT, Claude, Gemini, and many more models through a single endpoint
Track all requests, costs, latency, and errors in one unified dashboard
Built-in routing and fallback logic keeps your applications running
## Popular Integrations [#popular-integrations]
Most often, integrating to CoreValue is as simple as updating your `baseURL` to `"https://gateway.corevalue.dev/v1"` and adding your CoreValue API key as a header.
Works with both Python and TypeScript.
Building stateful, multi-actor applications with LLMs.
Get full observability for your LlamaIndex searches.
Route, monitor, debug, and improve your AI applications.
Use LLMs in your automation workflows with full observability.
Enhance your code generation capabilities.
## Related Documentation [#related-documentation]
Learn about CoreValue's AI Gateway features and capabilities
Configure intelligent routing and automatic failover
Browse all available models and providers
Get started with CoreValue AI Gateway in 5 minutes
# Semantic Kernel (https://docs.corevalue.dev/sdks/integrations/semantic-kernel)
## Introduction [#introduction]
[Semantic Kernel](https://learn.microsoft.com/en-us/semantic-kernel/) is Microsoft's open-source SDK for building AI agents and orchestrating LLM workflows across multiple languages (.NET, Python, Java). By integrating CoreValue AI Gateway with Semantic Kernel, you can:
* **Route to different models & providers** with automatic failover through a single endpoint
* **Unified billing** with pass-through billing or bring your own keys
* **Monitor all requests** with automatic cost tracking in one dashboard
This integration requires only **one line change** to your existing Semantic Kernel code - adding the AI Gateway endpoint.
## Integration Steps [#integration-steps]
Sign up at [corevalue.dev](https://www.corevalue.dev) and generate an [API key](https://app.corevalue.dev/governance?tab=keys).
You'll also need to configure your provider API keys (OpenAI, Anthropic, etc.) at [CoreValue Providers](https://app.corevalue.dev/gateway?tab=provider-keys) for BYOK (Bring Your Own Keys).
```bash
# Your CoreValue API key
export COVA_API_KEY=
```
Create a `.env` file in your project:
```bash
COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX
```
```csharp .NET
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using DotNetEnv;
// Load environment variables
Env.Load();
var corevalueApiKey = Environment.GetEnvironmentVariable("COVA_API_KEY");
// Create kernel builder
var builder = Kernel.CreateBuilder();
// Add OpenAI chat completion with CoreValue AI Gateway endpoint
builder.AddOpenAIChatCompletion(
modelId: "gpt-4.1-mini", // Any model from CoreValue registry
apiKey: corevalueApiKey, // Your CoreValue API key
endpoint: new Uri("https://gateway.corevalue.dev/v1") // CoreValue AI Gateway
);
var kernel = builder.Build();
```
```python Python
import semantic_kernel as sk
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
import os
# Load environment variables
corevalue_api_key = os.getenv("COVA_API_KEY")
# Create kernel
kernel = sk.Kernel()
# Add OpenAI chat completion with CoreValue AI Gateway endpoint
kernel.add_service(
OpenAIChatCompletion(
service_id="corevalue-gateway",
ai_model_id="gpt-4.1-mini", # Any model from CoreValue registry
api_key=corevalue_api_key, # Your CoreValue API key
endpoint="https://gateway.corevalue.dev/v1" # CoreValue AI Gateway
)
)
```
The **only change** from a standard Semantic Kernel setup is adding the `endpoint` parameter. Everything else stays the same!
Your existing Semantic Kernel code continues to work without any changes:
```csharp .NET
using Microsoft.SemanticKernel.ChatCompletion;
// Get the chat service
var chatService = kernel.GetRequiredService();
// Create chat history
var chatHistory = new ChatHistory();
chatHistory.AddUserMessage("What is the capital of France?");
// Get response
var response = await chatService.GetChatMessageContentAsync(chatHistory);
Console.WriteLine(response.Content);
```
```python Python
from semantic_kernel.contents import ChatHistory
# Get the chat service
chat_service = kernel.get_service("corevalue-gateway")
# Create chat history
chat_history = ChatHistory()
chat_history.add_user_message("What is the capital of France?")
# Get response
response = await chat_service.get_chat_message_content(
chat_history=chat_history
)
print(response.content)
```
All your Semantic Kernel requests are now visible in your [CoreValue dashboard](https://app.corevalue.dev/dashboard):
* Request/response bodies
* Latency metrics
* Token usage and costs
* Model performance analytics
* Error tracking
## Migration Example [#migration-example]
Here's what migrating an existing Semantic Kernel application looks like:
### Before (Direct OpenAI) [#before-direct-openai]
```csharp
var builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion(
modelId: "gpt-4o-mini",
apiKey: openAiApiKey
);
var kernel = builder.Build();
```
### After (CoreValue AI Gateway) [#after-corevalue-ai-gateway]
```csharp
var builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion(
modelId: "gpt-4.1-mini", // Use CoreValue model names
apiKey: corevalueApiKey, // Your CoreValue API key
endpoint: new Uri("https://gateway.corevalue.dev/v1") // Add this line!
);
var kernel = builder.Build();
```
That's it! Just one additional parameter and you're routing through CoreValue's AI Gateway.
## Complete Working Example [#complete-working-example]
Here's a full example that tests multiple models:
```csharp .NET
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using DotNetEnv;
// Load environment
Env.Load();
var apiKey = Environment.GetEnvironmentVariable("COVA_API_KEY");
if (string.IsNullOrEmpty(apiKey))
{
Console.WriteLine("❌ COVA_API_KEY not found in environment");
return;
}
Console.WriteLine("🚀 Testing multiple models through CoreValue AI Gateway\n");
// Test different models
await TestModel("gpt-4.1-mini", "OpenAI GPT-4.1 Mini");
await TestModel("anthropic/claude-opus-4-1-20250805", "Anthropic Claude Opus 4.1");
await TestModel("google/gemini-2.5-flash-lite", "Google Gemini 2.5 Flash Lite");
Console.WriteLine("\n✅ All models tested!");
Console.WriteLine("🔍 Check your dashboard: https://app.corevalue.dev/dashboard");
async Task TestModel(string modelId, string modelName)
{
try
{
var builder = Kernel.CreateBuilder();
// Configure with CoreValue AI Gateway
builder.AddOpenAIChatCompletion(
modelId: modelId,
apiKey: apiKey,
endpoint: new Uri("https://gateway.corevalue.dev/v1")
);
var kernel = builder.Build();
var chatService = kernel.GetRequiredService();
var chatHistory = new ChatHistory();
chatHistory.AddUserMessage("Say hello in one sentence.");
Console.Write($"🤖 Testing {modelName}... ");
var response = await chatService.GetChatMessageContentAsync(chatHistory);
Console.WriteLine("✅");
Console.WriteLine($" Response: {response.Content}\n");
}
catch (Exception ex)
{
Console.WriteLine("❌");
Console.WriteLine($" Error: {ex.Message}\n");
}
}
```
```python Python
import semantic_kernel as sk
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
from semantic_kernel.contents import ChatHistory
import os
import asyncio
# Load environment
corevalue_api_key = os.getenv("COVA_API_KEY")
if not corevalue_api_key:
print("❌ COVA_API_KEY not found in environment")
exit(1)
print("🚀 Testing multiple models through CoreValue AI Gateway\n")
async def test_model(model_id: str, model_name: str):
try:
# Create kernel
kernel = sk.Kernel()
# Configure with CoreValue AI Gateway
kernel.add_service(
OpenAIChatCompletion(
service_id="corevalue-gateway",
ai_model_id=model_id,
api_key=corevalue_api_key,
endpoint="https://gateway.corevalue.dev/v1"
)
)
chat_service = kernel.get_service("corevalue-gateway")
chat_history = ChatHistory()
chat_history.add_user_message("Say hello in one sentence.")
print(f"🤖 Testing {model_name}... ", end="")
response = await chat_service.get_chat_message_content(
chat_history=chat_history
)
print("✅")
print(f" Response: {response.content}\n")
except Exception as ex:
print("❌")
print(f" Error: {str(ex)}\n")
async def main():
# Test different models
await test_model("gpt-4.1-mini", "OpenAI GPT-4.1 Mini")
await test_model("anthropic/claude-opus-4-1-20250805", "Anthropic Claude Opus 4.1")
await test_model("google/gemini-2.5-flash-lite", "Google Gemini 2.5 Flash Lite")
print("\n✅ All models tested!")
print("🔍 Check your dashboard: https://app.corevalue.dev/dashboard")
if __name__ == "__main__":
asyncio.run(main())
```
## Related Documentation [#related-documentation]
Learn about CoreValue's AI Gateway features and capabilities
Configure intelligent routing and automatic failover
Browse all available models and providers
Add metadata to track and filter your requests
# Vercel AI SDK (https://docs.corevalue.dev/sdks/integrations/vercel-ai-sdk)
## Introduction [#introduction]
The [Vercel AI SDK](https://ai-sdk.dev) (`ai`) works with CoreValue through its OpenAI provider: point
the provider at the CoreValue gateway and every call is metered, budgeted and attributed. Headers you
add tell CoreValue which customer, feature or end user a call was for, which is what cost per customer
and the other unit-economics reports are built from.
## Set up [#set-up]
Sign in at [app.corevalue.dev](https://app.corevalue.dev/signin). Create a key under
[Governance › Keys](https://app.corevalue.dev/governance?tab=keys), and add the key for each
provider you call under [Gateway › Provider keys](https://app.corevalue.dev/gateway?tab=provider-keys).
```bash
COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX
```
```bash npm
npm install ai @ai-sdk/openai
```
```bash pnpm
pnpm add ai @ai-sdk/openai
```
```typescript
import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";
const cova = createOpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/gateway/v1",
});
const { text } = await generateText({
model: cova.chat("gpt-4o-mini"),
prompt: "Write a haiku about cost per customer.",
});
```
**Use `cova.chat(...)`.** Since AI SDK 5, calling the provider directly, as `cova("gpt-4o-mini")`,
uses OpenAI's Responses API ([AI SDK OpenAI provider](https://ai-sdk.dev/providers/ai-sdk-providers/openai)).
`cova.chat(...)` uses Chat Completions, which every provider and model prefix supports. For the
Responses API itself, see [Responses API](/features/routing/provider-features/responses-api).
### Which base URL [#which-base-url]
| Base URL | Use it for |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `https://gateway.corevalue.dev/gateway/v1` | Any model, with or without a provider prefix: `gpt-4o-mini`, `groq/llama-3.3-70b-versatile`, `vercel/openai/gpt-4o-mini`. **Use this one.** |
| `https://gateway.corevalue.dev/v1` | OpenAI models only. A prefixed model here is **not** routed by its prefix; the request goes to OpenAI. |
The prefixes are listed in [Call other providers](/features/routing/other-providers#provider-prefixes).
## Attribute cost to customers and features [#attribute-cost-to-customers-and-features]
Add CoreValue headers to the provider, where they go on every request, or to a single call. A call's
headers are sent together with the provider's.
```typescript
import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";
const cova = createOpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/gateway/v1",
headers: {
"Cova-Feature-Id": "support-assistant", // the same for every call from this client
},
});
const { text } = await generateText({
model: cova.chat("gpt-4o-mini"),
prompt: question,
headers: {
"Cova-Customer-Id": account.id, // who this call was for
"Cova-User-Id": user.id, // the person using your product
},
});
```
| Header | What CoreValue does with it |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `Cova-Customer-Id` | Cost per customer: [unit economics](/features/reports/unit-economics), customer budgets and the margin floor, chargeback |
| `Cova-Feature-Id` | Cost per feature, on the [Spend](/features/cost-tracking/spend) page |
| `Cova-User-Id` | Requests and spend per end user. Attribution only: no budget is set on it |
| `Cova-Property-` | Any label you choose, for filtering requests, e.g. `Cova-Property-Env: production` |
The team is not a header: it comes from the CoreValue API key that made the call. Every header is
described in the [Headers](/api-reference/headers) reference.
## Route through Vercel AI Gateway [#route-through-vercel-ai-gateway]
If you buy models through [Vercel AI Gateway](https://vercel.com/docs/ai-gateway) and have no
account with each provider, keep doing so and put CoreValue in front of it:
1. Add your Vercel AI Gateway API key under
[Gateway › Provider keys](https://app.corevalue.dev/gateway?tab=provider-keys) as **Vercel AI Gateway**.
2. Prefix Vercel's model id with `vercel/`, on the `/gateway/v1` base URL.
```typescript
const { text } = await generateText({
model: cova.chat("vercel/openai/gpt-4o-mini"), // Vercel's id is openai/gpt-4o-mini
prompt: question,
headers: { "Cova-Customer-Id": account.id },
});
```
Vercel still bills you. CoreValue records each request at the model's list price on its author's own
API, which is what Vercel charges, and applies your budgets, key caps and attribution as for any
provider. More in [Call other providers](/features/routing/other-providers#vercel-ai-gateway).
**Do not use the AI SDK's own Vercel AI Gateway provider** (`@ai-sdk/gateway`, the `gateway(...)`
model helper) for these calls. It sends requests straight to Vercel, so CoreValue never sees them:
no cost, no budgets, no attribution. Use the OpenAI provider pointed at CoreValue, as above.
## Examples [#examples]
### Streaming [#streaming]
Streamed calls are metered and attributed the same way.
```typescript
import { streamText } from "ai";
const result = streamText({
model: cova.chat("gpt-4o-mini"),
prompt: "Write a short story about a robot learning to paint.",
maxOutputTokens: 300,
headers: { "Cova-Customer-Id": account.id },
});
for await (const chunk of result.textStream) {
process.stdout.write(chunk);
}
```
In a Next.js route handler, return `result.toUIMessageStreamResponse()` as usual.
### Another provider [#another-provider]
```typescript
const { text } = await generateText({
model: cova.chat("anthropic/claude-sonnet-4-5-20250929"),
prompt: "Explain quantum computing in two sentences.",
});
```
The provider's key must be saved under
[Gateway › Provider keys](https://app.corevalue.dev/gateway?tab=provider-keys).
### Tool calling [#tool-calling]
```typescript
import { generateText, tool } from "ai";
import { z } from "zod";
const result = await generateText({
model: cova.chat("gpt-4o"),
prompt: "What is the weather like in San Francisco?",
tools: {
getWeather: tool({
description: "Get the weather for a location",
inputSchema: z.object({ location: z.string().describe("The city name") }),
execute: async ({ location }) => `It's sunny in ${location}`,
}),
},
});
```
## Related [#related]
Provider prefixes, including Vercel AI Gateway
Customers, features, end users and properties
Every Cova-\* header, request and response
Serve repeated requests from cache
# Xcode (https://docs.corevalue.dev/sdks/integrations/xcode)
This guide shows how to add CoreValue as a model provider in Xcode so your chats route through the CoreValue AI Gateway and show up in your CoreValue dashboard.
## Prerequisites [#prerequisites]
* A CoreValue account and API key
* Org/provider keys configured in CoreValue (so models can be listed)
## Steps [#steps]
1. Open Xcode Settings
* Xcode → Settings…
2. Add CoreValue as a model provider
* Select the Intelligence tab
* Click "Add a model provider…"
* Fill the form with:
* URL: `https://gateway.corevalue.dev`
* API Key: `Bearer `
* API Key Header: `Authorization`
* Description: `CoreValue` (you can name this however you like)
3. Confirm models are available
* After saving, Xcode should list available models from CoreValue
* There are many models; use Favorites to pin the ones you use most
4. Start chatting and view logs in CoreValue
* Use the chat in Xcode with your selected model
* Open the CoreValue dashboard to see your requests, tokens, and costs
5. Switch chat model
* In the chat widget, press the dropdown to select a new model.
## Notes [#notes]
* URL points to the CoreValue AI Gateway. Your CoreValue API key is sent via the `Authorization` header.
* If you don’t see models, verify your org/provider keys are set in CoreValue and that your key has access.
# Zapier (https://docs.corevalue.dev/sdks/integrations/zapier)
## Introduction [#introduction]
Use the CoreValue app on Zapier to generate chat completions from any supported model through the CoreValue AI Gateway. Provide your CoreValue API key, select a model, and pass your prompt data from any Zapier trigger — all with full observability in CoreValue.
The Zapier action uses CoreValue’s OpenAI-compatible Chat Completions API. No OpenAI or third‑party provider keys are required when using the AI Gateway.
## Integration Steps [#integration-steps]
Create a CoreValue account at corevalue.dev and generate an
API key.
You can use a write-only key for logging requests. Learn more about
gateway authentication.
* In Zapier, create a new Zap (or edit an existing one).
* For the Action step, search for and select "CoreValue".
* Choose the "Chat Completion" action.
* When prompted, connect your CoreValue account and paste your CoreValue API key.
* Model: pick any supported model (see model registry).
* Messages: map your trigger fields (e.g., prompt text) into the user message.
The action routes through CoreValue’s AI Gateway. You can later change the target provider or model in CoreValue without updating your Zap.
* Run a test to see the model’s response.
* Publish the Zap when you’re satisfied.
Open your CoreValue dashboard and check the Requests tab to see your Zapier‑originated requests, costs, and latencies.
## Example Use Cases [#example-use-cases]
* Auto‑draft replies from form submissions or support tickets
* Summarize new rows in Google Sheets or Airtable
* Generate product descriptions from e‑commerce triggers
## Alternate Use cases [#alternate-use-cases]
In case you want to utilize the observability side of CoreValue (rather than just the gateway), we've got you covered!
This integration has searches and other creation actions that you could otherwise do in the CoreValue Dashboard
It also has a blank curl request, in the event that you don't find an action you want. Create what you need!
## Troubleshooting [#troubleshooting]
* Ensure your CoreValue API key is valid and has write access.
* If a request fails, review the error in the Zap run details and the corresponding request in CoreValue for provider/model‑specific messages.
* To switch models later, update the model field in the Zap action or use CoreValue routing/policies to control traffic centrally.
# Context editing (https://docs.corevalue.dev/features/routing/provider-features/context-editing)
Context editing enables automatic management of conversation context by intelligently clearing old tool uses and thinking blocks. This can greatly reduce costs in long-running sessions with minimal tradeoffs in context performance.
Context editing is currently supported for **Anthropic models only**. The configuration is ignored when routing to other providers.
## Why Context Editing [#why-context-editing]
Automatically clear old tool results before hitting context limits
Keep only relevant context, reducing input tokens on subsequent calls
Run AI agents for longer periods without manual context management
***
## Quick Start [#quick-start]
Enable context editing with a simple configuration. The AI Gateway handles the translation to Anthropic's native format.
```typescript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/v1",
});
const response = await client.chat.completions.create({
model: "anthropic/claude-sonnet-4-20250514",
messages: [
{ role: "system", content: "You are a helpful coding assistant." },
{ role: "user", content: "Help me debug this application..." }
// ... many tool calls and responses
],
tools: [/* your tools */],
context_editing: {
enabled: true
}
});
```
```python
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ.get("COVA_API_KEY"),
base_url="https://gateway.corevalue.dev/v1",
)
response = client.chat.completions.create(
model="claude-sonnet-4-20250514",
messages=[
{"role": "system", "content": "You are a helpful coding assistant."},
{"role": "user", "content": "Help me debug this application..."}
# ... many tool calls and responses
],
tools=[# your tools],
context_editing={
"enabled": True
}
)
```
```bash
curl https://gateway.corevalue.dev/v1/chat/completions \
-H "Authorization: Bearer $COVA_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-4-20250514",
"messages": [
{"role": "system", "content": "You are a helpful coding assistant."},
{"role": "user", "content": "Help me debug this application..."}
],
"tools": [],
"context_editing": {
"enabled": true
}
}'
```
***
## Configuration Options [#configuration-options]
The `context_editing` object supports two strategies for managing context:
### Clear Tool Uses [#clear-tool-uses]
Automatically clear old tool use results when context grows too large:
```typescript
context_editing: {
enabled: true,
clear_tool_uses: {
// Trigger clearing when input tokens exceed this threshold
trigger: 100000,
// Keep the most recent N tool uses
keep: 5,
// Ensure at least this many tokens are cleared
clear_at_least: 20000,
// Never clear results from these tools
exclude_tools: ["get_user_preferences", "read_config"],
// Clear tool inputs (arguments) but keep outputs
clear_tool_inputs: true
}
}
```
| Parameter | Type | Description |
| ------------------- | --------- | --------------------------------------- |
| `trigger` | number | Token threshold to trigger clearing |
| `keep` | number | Number of recent tool uses to preserve |
| `clear_at_least` | number | Minimum tokens to clear when triggered |
| `exclude_tools` | string\[] | Tool names that should never be cleared |
| `clear_tool_inputs` | boolean | Clear tool inputs while keeping outputs |
### Clear Thinking [#clear-thinking]
Manage thinking/reasoning blocks in multi-turn conversations:
```typescript
context_editing: {
enabled: true,
clear_thinking: {
// Keep the N most recent thinking turns, or "all" to keep everything
keep: 3
}
}
```
| Parameter | Type | Description |
| --------- | --------------- | ------------------------------------------ |
| `keep` | number \| "all" | Number of thinking turns to keep, or "all" |
***
## Complete Example [#complete-example]
Here's a full configuration for a long-running coding agent:
```typescript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/v1",
});
const response = await client.chat.completions.create({
model: "anthropic/claude-sonnet-4-20250514",
messages: conversationHistory,
tools: [
{
type: "function",
function: {
name: "read_file",
description: "Read a file from the filesystem",
parameters: {
type: "object",
properties: {
path: { type: "string", description: "File path to read" }
},
required: ["path"]
}
}
},
{
type: "function",
function: {
name: "write_file",
description: "Write content to a file",
parameters: {
type: "object",
properties: {
path: { type: "string" },
content: { type: "string" }
},
required: ["path", "content"]
}
}
},
{
type: "function",
function: {
name: "run_command",
description: "Execute a shell command",
parameters: {
type: "object",
properties: {
command: { type: "string" }
},
required: ["command"]
}
}
}
],
reasoning_effort: "medium",
context_editing: {
enabled: true,
clear_tool_uses: {
trigger: 150000, // Trigger at 150k tokens
keep: 10, // Keep last 10 tool uses
clear_at_least: 50000, // Clear at least 50k tokens
exclude_tools: ["read_file"], // Always keep file reads
clear_tool_inputs: true // Clear large file contents from inputs
},
clear_thinking: {
keep: 5 // Keep last 5 thinking blocks
}
},
max_completion_tokens: 16000
});
```
```python
response = client.chat.completions.create(
model="claude-sonnet-4-20250514",
messages=conversation_history,
tools=[
{
"type": "function",
"function": {
"name": "read_file",
"description": "Read a file from the filesystem",
"parameters": {
"type": "object",
"properties": {
"path": {"type": "string", "description": "File path to read"}
},
"required": ["path"]
}
}
},
{
"type": "function",
"function": {
"name": "write_file",
"description": "Write content to a file",
"parameters": {
"type": "object",
"properties": {
"path": {"type": "string"},
"content": {"type": "string"}
},
"required": ["path", "content"]
}
}
}
],
reasoning_effort="medium",
context_editing={
"enabled": True,
"clear_tool_uses": {
"trigger": 150000,
"keep": 10,
"clear_at_least": 50000,
"exclude_tools": ["read_file"],
"clear_tool_inputs": True
},
"clear_thinking": {
"keep": 5
}
},
max_completion_tokens=16000
)
```
```bash
curl https://gateway.corevalue.dev/v1/chat/completions \
-H "Authorization: Bearer $COVA_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-4-20250514",
"messages": [...],
"tools": [...],
"reasoning_effort": "medium",
"context_editing": {
"enabled": true,
"clear_tool_uses": {
"trigger": 150000,
"keep": 10,
"clear_at_least": 50000,
"exclude_tools": ["read_file"],
"clear_tool_inputs": true
},
"clear_thinking": {
"keep": 5
}
},
"max_completion_tokens": 16000
}'
```
***
## Responses API Support [#responses-api-support]
Context editing works with both the Chat Completions API and the [Responses API](/features/routing/provider-features/responses-api):
```typescript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/v1",
});
const response = await client.responses.create({
model: "anthropic/claude-sonnet-4-20250514",
input: conversationInput,
tools: [/* your tools */],
context_editing: {
enabled: true,
clear_tool_uses: {
trigger: 100000,
keep: 5
}
}
});
```
***
## Default Behavior [#default-behavior]
When `context_editing.enabled` is `true` but no specific strategies are provided, the AI Gateway uses sensible defaults:
```typescript
// Minimal configuration
context_editing: {
enabled: true
}
// Equivalent to
context_editing: {
enabled: true,
clear_tool_uses: {} // Uses Anthropic defaults
}
```
***
## Related Features [#related-features]
* [Reasoning](/features/routing/provider-features/reasoning) - Extended thinking that benefits from context editing
* [Prompt Caching](/features/routing/provider-features/prompt-caching) - Cache static context for cost savings
* Sessions - Track and analyze long-running agent sessions
Anthropic Context Editing Documentation
# Image generation (https://docs.corevalue.dev/features/routing/provider-features/image-generation)
CoreValue's AI Gateway supports image generation through models with native image output capabilities. Use the unified OpenAI-compatible API to generate images - the Gateway handles provider-specific translations automatically.
Image generation is currently supported for **Nano Banana Pro (gemini-3-pro-image-preview)** via Google AI Studio. Support for additional providers will be added in future updates.
***
## Quick Start [#quick-start]
```typescript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/v1",
});
const response = await client.chat.completions.create({
model: "google/gemini-3-pro-image-preview",
messages: [
{ role: "user", content: "Generate an image of a sunset over mountains" }
],
max_tokens: 8192
});
// Access generated images
const images = response.choices[0].message.images;
```
```typescript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/v1",
});
const response = await client.responses.create({
model: "google/gemini-3-pro-image-preview",
input: "Generate an image of a sunset over mountains",
max_output_tokens: 8192
});
// Access generated images from output
const messageOutput = response.output.find(item => item.type === "message");
const imageContent = messageOutput?.content.find(c => c.type === "output_image");
```
***
## Configuration [#configuration]
To enable image generation:
1. Set the `model` to one that supports image output (currently `google/gemini-3-pro-image-preview`, also known as Nano Banana Pro)
2. Optionally configure `image_generation` to control aspect ratio and size
```typescript
{
model: "google/gemini-3-pro-image-preview",
messages: [...],
image_generation: {
aspect_ratio: "16:9",
image_size: "2K"
}
}
```
```typescript
{
model: "google/gemini-3-pro-image-preview",
input: "...",
image_generation: {
aspect_ratio: "16:9",
image_size: "2K"
}
}
```
### image\_generation [#image_generation]
| Parameter | Type | Description |
| -------------- | ------ | ------------------------------------------------------ |
| `aspect_ratio` | string | Image aspect ratio (e.g., `"16:9"`, `"1:1"`, `"9:16"`) |
| `image_size` | string | Image resolution (e.g., `"2K"`, `"1K"`) |
The `image_generation` field is optional. If omitted, the model uses default settings. However, if you specify `image_generation`, both `aspect_ratio` and `image_size` are required.
***
## Handling Responses [#handling-responses]
### Chat Completions [#chat-completions]
When streaming, images arrive in chunks via the `images` delta field:
```json
// Image chunks arrive in delta
{
"choices": [{
"delta": {
"images": [{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgo..."
}
}]
}
}]
}
```
Non-streaming responses include images in the message:
```json
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"model": "gemini-3-pro-image-preview",
"choices": [{
"index": 0,
"message": {
"role": "assistant",
"content": "Here's the image you requested:",
"images": [{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
}
}]
},
"finish_reason": "stop"
}],
"usage": {
"prompt_tokens": 12,
"completion_tokens": 1024,
"total_tokens": 1036
}
}
```
### Responses API [#responses-api]
Streaming events follow the Responses API format:
```json
// Content part added for image
{
"type": "response.content_part.added",
"item_id": "msg_abc123",
"output_index": 0,
"content_index": 0,
"part": {
"type": "output_image",
"image_url": ""
}
}
// Content part done with full image
{
"type": "response.content_part.done",
"item_id": "msg_abc123",
"output_index": 0,
"content_index": 0,
"part": {
"type": "output_image",
"image_url": "data:image/png;base64,iVBORw0KGgo..."
}
}
```
```json
{
"id": "resp_abc123",
"object": "response",
"status": "completed",
"model": "gemini-3-pro-image-preview",
"output": [
{
"id": "msg_abc123",
"type": "message",
"status": "completed",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "Here's the image you requested:"
},
{
"type": "output_image",
"image_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
}
]
}
],
"usage": {
"input_tokens": 12,
"output_tokens": 1024
}
}
```
***
## Supported Models [#supported-models]
| Model | Provider Route | Description |
| ----------------------------------- | ---------------- | ------------------------------------------------------------------------ |
| `google/gemini-3-pro-image-preview` | Google AI Studio | Nano Banana Pro - Google's multimodal model with native image generation |
***
## Related [#related]
* [Reasoning](/features/routing/provider-features/reasoning) - Enable reasoning for complex tasks
* [Responses API](/features/routing/provider-features/responses-api) - Alternative API format with image support
# Prompt caching (https://docs.corevalue.dev/features/routing/provider-features/prompt-caching)
Prompt caching allows you to cache frequently-used context (system prompts, examples, documents) and reuse it across multiple requests at significantly reduced costs.
## Why Prompt Caching [#why-prompt-caching]
Cached prompts are processed at significantly reduced rates by providers (up to 90% savings)
Providers skip re-processing cached prompt segments for faster response times
Works out-of-the-box with OpenAI compatible AI Gateway across all providers
***
## OpenAI and Compatible Providers [#openai-and-compatible-providers]
**Automatic caching** for prompts over 1024 tokens. Use the `prompt_cache_key` parameter for better cache hit control.
**Compatible providers:** OpenAI, Grok, Groq, Deepseek, Moonshot AI
### Quick Start [#quick-start]
```typescript
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://gateway.corevalue.dev/v1",
apiKey: process.env.COVA_API_KEY,
});
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [
{
role: "system",
content: "Very long system prompt that will be automatically cached..." // 1024+ tokens
},
{
role: "user",
content: "What is machine learning?"
}
],
prompt_cache_key: `doc-analysis-${documentId}` // Optional: control caching keys
});
```
### Pricing [#pricing]
OpenAI charges standard rates for cache writes and offers significant discounts for cache reads. Exact pricing varies by model.
View supported models and their caching capabilities
Official OpenAI prompt caching guide
***
## Anthropic (Claude) [#anthropic-claude]
Anthropic provides advanced caching with **cache control breakpoints** (up to 4 per request) and TTL control.
### Using OpenAI SDK with CoreValue Types [#using-openai-sdk-with-corevalue-types]
The `@cova/gateway` SDK extends OpenAI types to support Anthropic's cache control through the OpenAI-compatible interface:
```bash
npm install @cova/gateway
```
```typescript
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://gateway.corevalue.dev/v1",
apiKey: process.env.COVA_API_KEY,
});
const response = await client.chat.completions.create({
model: "anthropic/claude-3-5-haiku-20241022",
messages: [
{
role: "system",
content: "You are a helpful assistant...",
cache_control: {
type: "ephemeral",
ttl: "1h"
}
},
{
role: "assistant",
content: "Example assistant message.",
cache_control: { type: "ephemeral" }
},
{
role: "user",
content: [
{
type: "text",
text: "This content will be cached.",
cache_control: {
type: "ephemeral",
ttl: "5m"
}
},
{
type: "image_url",
image_url: {
url: "https://example.com/image.jpg",
detail: "low"
},
cache_control: { type: "ephemeral" }
}
]
}
],
temperature: 0.7
});
```
### Cache Key Mapping [#cache-key-mapping]
Anthropic uses `user_id` as a cache key on their servers. When using the OpenAI-compatible AI Gateway, these parameters automatically map to Anthropic's `user_id`:
* `prompt_cache_key`
* `safety_identifier`
* `user`
```typescript
const response = await client.chat.completions.create({
model: "anthropic/claude-3-5-haiku-20241022",
messages: [/* your messages */],
prompt_cache_key: "doc-analysis-v1", // Maps to Anthropic's user_id for cache keying
cache_control: {
type: "ephemeral",
ttl: "1h"
}
});
```
**Current Limitation**: Anthropic cache control is currently enabled for caching messages only. Support for caching tools is coming soon.
### Pricing Structure [#pricing-structure]
Anthropic uses a simple multiplier-based pricing model for prompt caching.
| Operation | Multiplier | Example (Claude Sonnet @ $3/MTok) |
| -------------------- | ---------- | --------------------------------- |
| Cache Read | 0.1× | $0.30/MTok |
| Cache Write (5 min) | 1.25× | $3.75/MTok |
| Cache Write (1 hour) | 2.0× | $6.00/MTok |
### Key Points [#key-points]
* **TTL Options**: 5 minutes or 1 hour
* **Providers**: Available on Anthropic API, Vertex AI, and AWS Bedrock
* **Limitation**: Vertex AI and Bedrock only support 5-minute caching
* **Minimum**: 1024 tokens for most models
### Calculation Example [#calculation-example]
```
Base input price: $3/MTok
5-min cache write: $3 × 1.25 = $3.75/MTok
1-hour cache write: $3 × 2.0 = $6.00/MTok
Cache read: $3 × 0.1 = $0.30/MTok
```
Anthropic Prompt Caching Documentation
***
## Google Gemini [#google-gemini]
Google uses a multiplier plus storage cost model for context caching.
### Pricing Structure [#pricing-structure-1]
| Operation | Multiplier | Storage Cost |
| ----------- | ---------- | ------------- |
| Cache Read | 0.25× | N/A |
| Cache Write | 1.0× | + Storage fee |
**Storage Rates:**
* Gemini 2.5 Pro: $4.50/MTok/hour
* Gemini 2.5 Flash: $1.00/MTok/hour
* Gemini 2.5 Flash-Lite: $1.00/MTok/hour
### Key Points [#key-points-1]
* **TTL**: 5 minutes only
* **Cache Types**: Implicit (automatic) and Explicit (manual)
* **Minimum**: 1024 tokens (Flash), 2048 tokens (Pro)
* **Discount**: 75% off input costs for cache reads
### Calculation Example [#calculation-example-1]
For Gemini 2.5 Pro (≤200K tokens):
```
Base input price: $1.25/MTok
Storage rate: $4.50/MTok/hour
Cache write (5 min):
- Input cost: $1.25 × 1.0 = $1.25
- Storage cost: $4.50 × (5/60) = $0.375
- Total: $1.625/MTok
Cache read: $1.25 × 0.25 = $0.31/MTok
```
### Tiered Pricing [#tiered-pricing]
Gemini 2.5 Pro has different rates for larger contexts:
| Context Size | Input Price | Cache Read | Cache Write (5 min) |
| ------------ | ----------- | ----------- | ------------------- |
| ≤200K tokens | $1.25/MTok | $0.31/MTok | $1.625/MTok |
| >200K tokens | $2.50/MTok | $0.625/MTok | $2.875/MTok |
# Reasoning models (https://docs.corevalue.dev/features/routing/provider-features/reasoning)
CoreValue's AI Gateway provides a unified interface for reasoning across providers. Use the same parameters regardless of provider - the Gateway handles the translation automatically.
***
## Quick Start [#quick-start]
```typescript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/v1",
});
const response = await client.chat.completions.create({
model: "anthropic/claude-sonnet-4-20250514",
messages: [
{ role: "user", content: "What is the sum of the first 100 prime numbers?" }
],
reasoning_effort: "medium",
max_completion_tokens: 16000
});
```
```typescript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/v1",
});
const response = await client.responses.create({
model: "anthropic/claude-sonnet-4-20250514",
input: "What is the sum of the first 100 prime numbers?",
reasoning: {
effort: "medium"
}
});
```
***
## Configuration [#configuration]
```typescript
{
reasoning_effort: "low" | "medium" | "high",
reasoning_options: {
budget_tokens: 8000 // Optional token budget
}
}
```
```typescript
{
reasoning: {
effort: "low" | "medium" | "high"
},
reasoning_options: {
budget_tokens: 8000 // Optional token budget
}
}
```
### reasoning\_effort [#reasoning_effort]
| Level | Description |
| -------- | ----------------------------------- |
| `low` | Light reasoning for simple tasks |
| `medium` | Balanced reasoning |
| `high` | Deep reasoning for complex problems |
For Anthropic models, the default is 4096 max completion tokens with 2048 budget reasoning tokens.
### reasoning\_options.budget\_tokens [#reasoning_optionsbudget_tokens]
The `budget_tokens` parameter sets the maximum number of tokens the model can use for reasoning.
**For Google (Gemini) models:** `reasoning_effort` is **required** to enable thinking. Passing `budget_tokens` alone will **not** enable reasoning - you must also specify `reasoning_effort`.
```typescript
// ✅ Correct: reasoning_effort enables thinking, budget_tokens limits it
{
reasoning_effort: "high",
reasoning_options: { budget_tokens: 4096 }
}
// ❌ Incorrect for Gemini: budget_tokens alone does nothing
{
reasoning_options: { budget_tokens: 4096 } // Reasoning will be disabled
}
```
***
## Handling Responses [#handling-responses]
### Chat Completions [#chat-completions]
When streaming, reasoning content arrives in chunks via the `reasoning` delta field, followed by content, and finally `reasoning_details` with the finish reason:
```json
// Reasoning chunks arrive first
{
"choices": [{
"delta": { "reasoning": "Let me think about this..." }
}]
}
// Then content chunks
{
"choices": [{
"delta": { "content": "The answer is 42." }
}]
}
// Final chunk includes reasoning_details with signature
{
"choices": [{
"delta": {
"reasoning_details": [{
"thinking": "The user is asking for...",
"signature": "EpICCkYIChgCKkCfWt1pnGxEcz48yQJvie3ppkXZ8ryd..."
}]
},
"finish_reason": "stop"
}]
}
```
Non-streaming responses include the full reasoning in the message:
```json
{
"id": "msg_01S1QpjYur8kLeEVKVoKxdTP",
"object": "chat.completion",
"model": "claude-haiku-4-5-20251001",
"choices": [{
"index": 0,
"message": {
"role": "assistant",
"content": "Why don't scientists trust atoms?\n\nBecause they make up everything!",
"reasoning": "The user is asking for a very short joke. I should provide something quick, light, and funny...",
"reasoning_details": [{
"thinking": "The user is asking for a very short joke...",
"signature": "Ev8DCkYIChgCKkBeHyembBdwl8C/a/8luinDP0w5/oQP..."
}]
},
"finish_reason": "stop"
}],
"usage": {
"prompt_tokens": 58,
"completion_tokens": 108,
"total_tokens": 166
}
}
```
### Responses API [#responses-api]
Streaming events follow the Responses API format:
```json
// Reasoning summary text delta
{
"type": "response.reasoning_summary_text.delta",
"item_id": "rs_0ab50bce3156357b...",
"output_index": 0,
"summary_index": 0,
"delta": "Let me think about this..."
}
// Reasoning item complete
{
"type": "response.output_item.done",
"output_index": 0,
"item": {
"id": "rs_0ab50bce3156357b...",
"type": "reasoning",
"summary": [{
"type": "summary_text",
"text": "**Crafting the response**\n\nThe user wants..."
}]
}
}
```
```json
{
"id": "resp_038bfaf6e50f1c45...",
"object": "response",
"status": "completed",
"model": "gpt-5-mini-2025-08-07",
"output": [
{
"id": "rs_038bfaf6e50f1c45...",
"type": "reasoning",
"summary": [{
"type": "summary_text",
"text": "**Generating programming jokes**\n\nThe user wants a short joke..."
}]
},
{
"id": "msg_038bfaf6e50f1c45...",
"type": "message",
"status": "completed",
"role": "assistant",
"content": [{
"type": "output_text",
"text": "To understand recursion, you must first understand recursion."
}]
}
],
"usage": {
"input_tokens": 17,
"output_tokens": 336,
"output_tokens_details": {
"reasoning_tokens": 320
}
}
}
```
Anthropic responses include `encrypted_content` for reasoning validation:
```json
{
"id": "msg_017G4K2w5s6zEn3KZ6jp455j",
"object": "response",
"status": "completed",
"model": "claude-haiku-4-5-20251001",
"output": [
{
"id": "rs_msg_017G4K2w5s6zEn3KZ6jp455j_0",
"type": "reasoning",
"summary": [{
"type": "summary_text",
"text": "The user wants me to tell a short joke about programming..."
}],
"encrypted_content": "EuYGCkYIChgCKkBxEozbYO/Z5AL2tlDHwBHcBEOG..."
},
{
"id": "msg_msg_017G4K2w5s6zEn3KZ6jp455j",
"type": "message",
"status": "completed",
"role": "assistant",
"content": [{
"type": "output_text",
"text": "Why do programmers prefer dark mode?\n\nBecause light attracts bugs!"
}]
}
],
"usage": {
"input_tokens": 47,
"output_tokens": 294
}
}
```
Anthropic models always return `encrypted_content` (signatures) in reasoning items. These signatures validate the reasoning chain and are required for multi-turn conversations. Other providers like OpenAI can optionally return signatures when configured.
***
## Related [#related]
* [Responses API](/features/routing/provider-features/responses-api) - Alternative API format with reasoning support
* [Context Editing](/features/routing/provider-features/context-editing) - Manage context in long reasoning sessions
# Responses API (https://docs.corevalue.dev/features/routing/provider-features/responses-api)
The Responses API is OpenAI's newer interface for conversational AI that supports advanced features like reasoning, tool use, and streaming. CoreValue's AI Gateway supports the Responses API format for both OpenAI and Anthropic models.
## Quick Start [#quick-start]
Use your CoreValue API key and the AI Gateway base URL. Then call the OpenAI SDK's `responses.create` method as usual.
```typescript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.COVA_API_KEY,
baseURL: "https://gateway.corevalue.dev/v1",
});
const response = await client.responses.create({
model: "gpt-5",
input: "Write a one-sentence bedtime story about a unicorn.",
});
console.log(response.output_text);
```
```python
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ.get("COVA_API_KEY"),
base_url="https://gateway.corevalue.dev/v1",
)
response = client.responses.create(
model="gpt-5",
input="Write a one-sentence bedtime story about a unicorn.",
)
print(response.output_text)
```
```bash
curl https://gateway.corevalue.dev/v1/responses \
-H "Authorization: Bearer $COVA_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-5",
"input": "Write a one-sentence bedtime story about a unicorn."
}'
```
For Chat Completions usage and more background on the AI Gateway, see the
[AI Gateway Overview](/).
## Parameter Stripping (M2 0.5) [#parameter-stripping-m2-05]
The gateway runs a `paramStripper` on Responses API requests to remove parameters that are not supported by the target provider before forwarding. A `NEVER_STRIP` denylist protects critical parameters (such as `model`, `input`, `stream`, and `reasoning`) so they are always preserved regardless of provider.
The `NEVER_STRIP` denylist ensures that essential Responses API parameters are never stripped during provider translation. Provider-specific unsupported fields are dropped silently; check the `Cova-Translation-Warning` response header (in translated mode) to see which fields were dropped.
## References [#references]
* OpenAI Responses guide: [https://platform.openai.com/docs/guides/text](https://platform.openai.com/docs/guides/text)
* CoreValue AI Gateway overview: [https://docs.corevalue.dev/gateway/overview](https://docs.corevalue.dev/gateway/overview)
The first time you sign in you are asked to name your organisation. Everything you set up — keys, budgets, requests — belongs to that organisation, and you can invite teammates to it later.