How cost is calculated

Real Cost — Cova-Request-Cost (M0)

As of M0 (0.0a), the gateway computes real cost for every request — not an estimate. Where it is reported depends on the response: the Cova-Request-Cost header for non-streaming requests, and the final SSE cova frame or the /meta endpoint for streaming ones (see below). The value is derived from extracted upstream usage and the model's pricing table in packages/cost/models/.

Non-Streaming Requests

For non-streaming requests, the upstream provider returns a usage object (prompt tokens, completion tokens, cache tokens, etc.). The gateway extracts this usage, looks up the model's per-token pricing, and sets Cova-Request-Cost on the response headers.

"usage": {
  "prompt_tokens": 11,
  "completion_tokens": 9,
  "total_tokens": 20
}

The corresponding header:

Cova-Request-Cost: 0.0015000000

Streaming Requests

A streamed response never carries Cova-Request-Cost. Response headers are sent before the first chunk, and the cost is only known once the stream has ended and usage has been read from it.

Where to read the cost instead depends on the stream's wire format:

The frame is only appended to SSE streams. Appending text to a binary event stream or a JSON document would corrupt it, so those responses pass through byte for byte.

For SSE streams, the gateway appends a final data: frame after the upstream stream closes (including after data: [DONE] for OpenAI-format providers). The cost is included in the cova object:

data: {"cova":{"requestId":"...","provider":"groq","model":"groq/llama-3.3-70b-versatile","status":"success","requestCost":0.0015}}

For any stream, read Cova-Id from the response headers and call GET /gateway/v1/meta/:requestId after the stream ends. It returns { "status": "pending" } with 202 until settlement has finished, and keeps the metadata for 5 minutes.

For the full streaming FinOps flow — including the @cova/gateway SDK's streamChat() helper and the /meta fallback — see Streaming FinOps.

Upfront Estimate — Cova-Request-Cost-Estimate (M4)

When escrow (pass-through billing) is enabled and a PTB key may serve the request, the gateway sets Cova-Request-Cost-Estimate on the response. This is an upfront worst-case cost estimate — the escrow reservation amount — provided as a UX hint only.

Cova-Request-Cost-Estimate: 0.0230000000

Anthropic Requests

For Anthropic requests, the gateway extracts usage from the Anthropic response format (which differs from OpenAI's). Token counting and cost calculation use the same pricing-table lookup as other providers. Cross-provider translation (M4) maps the response back to OpenAI format before returning it to the client.

Developer Reference

For the full cost calculation implementation, see the packages/cost package in the open-source repository.