Cache responses to reduce cost and latency for repeated requests via Cova-Cache-* headers.

Response Caching

Cache responses to reduce cost and latency for repeated requests. Cache is opt-in per request via Cova-Cache-* headers.

Tier: Available on all tiers.

Semantic cache deferred: The exact-match cache is shipped. Semantic cache (vector similarity) is planned behind SEMANTIC_CACHE_ENABLED — the flag exists in env.ts but is not yet wired into the cache middleware.

How It Works

When cache is enabled, the gateway:

Computes a cache key from the request body (excluding ignored keys)
Checks the cache bucket for a matching response
On hit: returns the cached response with Cova-Cache: HIT header
On miss: forwards to upstream, saves the response to the cache bucket

Cache Request Headers

Header	Purpose	Default
`Cova-Cache-Enabled`	Enable read + write	—
`Cova-Cache-Save`	Enable write only	—
`Cova-Cache-Read`	Enable read only	—
`Cova-Cache-Bucket-Max-Size`	Max cached responses per key	1 (max 20)
`Cova-Cache-Seed`	Cache isolation seed	—
`Cova-Cache-Ignore-Keys`	Body keys to ignore in cache key	—

Usage

Read + Write (Default)

curl https://gateway.corevalue.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-cova-..." \
  -H "Content-Type: application/json" \
  -H "Cova-Cache-Enabled: true" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"What is 2+2?"}]}'

Write Only (Seed Cache)

curl https://gateway.corevalue.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-cova-..." \
  -H "Cova-Cache-Save: true" \
  -d '{"model":"gpt-4o","messages":[...]}'

Read Only (No Saving)

curl https://gateway.corevalue.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-cova-..." \
  -H "Cova-Cache-Read: true" \
  -d '{"model":"gpt-4o","messages":[...]}'

Cache Response Headers

On a cache hit, the gateway sets:

Header	Purpose
`Cova-Cache`	"HIT"
`Cova-Cache-Bucket-Idx`	Cache bucket index
`Cova-Cache-Latency`	Cached response latency (ms)

Escrow Interaction

When ESCROW_ENABLED is true, the gateway still consults the request cache after reserving escrow. On a cache hit, the escrow hold is cancelled (a cached response incurs no PTB cost), so the user is not charged for a cached answer. The request cache itself is never bypassed by escrow.

Related: Escrow for reserve-before-execute behavior. Cova-* Headers Guide for all headers.

Response Caching

On this page