Rate Limiting

CoreValue Gateway applies two layers of rate limiting: a global sliding window and a configurable token bucket.

Layer 1: Global Sliding Window

Always applied. Prevents DoS at the org level.

• Algorithm: Redis sorted sets (sliding window)
• Scope: Per-org, per-segment (global, user, model)
• Default: 1000 requests per 60 seconds
• Response: 429 with Retry-After header (seconds)

Layer 2: Token Bucket

Configurable per-org. Supports count-based or cost-based limits.

• Algorithm: Token bucket with lazy refill (Lua script for atomicity)
• Units: "request" (count-based) or "cents" (cost-based)
• Scope: Per-org, per-segment-type (global, user, property, model)
• Configuration: Org-level config or header-based via Cova-RateLimit-Policy

Segments

Header-Based Override

Set Cova-RateLimit-Policy to override the org-level config for a single request. The header uses a semicolon-delimited format: quota;w=windowSeconds;u=unit;s=segment (e.g. 100;w=3600;u=cents for 100 cents per hour, or 50;w=3600;u=request;s=user for 50 requests per hour per user). When w is omitted the window defaults to 3600 seconds (1 hour).

curl https://gateway.corevalue.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-cova-..." \
  -H "Content-Type: application/json" \
  -H "Cova-RateLimit-Policy: 100;w=60;u=request" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hello"}]}'

Response Format

429 Rate Limited

HTTP/1.1 429 Too Many Requests
Retry-After: 30

Configuration

Rate limit policies are configured via the CoreValue dashboard or API. The gateway fetches policies from Core and caches them.