streamChat — Streaming with FinOps Metadata
Stream chat completions through the gateway with per-call FinOps metadata using streamChat.
streamChat — Streaming with FinOps Metadata
Wrap openai.chat.completions.create({stream:true}) with streamChat() to get per-call cost and credits after the stream completes.
Tier: Available on all tiers.
How It Works
streamChat() passes provider chunks through unchanged. After the gateway sends data: [DONE], it appends a final SSE event with CovaGatewayMeta:
data: {"cova":{"requestId":"...","provider":"openai","model":"gpt-4o","status":"success","requestCost":0.0015,"creditsRemaining":48.77}}The SDK yields this as a final { cova: CovaGatewayMeta } object.
TypeScript
import OpenAI from "openai";
import { streamChat } from "@cova/gateway";
const client = new OpenAI({
apiKey: "sk-cova-...",
baseURL: "https://gateway.corevalue.dev/v1",
});
for await (const chunk of streamChat(
client,
{
model: "gpt-4o",
messages: [{ role: "user", content: "Hello" }],
stream: true,
},
{ userId: "user_123" }
)) {
if ("cova" in chunk) {
console.log(chunk.cova.requestCost);
console.log(chunk.cova.creditsRemaining);
} else {
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
}Python
from openai import AsyncOpenAI
from cova_gateway import stream_chat, CovaHeaderOptions
client = AsyncOpenAI(
api_key="sk-cova-...",
base_url="https://gateway.corevalue.dev/v1",
)
async for chunk in stream_chat(
client,
{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}], "stream": True},
CovaHeaderOptions(user_id="user_123"),
):
if "cova" in chunk:
print(chunk["cova"].request_cost)
print(chunk["cova"].credits_remaining)
else:
print(chunk.choices[0].delta.content or "", end="")Final event only: The { cova: CovaGatewayMeta } object is yielded once, as the
final item in the async generator. All prior chunks are standard OpenAI streaming chunks.
Fallback: /meta Endpoint
If no metadata event is available (e.g., upstream interrupted), use the /meta endpoint:
curl https://gateway.corevalue.dev/gateway/v1/meta/<requestId> \
-H "Authorization: Bearer sk-cova-..."200— metadata available202— request still in flight or expired (5-minute TTL)404— requestId belongs to a different org
See Per-Call FinOps Metadata for details.