LangChain Integration

Integrate CoreValue AI Gateway with LangChain to access LLM providers with unified observability.

Introduction

LangChain is a popular open-source framework for building applications with large language models across Python, TypeScript, and other languages. By integrating CoreValue AI Gateway with LangChain, you can:

Route to different models & providers with automatic failover through a single endpoint
Unified billing with pass-through billing or bring your own keys
Monitor all requests with automatic cost tracking in one dashboard
Stream responses with full observability for real-time applications

This integration requires only two changes to your existing LangChain code - updating the base URL and API key.

Integration Steps

You'll also need to configure your provider API keys (OpenAI, Anthropic, etc.) at CoreValue Providers for BYOK (Bring Your Own Keys).

# Your CoreValue API key
export COVA_API_KEY=<your-corevalue-api-key>

Create a .env file in your project:

COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX

npm install @langchain/openai @langchain/core dotenv
# or
yarn add @langchain/openai @langchain/core dotenv

pip install langchain-openai langchain-core python-dotenv

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage, SystemMessage } from "@langchain/core/messages";
import dotenv from 'dotenv';

dotenv.config();

// Initialize ChatOpenAI with CoreValue AI Gateway
const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',  // many models supported
    apiKey: process.env.COVA_API_KEY,
    configuration: {
        baseURL: "https://gateway.corevalue.dev/v1",
        defaultHeaders: {
            // Optional: Add custom tracking headers
            // Session tracking via custom properties:
            "Cova-Property-Session-Id": "my-session",
            "Cova-User-Id": "user-123",
            "Cova-Property-Environment": "production",
        },
    },
});

import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from dotenv import load_dotenv

load_dotenv()

# Initialize ChatOpenAI with CoreValue AI Gateway
chat = ChatOpenAI(
    model='gpt-4.1-mini',  # many models supported
    api_key=os.getenv('COVA_API_KEY'),
    base_url="https://gateway.corevalue.dev/v1",
    default_headers={
        # Optional: Add custom tracking headers
        'Cova-Property-Session-Id': 'my-session',
        'Cova-User-Id': 'user-123',
        'Cova-Property-Environment': 'production',
    },
)

The only changes from a standard LangChain setup are the apiKey, baseURL (or base_url in Python), and optional tracking headers. Everything else stays the same!

You can find all supported models at corevalue.dev/models.

Your existing LangChain code continues to work without any changes:

// Simple completion
const response = await chat.invoke([
    new SystemMessage("You are a helpful assistant."),
    new HumanMessage("What is the capital of France?"),
]);

console.log(response.content);

# Simple completion
messages = [
    SystemMessage(content="You are a helpful assistant."),
    HumanMessage(content="What is the capital of France?"),
]

response = chat.invoke(messages)
print(response.content)

All your LangChain requests are now visible in your CoreValue dashboard.

Request/response bodies
Latency metrics
Token usage and costs
Model performance analytics
Error tracking
Session tracking

While you're here, why not give us a star on GitHub? It helps us a lot!

Migration Example

Here's what migrating an existing LangChain application looks like:

Before (Direct OpenAI)

import { ChatOpenAI } from "@langchain/openai";

const chat = new ChatOpenAI({
    model: 'gpt-4o-mini',
    apiKey: process.env.OPENAI_API_KEY,
});

from langchain_openai import ChatOpenAI

chat = ChatOpenAI(
    model='gpt-4o-mini',
    api_key=os.getenv('OPENAI_API_KEY'),
)

After (CoreValue AI Gateway)

import { ChatOpenAI } from "@langchain/openai";

const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',                      // many models supported
    apiKey: process.env.COVA_API_KEY,      // Your CoreValue API key
    configuration: {
        baseURL: "https://gateway.corevalue.dev/v1"  // Add this!
    },
});

from langchain_openai import ChatOpenAI

chat = ChatOpenAI(
    model='gpt-4.1-mini',                      # many models supported
    api_key=os.getenv('COVA_API_KEY'),    # Your CoreValue API key
    base_url="https://gateway.corevalue.dev/v1"  # Add this!
)

That's it! Just two changes and you're routing through CoreValue's AI Gateway.

Complete Working Examples

Basic Example

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage, SystemMessage } from "@langchain/core/messages";
import dotenv from 'dotenv';

dotenv.config();

const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',  // many models supported
    apiKey: process.env.COVA_API_KEY,
    configuration: {
        baseURL: "https://gateway.corevalue.dev/v1",
        defaultHeaders: {
            // Session tracking via custom properties:
                "Cova-Property-Session-Id": "langchain-example",
            "Cova-User-Id": "demo-user",
        },
    },
});

async function main() {
    console.log('🦜 Starting LangChain + CoreValue AI Gateway example...\n');

    const response = await chat.invoke([
        new SystemMessage("You are a helpful assistant."),
        new HumanMessage("Tell me a joke about programming."),
    ]);

    console.log('🤖 Assistant response:');
    console.log(response.content);
    console.log('\n✅ Completed successfully!');
}

main().catch(console.error);

import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from dotenv import load_dotenv

load_dotenv()

chat = ChatOpenAI(
    model='gpt-4.1-mini',  # many models supported
    api_key=os.getenv('COVA_API_KEY'),
    base_url="https://gateway.corevalue.dev/v1",
    default_headers={
        'Cova-Property-Session-Id': 'langchain-example',
        'Cova-User-Id': 'demo-user',
    },
)

def main():
    print('🐍 Starting LangChain + CoreValue AI Gateway example...\n')

    messages = [
        SystemMessage(content="You are a helpful assistant."),
        HumanMessage(content="Tell me a joke about Python programming."),
    ]

    response = chat.invoke(messages)

    print('🤖 Assistant response:')
    print(response.content)
    print('\n✅ Completed successfully!')

if __name__ == "__main__":
    main()

Streaming Example

async function streamingExample() {
    console.log('\n🌊 Streaming example...\n');

    const stream = await chat.stream([
        new SystemMessage("You are a helpful assistant."),
        new HumanMessage("Write a short story about a robot learning to code."),
    ]);

    console.log('🤖 Assistant (streaming):');
    for await (const chunk of stream) {
        process.stdout.write(chunk.content as string);
    }
    console.log('\n\n✅ Streaming completed!');
}

streamingExample().catch(console.error);

def streaming_example():
    print('\n🌊 Streaming example...\n')

    messages = [
        SystemMessage(content="You are a helpful assistant."),
        HumanMessage(content="Write a short story about a robot learning to code."),
    ]

    print('🤖 Assistant (streaming):')
    for chunk in chat.stream(messages):
        print(chunk.content, end='', flush=True)

    print('\n\n✅ Streaming completed!')

streaming_example()

Multiple Models Example

async function testMultipleModels() {
    console.log('🚀 Testing multiple models through CoreValue AI Gateway\n');

    const models = [
        { id: 'gpt-4.1-mini', name: 'OpenAI GPT-4.1 Mini' },
        { id: 'anthropic/claude-opus-4-1', name: 'Anthropic Claude Opus 4.1' },
        { id: 'google/gemini-2.5-flash-lite', name: 'Google Gemini 2.5 Flash Lite' },
    ];

    for (const model of models) {
        try {
            const chat = new ChatOpenAI({
                model: model.id,
                apiKey: process.env.COVA_API_KEY,
                configuration: {
                    baseURL: "https://gateway.corevalue.dev/v1",
                },
            });

            console.log(`🤖 Testing ${model.name}... `);
            const response = await chat.invoke([
                new HumanMessage("Say hello in one sentence."),
            ]);
            console.log(`   Response: ${response.content}\n`);
        } catch (error) {
            console.error(`   Error: ${error}\n`);
        }
    }

    console.log('✅ All models tested!');
    console.log('🔍 Check your dashboard: https://us.corevalue.dev/dashboard');
}

testMultipleModels().catch(console.error);

def test_multiple_models():
    print('🚀 Testing multiple models through CoreValue AI Gateway\n')

    models = [
        {'id': 'gpt-4.1-mini', 'name': 'OpenAI GPT-4.1 Mini'},
        {'id': 'anthropic/claude-opus-4-1', 'name': 'Anthropic Claude Opus 4.1'},
        {'id': 'google/gemini-2.5-flash-lite', 'name': 'Google Gemini 2.5 Flash Lite'},
    ]

    for model in models:
        try:
            chat = ChatOpenAI(
                model=model['id'],
                api_key=os.getenv('COVA_API_KEY'),
                base_url="https://gateway.corevalue.dev/v1",
            )

            print(f"🤖 Testing {model['name']}... ")
            response = chat.invoke([
                HumanMessage(content="Say hello in one sentence."),
            ])
            print(f"   Response: {response.content}\n")
        except Exception as error:
            print(f"   Error: {error}\n")

    print('✅ All models tested!')
    print('🔍 Check your dashboard: https://us.corevalue.dev/dashboard')

test_multiple_models()

Batch Processing Example (Python)

def batch_example():
    print('\n📦 Batch processing example...\n')

    message_batches = [
        [HumanMessage(content="What is Python?")],
        [HumanMessage(content="What is JavaScript?")],
        [HumanMessage(content="What is TypeScript?")],
    ]

    responses = chat.batch(message_batches)

    print('🤖 Batch responses:')
    for i, response in enumerate(responses, 1):
        print(f'\nResponse {i}: {response.content}')

    print('\n✅ Batch processing completed!')

batch_example()

CoreValue Prompts Integration

You can use CoreValue Prompts for centralized prompt management and versioning by passing parameters through modelKwargs:

const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',
    apiKey: process.env.COVA_API_KEY,
    modelKwargs: {
        prompt_id: 'customer-support-prompt',
        version_id: 'version-uuid',
        environment: 'production',
        inputs: { customer_name: 'John', issue_type: 'billing' },
    },
    configuration: {
        baseURL: "https://gateway.corevalue.dev/v1",
    },
});

chat = ChatOpenAI(
    model='gpt-4.1-mini',
    api_key=os.getenv('COVA_API_KEY'),
    base_url="https://gateway.corevalue.dev/v1",
    model_kwargs={
        'prompt_id': 'customer-support-prompt',
        'version_id': 'version-uuid',
        'environment': 'production',
        'inputs': {'customer_name': 'John', 'issue_type': 'billing'},
    },
)

All prompt parameters (prompt_id, version_id, environment, inputs) are optional. Learn more about Prompts with AI Gateway.

Custom Headers and Properties

You can add custom properties to track and filter your requests:

const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',
    apiKey: process.env.COVA_API_KEY,
    configuration: {
        baseURL: "https://gateway.corevalue.dev/v1",
        defaultHeaders: {
            // Session tracking
            // Session tracking via custom properties:
                "Cova-Property-Session-Id": "session-abc-123",
            "Cova-Property-Session-Name": "Customer Support Chat",
            "Cova-Property-Session-Path": "/support/chat/456",

            // User tracking
            "Cova-User-Id": "user-789",

            // Custom properties for filtering
            "Cova-Property-Environment": "production",
            "Cova-Property-App-Version": "2.1.0",
            "Cova-Property-Feature": "customer-support",

            // Rate limiting (optional)
            "Cova-RateLimit-Policy": "basic-100",
        },
    },
});

chat = ChatOpenAI(
    model='gpt-4.1-mini',
    api_key=os.getenv('COVA_API_KEY'),
    base_url="https://gateway.corevalue.dev/v1",
    default_headers={
        # Session tracking
        'Cova-Property-Session-Id': 'session-abc-123',
        'Cova-Property-Session-Name': 'Customer Support Chat',
        'Cova-Property-Session-Path': '/support/chat/456',

        # User tracking
        'Cova-User-Id': 'user-789',

        # Custom properties for filtering
        'Cova-Property-Environment': 'production',
        'Cova-Property-App-Version': '2.1.0',
        'Cova-Property-Feature': 'customer-support',

        # Rate limiting (optional)
        'Cova-RateLimit-Policy': 'basic-100',
    },
)

Request a CoreValue Integration

Looking for a framework or tool not listed here? Request it here!

LangChain Integration

Introduction

Integration Steps

Migration Example

Before (Direct OpenAI)

After (CoreValue AI Gateway)

Complete Working Examples

Basic Example

Streaming Example

Multiple Models Example

Batch Processing Example (Python)

CoreValue Prompts Integration

Custom Headers and Properties

AI Gateway Overview

Provider Routing

Model Registry

Prompt Management

Custom Properties

Sessions

Rate Limiting

On this page