DSPy

Integrate CoreValue AI Gateway with DSPy to access LLM providers with unified observability and optimization.

Introduction

DSPy is a declarative framework for building modular AI software with structured code instead of brittle prompts, offering algorithms that compile AI programs into effective prompts and weights for language models across classifiers, RAG pipelines, and agent loops.

Integration Steps

Log into CoreValue or create an account. Once you have an account, you can generate an API key here.

Create a .env file in your project.

COVA_API_KEY=sk-cova-XXXXXXXXXXXXXXXX

Install required dependencies

pip install dspy

View requests in the Cova dashboard

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure DSPy to use CoreValue AI Gateway
lm = dspy.LM(
  'gpt-4o-mini',  # or any other model from the CoreValue model registry
  api_key=os.getenv('COVA_API_KEY'),
  api_base='https://gateway.corevalue.dev/v1'
)

dspy.configure(lm=lm)

print(lm("Hello, world!"))

You can find all supported models at corevalue.dev/models.

With the above setup, any calls to DSPy will automatically be logged and monitored by CoreValue. Review them in your CoreValue dashboard.

While you're here, why not give us a star on GitHub? It helps us a lot!

Complete Working Examples

Basic Chain of Thought

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure CoreValue AI Gateway
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('COVA_API_KEY'),
    api_base='https://gateway.corevalue.dev/v1'
)
dspy.configure(lm=lm)

# Define a module
qa = dspy.ChainOfThought('question -> answer')

# Run inference
response = qa(question="How many floors are in the castle David Gregory inherited?")

print('Answer:', response.answer)
print('Reasoning:', response.reasoning)

Custom Generation Configuration

Configure temperature, max_tokens, and other parameters:

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure with custom generation parameters
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('COVA_API_KEY'),
    api_base='https://gateway.corevalue.dev/v1',
    temperature=0.9,
    max_tokens=2000
)
dspy.configure(lm=lm)

# Use with any DSPy module
predict = dspy.Predict("question -> creative_answer")
response = predict(question="Write a creative story about AI")
print(response.creative_answer)

Tracking with Custom Properties

Add custom properties to track and filter your requests in the CoreValue dashboard:

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure with custom CoreValue headers
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('COVA_API_KEY'),
    api_base='https://gateway.corevalue.dev/v1',
    extra_headers={
        # Session tracking
        'Cova-Property-Session-Id': 'dspy-example-session',
        'Cova-Property-Session-Name': 'Question Answering',

        # User tracking
        'Cova-User-Id': 'user-123',

        # Custom properties for filtering
        'Cova-Property-Environment': 'production',
        'Cova-Property-Module': 'chain-of-thought',
        'Cova-Property-Version': '1.0.0'
    }
)
dspy.configure(lm=lm)

# Use normally
qa = dspy.ChainOfThought('question -> answer')
response = qa(question="What is DSPy?")
print(response.answer)

CoreValue Prompts Integration

Use CoreValue Prompts for centralized prompt management with DSPy signatures:

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure with prompt parameters
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('COVA_API_KEY'),
    api_base='https://gateway.corevalue.dev/v1',
    extra_body={
        'prompt_id': 'customer-support-prompt-id',
        'version_id': 'version-uuid',
        'environment': 'production',
        'inputs': {
            'customer_name': 'Sarah',
            'issue_type': 'technical'
        }
    }
)
dspy.configure(lm=lm)

Learn more about Prompts with AI Gateway.

Advanced Features

Rate Limiting

Configure rate limits for your DSPy applications:

lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('COVA_API_KEY'),
    api_base='https://gateway.corevalue.dev/v1',
    extra_headers={
        'Cova-RateLimit-Policy': 'basic-100'
    }
)

Caching

Enable intelligent caching to reduce costs:

lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('COVA_API_KEY'),
    api_base='https://gateway.corevalue.dev/v1',
    cache=True  # DSPy's built-in caching works with CoreValue
)

Session Tracking for Multi-Turn Conversations

Track entire conversation flows in DSPy programs:

import uuid

session_id = str(uuid.uuid4())

lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('COVA_API_KEY'),
    api_base='https://gateway.corevalue.dev/v1',
    extra_headers={
        'Cova-Property-Session-Id': session_id,
        'Cova-Property-Session-Name': 'Customer Support',
        'Cova-Property-Session-Path': '/support/chat'
    }
)
dspy.configure(lm=lm)

# All calls in this session will be grouped together
qa = dspy.ChainOfThought('question -> answer')

# Multiple turns
response1 = qa(question="What is your return policy?")
response2 = qa(question="How long does shipping take?")
response3 = qa(question="Do you ship internationally?")

# View the full conversation in CoreValue Sessions

DSPy

Introduction

Integration Steps

Complete Working Examples

Basic Chain of Thought

Custom Generation Configuration

Tracking with Custom Properties

CoreValue Prompts Integration

Advanced Features

Rate Limiting

Caching

Session Tracking for Multi-Turn Conversations

AI Gateway Overview

Provider Routing

Model Registry

Prompt Management

Custom Properties

Sessions

Rate Limiting

Caching

On this page