Prompt Management Overview

When building LLM applications, you need to manage prompt templates, handle variable substitution, and deploy changes without code deployments. Prompt Management solves this by providing a centralized system for composing, versioning, and deploying prompts with dynamic variables.

Why Prompt Management?

Traditional prompt development involves hardcoded prompts in application code, messy string substitution, and frustrating and rebuilding deployments for every iteration. This creates friction that slows down experimentation and your team's ability to ship.

Quick Start

Build a prompt in the Playground. Save any prompt with clear commit histories and tags.

Experiment with different variables, inputs, and models until you reach desired output. Variables can be used anywhere, even in tool schemas.

Use your prompt instantly by referencing its ID in your AI Gateway. No code changes, no rebuilds.

Prompt Management is available for Chat Completions on the AI Gateway. Simply include prompt_id and inputs in your chat completion requests.

import { OpenAI } from "openai";
import { CovaChatCreateParams } from "@cova/helpers";

const openai = new OpenAI({
  baseURL: "https://gateway.corevalue.dev/v1",
  apiKey: process.env.COVA_API_KEY,
});

const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  prompt_id: "abc123", // Reference your saved prompt
  environment: "production", // Optional: specify environment
  messages: [
    {
      role: "user",
      content: "Hello there!"
    }
  ], // optional: saved prompt also provides messages
  inputs: {
    customer_name: "John Doe",
    product: "AI Gateway"
  }
} as CovaChatCreateParams);

import openai
import os

client = openai.OpenAI(
    base_url="https://gateway.corevalue.dev/v1",
    api_key=os.environ.get("COVA_API_KEY")
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    prompt_id="abc123",  # Reference your saved prompt
    environment="production",  # Optional: specify environment
    inputs={
        "customer_name": "John Doe",
        "product": "AI Gateway"
    }
)

curl https://gateway.corevalue.dev/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $COVA_API_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "prompt_id": "abc123",
    "environment": "production",
    "inputs": {
      "customer_name": "John Doe",
      "product": "AI Gateway"
    }
  }'

Variables

Variables make your prompts dynamic and reusable. Define them once in your prompt template, then provide different values at runtime without changing your code.

Variable Syntax

Variables use the format {{hc:name:type}} where:

• name is your variable identifier

• type defines the expected data type

  {{hc:customer_name:string}}
  {{hc:age:number}}
  {{hc:is_premium:boolean}}
  {{hc:context:any}}

  You are a helpful assistant for {{hc:company:string}}.
  
  The customer {{hc:customer_name:string}} is {{hc:age:number}} years old.
  Premium status: {{hc:is_premium:boolean}}
  
  Additional context: {{hc:context:any}}

Supported Types

Boolean variables accept multiple formats:

• true / false (boolean)
• "yes" / "no" (string)
• "true" / "false" (string)

Schema Variables

Variables can be used within JSON schemas for tools and response formatting. This enables dynamic schema generation based on runtime inputs.

{
  "name": "moviebot_response",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "markdown_response": {
        "type": "string"
      },
      "tools_used": {
        "type": "array",
        "items": {
          "type": "string",
          "enum": "{{hc:tools:array}}"
        }
      },
      "user_tier": {
        "type": "string",
        "enum": "{{hc:tiers:array}}"
      }
    },
    "required": [
      "markdown_response",
      "tools_used",
      "user_tier"
    ],
    "additionalProperties": false
  }
}

{
  "tools": ["search", "calculator", "weather"],
  "tiers": ["basic", "premium", "enterprise"]
}

{
  "name": "moviebot_response",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "markdown_response": {
        "type": "string"
      },
      "tools_used": {
        "type": "array",
        "items": {
          "type": "string",
          "enum": ["search", "calculator", "weather"]
        }
      },
      "user_tier": {
        "type": "string",
        "enum": ["basic", "premium", "enterprise"]
      }
    },
    "required": [
      "markdown_response",
      "tools_used",
      "user_tier"
    ],
    "additionalProperties": false
  }
}

Replacement Behavior

Value Replacement: When a variable tag is the only content in a string, it gets replaced with the actual data type:

"enum": "{{hc:tools:array}}" → "enum": ["search", "calculator", "weather"]

String Substitution: When variables are part of a larger string, normal regex replacement occurs:

"description": "Available for {{hc:name:string}} users" → "description": "Available for premium users"

Keys and Values: Variables work in both JSON keys and values throughout tool schemas and response schemas.

Managing Environments

You can easily manage different deployment environments for your prompts directly in the Cova dashboard. Create and deploy prompts to production, staging, development, or any custom environment you need.

Prompt Partials

When building multiple prompts, you often need to reuse the same message blocks across different prompts. Prompt partials allow you to reference messages from other prompts, eliminating duplication and making your prompt library more maintainable.

Syntax

Prompt partials use the format {{hcp:prompt_id:index:environment}} where:

• prompt_id - The 6-character alphanumeric identifier of the prompt to reference

• index - The message index (0-based) to extract from that prompt

• environment - Optional environment identifier (defaults to production if omitted)

  {{hcp:abc123:0}}                   // Message 0 from prompt abc123 (production)
  {{hcp:abc123:1:staging}}           // Message 1 from prompt abc123 (staging)
  {{hcp:xyz789:2:development}}       // Message 2 from prompt xyz789 (development)

  {{hcp:abc123:0}}
  
  {{hc:user_name:string}}, here's your personalized response:

How It Works

When a prompt containing a partial is compiled:

1. Partial Resolution: The partial tag {{hcp:prompt_id:index:environment}} is replaced with the actual message content from the referenced prompt at the specified index
2. Variable Substitution: After partials are resolved, variables in both the main prompt and the resolved partials are substituted with their values

This order matters: since partials are resolved before variables, you can control variables that exist within the partial from the main prompt's inputs.

{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant for {{hc:company:string}}."
    }
  ]
}

{
  "messages": [
    {
      "role": "user",
      "content": "{{hcp:abc123:0}} Please help me with my account."
    }
  ]
}

{
  "company": "Acme Corp"
}

{
  "role": "user",
  "content": "You are a helpful assistant for Acme Corp. Please help me with my account."
}

Using Prompts

Cova provides two ways to use prompts:

1. AI Gateway Integration - The recommended approach. Use prompts through the Cova AI Gateway for automatic compilation, input tracing, and lower latency.

2. SDK Integration - Alternative integration method for users that need direct interaction with compiled prompt bodies without using the AI Gateway.

Learn more about how prompts are assembled and compiled in the Prompt Assembly guide.

Related Documentation