DSPy

The mental model

Declare the task; let an optimizer write the prompt.

The whole idea of DSPy is to stop hand-writing prompt strings. Instead you declare a signature, a typed spec of the inputs and outputs a step should have, and compose signatures into modules. That gives you a running program immediately, with DSPy generating a reasonable prompt for you. The payoff comes next: you hand the program, a trainset of examples, and a metric to an optimizer, and it compiles your program, searching for the instruction wording and few-shot demonstrations that score best on your metric.

So the prompt is no longer a string you babysit; it is an artifact the optimizer produces. You change the model or the data, you re-compile, you do not re-write. That reframing, prompt engineering as an optimization problem, is what DSPy is for.

flowchart LR
  P["your program
Signatures + Modules"] --> OPT{"optimizer
compile"}
  T["trainset
Examples"] --> OPT
  M["metric
a scoring function"] --> OPT
  OPT --> C["compiled program
tuned instructions + few-shot demos"]
  classDef code fill:#D6F5E3,stroke:#1B7F46,color:#09244B;
  classDef model fill:#EEE6FF,stroke:#6D28D9,color:#09244B;
  class P,T,M code;
  class OPT,C model;
    

Install & setup

One package, one model handle.

DSPy is a single Python package. It talks to any provider through one dspy.LM class (LiteLLM under the hood), so you switch models by changing a string. Set your provider key, build an LM, and register it once with dspy.configure.

# one package, plus whichever provider key you use
pip install dspy

export OPENAI_API_KEY="sk-..."   # or ANTHROPIC_API_KEY, etc.

import dspy

# one LM class for every provider; the string is "provider/model"
lm = dspy.LM("openai/gpt-4o-mini", temperature=0.0)
dspy.configure(lm=lm)

# swap provider by changing the string only:
# dspy.LM("anthropic/claude-sonnet-4-5")

set the LM once, globally; every module you build uses it

Signatures

A typed spec of inputs and outputs.

A signature declares what a step does, not how to prompt for it. Two forms. The inline string form is the fastest: field names on each side of an arrow, comma-separated. The names carry the meaning, so "document -> summary" already tells DSPy enough to build a prompt.

# inline: "inputs -> outputs", field names carry the meaning
classify = dspy.Predict("sentence -> sentiment")
print(classify(sentence="I loved every minute").sentiment)

# multiple fields on either side, comma-separated
qa = dspy.Predict("context, question -> answer")

The class-based form is what you reach for in real code: a docstring becomes the task instruction, and dspy.InputField / dspy.OutputField carry descriptions and Python type annotations. Types are enforced, so a Literal constrains the output to a fixed set and a float is parsed for you.

from typing import Literal

class Classify(dspy.Signature):
    """Classify the sentiment of a customer message."""   # the instruction
    message: str = dspy.InputField(desc="the raw customer message")
    sentiment: Literal["positive", "neutral", "negative"] = dspy.OutputField()
    confidence: float = dspy.OutputField(desc="0 to 1")

classify = dspy.Predict(Classify)
out = classify(message="the refund never arrived")
print(out.sentiment, out.confidence)   # "negative" 0.92

Modules

Strategies that turn a signature into calls.

A module takes a signature and decides how to prompt for it. Same signature, different module, different behaviour. Three you will use constantly: Predict (one call, fill the fields), ChainOfThought (adds a hidden reasoning field before the answer), and ReAct (a tool-using agent loop over your Python functions).

# Predict: fill the signature in a single call
predict = dspy.Predict("question -> answer")

# ChainOfThought: inserts a "reasoning" field before the answer
cot = dspy.ChainOfThought("question -> answer")
print(cot(question="A bat and ball cost 1.10 total ...").answer)

# ReAct: reason-act loop that calls your tools until it can answer
def get_weather(city: str) -> str:
    """Return the current weather for a city."""   # docstring = tool desc
    return f"18C and cloudy in {city}."

agent = dspy.ReAct("question -> answer", tools=[get_weather], max_iters=6)
print(agent(question="What should I wear in Oslo today?").answer)

Every module returns a Prediction whose attributes are the output fields of the signature. Because the module is a strategy, not a prompt, an optimizer can compile any of them the same way.

Composing

A custom module is just __init__ and forward.

Real programs chain several modules. Subclass dspy.Module, create sub-modules in __init__, and wire them together in forward, which is plain Python. Retrieval, branching, loops: all ordinary code between module calls. The optimizer later tunes every sub-module inside it at once.

class MultiHop(dspy.Module):
    def __init__(self):
        super().__init__()
        self.generate_query = dspy.ChainOfThought("question -> search_query")
        self.answer         = dspy.ChainOfThought("context, question -> answer")

    def forward(self, question):
        query   = self.generate_query(question=question).search_query
        context = search(query)          # your retriever, plain Python
        return self.answer(context=context, question=question)

program = MultiHop()
program(question="who wrote the paper behind DSPy?")

Examples & data

Data is a list of Examples with inputs marked.

DSPy carries data as dspy.Example objects, dict-like records of named fields. The one required step is .with_inputs(...), which tells DSPy which fields are given at inference time; everything else is treated as a label to grade against. A trainset is just a list of these.

# an Example holds fields; mark which are inputs vs labels
ex = dspy.Example(question="capital of France?", answer="Paris").with_inputs("question")

print(ex.inputs())   # Example({question: ...})  -> fed to the program
print(ex.labels())   # Example({answer: ...})    -> used by the metric

trainset = [
    dspy.Example(question=q, answer=a).with_inputs("question")
    for q, a in raw_pairs
]

Metrics

A function that scores a prediction.

The metric is the ruler the optimizer climbs. At its simplest it is a function taking (example, prediction, trace=None) and returning a number or a bool. When correctness is fuzzy, make the metric itself a small DSPy program: an LLM-as-judge that scores faithfulness. This is the same evaluation discipline as the Week 5 eval harness, expressed as code the optimizer can call thousands of times.

# 1) a plain function metric
def exact_match(example, pred, trace=None):
    return example.answer.lower() == pred.answer.lower()

# 2) an LLM-as-judge metric, built from a signature
class Judge(dspy.Signature):
    """Is the prediction faithful to the reference answer?"""
    question: str   = dspy.InputField()
    reference: str  = dspy.InputField()
    prediction: str = dspy.InputField()
    correct: bool    = dspy.OutputField()

judge = dspy.ChainOfThought(Judge)

def llm_metric(example, pred, trace=None):
    r = judge(question=example.question, reference=example.answer,
              prediction=pred.answer)
    return r.correct

DSPy also ships built-in metrics, for example dspy.evaluate.SemanticF1 for open-ended answers, so you do not always have to write your own.

Optimizers

Compile the program against the metric.

An optimizer (historically called a teleprompter) takes your program, trainset, and metric, and returns a new program with better prompts. The core loop is the same across optimizers: bootstrap traces by running your own program, propose candidate instructions and demonstrations, score each candidate on the metric, keep the winners.

flowchart TB
  S["student program"] --> B["bootstrap
run on trainset, keep traces that score"]
  B --> PR["propose
candidate instructions plus demos"]
  PR --> EV{"evaluate
score each candidate on the metric"}
  EV -->|"better"| K["keep candidate"]
  EV -->|"worse"| PR
  K --> OUT["compiled program"]
  classDef code fill:#D6F5E3,stroke:#1B7F46,color:#09244B;
  classDef model fill:#EEE6FF,stroke:#6D28D9,color:#09244B;
  class S,OUT code;
  class B,PR,EV,K model;
    

You always call optimizer.compile(program, trainset=...) and get back a compiled program you use exactly like the original. Pick the optimizer by budget and data size.

# few-shot only: bootstrap demonstrations from your own program
bootstrap = dspy.BootstrapFewShot(metric=exact_match, max_bootstrapped_demos=4)
compiled  = bootstrap.compile(program, trainset=trainset)

# instructions AND demos, searched with Bayesian optimization
mipro    = dspy.MIPROv2(metric=exact_match, auto="medium")
compiled = mipro.compile(program, trainset=trainset, valset=devset)

# reflective, feedback-driven evolution (DSPy 3.x)
gepa     = dspy.GEPA(metric=feedback_metric, auto="light",
                     reflection_lm=dspy.LM("openai/gpt-4o"))
compiled = gepa.compile(program, trainset=trainset, valset=devset)

GEPA is the DSPy 3.x reflective optimizer: its metric can return dspy.Prediction(score=..., feedback=...) so the optimizer reads why a trace failed, not just a number, and evolves prompts along a Pareto frontier.

Evaluate

Measure before and after, on a held-out set.

dspy.Evaluate runs your program over a devset with your metric and returns an aggregate score, multithreaded and with a progress bar. Always measure the un-optimized baseline first, then the compiled version, so the improvement is a number, not a vibe.

evaluate = dspy.Evaluate(devset=devset, metric=exact_match,
                         num_threads=16, display_progress=True, display_table=5)

result = evaluate(program)      # baseline
print(result.score)            # e.g. 54.0

evaluate(compiled)             # after compile, e.g. 71.0

Save & load

The compiled prompts are an artifact you ship.

Compilation costs money and time, so you do it once, offline, and serve the saved result. Save state only as JSON (the instructions and demos) and reload it into a freshly constructed program, or save the whole program (architecture plus state) and load it without rebuilding the class.

# state only (prompts + demos) as JSON; recreate the class to load
compiled.save("program.json")
loaded = MultiHop()
loaded.load("program.json")

# whole program (architecture + state), dspy >= 2.6
compiled.save("program_dir/", save_program=True)
loaded = dspy.load("program_dir/")

Version the JSON next to the code that built it, and next to the metric and model it was tuned for. A compiled prompt is a build output, treat it like one.

Observability

See every prompt the optimizer produced.

Because DSPy writes prompts for you, you need to be able to read what it wrote. Three levels: dspy.inspect_history for a quick local peek at raw LM calls, MLflow autologging for a full traced timeline of modules, LM calls, and optimizer steps, and callbacks for custom hooks.

# quick: print the last N raw LM prompts and completions
dspy.inspect_history(n=3)

# full tracing: one line auto-logs every module, LM call, and compile step
import mlflow
mlflow.dspy.autolog()
mlflow.set_experiment("support-triage")

# custom: subclass BaseCallback for granular hooks
from dspy.utils.callback import BaseCallback

class LogModule(BaseCallback):
    def on_module_end(self, call_id, outputs, exception):
        print("module output:", outputs)

dspy.configure(callbacks=[LogModule()])

Putting it together

A ticket-triage classifier, compiled end to end.

The full arc in one file: declare the task, compose a module, build data, define a metric, measure the baseline, compile, measure again, save the winner. The before/after numbers are the whole point.

import dspy
from typing import Literal

lm = dspy.LM("openai/gpt-4o-mini", temperature=0.0)
dspy.configure(lm=lm)

# 1. declare the task as a signature
class Triage(dspy.Signature):
    """Route a support ticket to the right queue."""
    ticket: str = dspy.InputField()
    queue: Literal["billing", "technical", "account"] = dspy.OutputField()

# 2. compose it into a module (chain-of-thought reasoning here)
program = dspy.ChainOfThought(Triage)

# 3. data: labelled Examples, inputs marked
trainset = [dspy.Example(ticket=t, queue=q).with_inputs("ticket") for t, q in train_pairs]
devset   = [dspy.Example(ticket=t, queue=q).with_inputs("ticket") for t, q in dev_pairs]

# 4. metric: does the predicted queue match the label?
def accuracy(example, pred, trace=None):
    return example.queue == pred.queue

# 5. measure the un-optimized baseline
evaluate = dspy.Evaluate(devset=devset, metric=accuracy, num_threads=16,
                         display_progress=True)
evaluate(program)                       # e.g. 62.0

# 6. compile: let the optimizer write the instruction and pick demos
mipro     = dspy.MIPROv2(metric=accuracy, auto="medium")
optimized = mipro.compile(program, trainset=trainset, valset=devset)

# 7. measure again, then save the winner
evaluate(optimized)                     # e.g. 81.0
optimized.save("triage.json")

you never wrote a prompt string; the optimizer wrote it, and you can read it with inspect_history

Production

From notebook to service.

• Optimize offline, serve the artifact. Compilation is a build step. Save the compiled JSON and load it at serve time; never re-optimize per request.
• Version the artifact next to the code, the metric, and the exact model it was tuned for. A prompt tuned for one model is not portable to another.
• Budget the optimizer. MIPROv2 and GEPA fan out many LM calls. Start with auto="light", use a cheaper task model, and cap num_threads.
• Lean on the cache. DSPy caches LM calls by default, which cuts cost during repeated compile and eval runs; keep it on.
• Trace in production with MLflow autolog and alert on metric drift, so you know when it is time to re-compile.

Gotchas

What trips people up.

The principle it teaches

Where it fits your labs

A Week 5 tool that bridges the prompting discipline from Week 1 and the evals you build in Week 5. Take a step you currently prompt by hand, declare it as a signature, and let an optimizer compile it against your eval metric so the improvement is measured, not asserted. It reframes prompt engineering as the optimization problem the course keeps pointing at.