Week 5 · Field guide · Observability standard

OpenTelemetry & OpenLLMetry

The vendor-neutral tracing standard, plus LLM-specific instrumentation: emit spans once and send them anywhere.

open standardvendor-neutralOpenLLMetry

What it is

Instrument once, send anywhere.

OpenTelemetry is the vendor-neutral tracing standard. OpenLLMetry, by Traceloop, is OpenTelemetry instrumentation built for LLM apps. Together they let you emit spans once and ship them to any backend: Langfuse, Phoenix, Datadog, and others. The instrumentation is decoupled from the destination, so your code describes what happened, and the backend is a config choice, not a rewrite.

The one job

Instrument once and stay backend-agnostic.

Describe your retrieve, model, and tool steps as spans one time, and keep the freedom to point them at whichever backend you choose, now or later.

Reach for it when

You do not want to lock tracing to one vendor and want an exit ramp.
You already run OpenTelemetry in your stack and want LLM spans in the same pipeline.
You want spans for retrieve, model, and tool steps, exactly Session 10's trace shape.
You need SDK-level control over span names, attributes, and nesting.

Skip it when

You are at the earliest prototype: a vendor SDK is faster to start with.
You want prebuilt dashboards now, not raw spans you still have to route.
You want a metrics library or prompt tools in the box: pair it with Phoenix or Langfuse.

A minimal look

A span around the call.

import { trace } from "@opentelemetry/api";
import OpenAI from "openai";

const openai = new OpenAI();
const tracer = trace.getTracer("agent");

await tracer.startActiveSpan("model", async (span) => {
  span.setAttribute("llm.model", "gpt-4o");   // standard attrs
  const res = await openai.chat.completions.create({ model: "gpt-4o", messages });
  span.end();                                // export to ANY OTel backend
});

The principle it teaches

Course principle

Your course literally teaches OpenTelemetry-style tracing (Session 10), and this is that pattern, standardized. Instrument the boundaries once, avoid vendor lock-in on observability, and let the backend be a decision you can change without touching your code.

Where it fits your labs

This is a Week 5 tool. The Session 10 tracing you build is OTel-shaped by design, and the trace JSON in the evals cheat sheet is an OTel-style span tree, so instrumenting with OpenTelemetry is doing the lesson directly.

Use in

Week 5 Session 10 tracing, where retrieve, model, and tool steps become spans.

week 5

Pairs with

Any backend above; the evals cheat sheet trace JSON is an OTel-shaped span tree.