Week 3 · Field guide · Gateway + guardrails

Portkey

The heaviest LLMOps surface in one place: routing, caching, 40+ guardrails, and observability behind a single gateway, fully open source since March 2026.

gatewayopen source (Apache 2.0)1600+ models40+ guardrails

What it is

The gateway that does everything at once.

Portkey is an AI gateway spanning 1600+ models across 250+ providers, with conditional routing, circuit breakers, semantic caching, 40+ pre-built guardrails, and an MCP gateway for tool access. Where a thin proxy just translates and forwards, Portkey folds the whole cross-cutting stack, cost, safety, and tracing, into one control plane. It went fully open source under Apache 2.0 in March 2026, so you can self-host the same surface the cloud runs.

The one job

Routing, caching, guardrails, and tracing together.

It is the one gateway that does all four at once, so you stop stitching a router, a cache, a guardrail layer, and an observability tool into each other and instead configure them as one thing at the edge.

Reach for it when

You want the heaviest LLMOps surface consolidated in one place.
You need guardrails plus routing plus caching working together, not bolted on.
You want an MCP gateway to broker tool access alongside model calls.
You value self-hostable control now that the whole thing is Apache 2.0.

Skip it when

You only need a thin proxy: LiteLLM is the lighter fit.
You prize minimalism and do not want a broad control plane.
You already have separate observability and guardrails you like.

A minimal look

One endpoint, config in the headers.

import OpenAI from "openai";

// point the OpenAI client at Portkey; policy travels in the headers
const client = new OpenAI({
  baseURL: "https://api.portkey.ai/v1",
  apiKey: process.env.OPENAI_API_KEY,
  defaultHeaders: {
    "x-portkey-api-key": process.env.PORTKEY_KEY,
    "x-portkey-config": process.env.PORTKEY_CONFIG,  // routing + cache + guardrails
  },
});

The principle it teaches

Course principle

Consolidate cross-cutting concerns, cost, safety, and tracing, at the gateway rather than in app code. Portkey is a single tool that touches Weeks 3, 4, and 5 at once, which is exactly why the boundary is the right place for them.

Where it fits your labs

This spans Weeks 3 to 5. The cost work of Week 3, the safety and guardrail work of Week 4, and the observability of Week 5 all have a home in this one gateway, so you can see how a real control plane earns its keep.

Use in

The cost, guardrail, and observability threads across all three weeks.

weeks 3–5

Pairs with

Your app's OpenAI client, pointed at the Portkey endpoint.