The quickstart gets you a working API call in ten minutes. Then real users arrive and it falls over. Tonight is the other 90%: the five things that break a first LLM feature, how to turn a vague prompt into a contract, and the one line that decides whether it ships, between the model and your code.
Over 60% of AI pilots never reach production, and it is almost never the model. A quickstart shows you a happy-path call with a clean string and one polite user. Production hands you long inputs, flaky networks, a real bill, hostile text, and prompts that quietly drift. The gap between a demo and a feature is engineering, not prompting.
The quickstart is a kid on a bike in an empty car park. Shipping is the same bike in rush-hour traffic. Same machine, completely different problem, and tonight is about the traffic.
A model call is a slow request to a service that can be full, rate-limited, or down. The quickstart skips all of it. You don't get to.
Count tokens and truncate or chunk before you send. Wrap the call in a timeout, retry with backoff on 429 and 5xx, and have a fallback for when it still fails.
// the quickstart
const res = await client.messages.create({ model, messages });
return res.content; // 🤞 hope it fits, hope it works
// beyond it — TypeScript, Anthropic SDK
const client = new Anthropic({ maxRetries: 4 }); // retries 429 / 5xx
const body = await clipToBudget(text, TICKET_TOKEN_BUDGET);
try {
return await triage(body, { timeout: 20_000 }); // bound the call
} catch (err) {
if (err instanceof Anthropic.RateLimitError)
return queueForLater(); // honest fallback, not a crash
throw err;
}The model has no memory, so most people resend the entire conversation each call. By message twenty you are paying for nineteen old ones to get one new reply, slower and pricier every turn.
Send only what this turn needs: a rolling window, a running summary, or just the few relevant facts. Context is a budget you spend on purpose, not a log you append to.
flowchart TB
subgraph naive["Resend everything (cost grows)"]
direction LR
n1["msg 1..19"]:::old --> n2["+ new msg"]:::new --> n3["LLM"]:::model
end
subgraph lean["Send what's needed (cost flat)"]
direction LR
l1["running summary"]:::win --> l3["LLM"]:::model
l2["last few + new"]:::new --> l3
end
classDef old fill:#FEE4E2,stroke:#0D1B33,color:#0D1B33;
classDef new fill:#DCEBFE,stroke:#0D1B33,color:#0D1B33;
classDef win fill:#FEF3C7,stroke:#0D1B33,color:#0D1B33;
classDef model fill:#D6F5E3,stroke:#0D1B33,color:#0D1B33;
Concatenating raw input into your prompt is the LLM version of string-building a SQL query. A user can write "ignore the above and…" and now they are steering your system.
Keep instructions and user data in separate roles or clearly delimited blocks. Validate and size-check input first, and never let model output act on anything without a check in code.
flowchart LR
uIN["User input"]:::in --> uV{"Validate and delimit"}:::q
uV -->|too long / junk| uR["Reject or trim"]:::warn
uV -->|ok| uP["Data block, separate from instructions"]:::win
sys["Your instructions"]:::ok --> uLLM["LLM"]:::model
uP --> uLLM
uLLM --> uC{"Check in code"}:::q
uC -->|valid| uOUT["Use it"]:::ok
uC -->|not| uR
classDef in fill:#DCEBFE,stroke:#0D1B33,color:#0D1B33;
classDef q fill:#FEF3C7,stroke:#0D1B33,color:#0D1B33;
classDef win fill:#EEE6FF,stroke:#0D1B33,color:#0D1B33;
classDef ok fill:#D6F5E3,stroke:#0D1B33,color:#0D1B33;
classDef warn fill:#FEE4E2,stroke:#0D1B33,color:#0D1B33;
classDef model fill:#D6F5E3,stroke:#0D1B33,color:#0D1B33;
"Summarize this nicely" is a wish. A contract names the inputs, the exact output shape, and the rules, so the model's job is small and the same every time.
Stop asking the model to be clever. Ask it to fill in a form. A form you can validate, test, and trust the next line of code to read.
// a wish
"Summarize this ticket nicely."
// a contract — the model's whole job is to fill in this form
const TicketTriage = z.object({
summary: z.string(), // one line, <= 200 chars
category: z.enum(["billing", "bug", "account", "other"]),
urgency: z.union([z.literal(1), z.literal(2), z.literal(3)]),
});
// then, in code — parse + validate against the contract
const res = await client.messages.parse({
output_config: { format: zodOutputFormat(TicketTriage) },
messages: [{ role: "user", content: `<ticket>\n${body}\n</ticket>` }],
});
const triage = res.parsed_output; // typed & validated — safe to useA small, structured job is testable. You can build a handful of example inputs with expected outputs and catch a regression when you tweak the prompt, the thing you simply cannot do with "nicely".
| The job | Hand it to… | Why |
|---|---|---|
| Understand messy language | the model | fuzzy, ambiguous, no fixed rules |
| Decide a category or intent | the model | judgement over unstructured text |
| Math, dates, totals, lookups | code | must be exact and repeatable |
| Permissions, payments, writes | code | never let probabilistic output act unchecked |
| Validation & control flow | code | deterministic, testable, cheap |
async function processTicket(raw: string) {
// 1 · code — validate & size the untrusted input
const input = validateInput(raw);
if (!input.ok) return reject(input.reason);
const body = await clipToBudget(input.value, TICKET_TOKEN_BUDGET);
// 2 · model — the thin contract call (retried, structured)
let t: TicketTriage;
try { t = await triage(body); }
catch (err) { return routeToHuman(err); } // honest fallback
// 3 · code — decide & act. The model proposes, code disposes.
const { queue, sla, escalate } = route(t); // deterministic
return dispatch(queue, sla, escalate);
}npm run feature.Before you ship: tokens budgeted, calls retried, context trimmed, input validated, prompt a contract, output schema-checked, and every action in code, not in the model. The quickstart was the easy 10%. Do these and you're in the 40% that reach production.
Production-Ready Systems with LLMs and Agents: An Intensive for Engineers. Six weeks, twelve live sessions (Tue/Thu), five graded projects where you ship real LLM and agent systems, not notebooks. Jul 13 to Aug 23.