Production LLM system design checklist.

The whole course on one page: the seven-section design document, the system on a page, the reviewer's hard questions, the common gaps, and a pre-ship checklist. Bring it to any design review.

The seven sections System on a page Hard questions Common gaps Pre-ship checklist Durable habits

The seven-section design document

Each section answers exactly one question. Six of the seven you already wrote, as your five projects.

Section	Answers	From
Problem & scope	what does this do, for whom, at what bar?	goals
Boundary	what is code, what is model?	S2
Context & retrieval	what does the model see?	P1
Cost, latency, reliability	what are the ceilings, and how does it fail safely?	P2
Architecture & security	which pattern, and how is it defended?	P3
Evals & observability	how do you know it works, and stays working?	P4
Trade-offs & risks	what did you choose against, and what remains?	all

The whole system on one page

A reviewer should grasp it in thirty seconds. If it does not fit on a page, you do not fully understand it yet.

flowchart LR
  IN["Request"] --> CODE["Code shell
auth, rules"]
  CODE --> RET["Retrieve + rank"]
  RET --> M{"Model core"}
  M --> GUARD["Validate + guardrails"]
  GUARD --> ACT["Action / answer"]
  TR[("Traces to evals to dashboards")] -.-> M
  TR -.-> GUARD
  classDef code fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
  classDef model fill:#EEE6FF,stroke:#1F2937,color:#0E1726;
  classDef ops fill:#DCEBFE,stroke:#1F2937,color:#0E1726;
  class CODE,GUARD,ACT code; class M model; class TR ops;

Pre-answer the reviewer's hard questions

"Why not just an agent?"

Answer with the three gates and where your task landed.

"What's your worst failure?"

Name it, show the fallback ladder and the runbook entry.

"What happens under injection?"

The trust boundary and least-privilege tools.

"Whose tools are those?"

Name every imported MCP server, and say how it is pinned and reviewed.

"What if it dies mid-run?"

Checkpoints, idempotency keys, and which steps are safe to replay.

"How do you know it works?"

The eval pass-rate and the online sampling.

Four gaps the clinic always finds

No number

"Fast and cheap" with no ceiling. Add the budget.

Sections don't connect

The architecture ignores the context design.

Security bolted on

Injection mentioned but tools still over-privileged.

No way to know

A beautiful design with no evals. A hope, not a system.

Pre-ship checklist

Do not ship until every box is honestly ticked.

Boundary drawn: the model does the smallest fuzzy job, wrapped in code.
Context engineered: ranked, trimmed, and you can name what you excluded.
Budget set: a cost and latency ceiling, typical and worst case.
Fallback ladder: it degrades instead of breaking; effects are idempotent.
Tools least-privilege: irreversible actions need approval.
Imported tools accounted for: every MCP server named, pinned, allow-listed, descriptions reviewed.
Long runs resumable: checkpointed state, so an interrupted run resumes instead of restarting and re-firing side effects.
Injection defended: untrusted content cannot trigger a high-privilege action.
Evals green: a golden set incl. a safety case, run on every change.
Traceable: every request replayable; three dashboards with alert lines.
Runbook: the top failures have symptom to check to act entries.
Trade-offs named: you can say what you chose against and the residual risk.

Four habits that outlast any model release

Start simple

Workflow before agent, smallest fuzzy job. The reflex that ages well.

Put a number on it

Budgets and evals turn opinions into decisions.

Assume hostility + failure

Untrusted input, flaky model. Design for both by default.

Feed the flywheel

Every failure becomes a test. Compounding beats clever.

The whole course in one line: not another demo, a system that holds.