Session 11 · Week 6 · Tue 18 Aug 2026 · 90 min

Everything, on one page.

Six weeks of decisions become one artifact: a system design document for a production LLM system. Tonight we assemble it, then pressure-test yours live, so Thursday you present something that holds.

Ehsan Gazar
Production-Ready Systems with LLMs and Agents · session 11 of 12
Where we're going · 90 minutes

The run sheet.

0–12Six weeks → one documentthe structure
12–35Section by sectionwhat each must answer
35–50The whole system on a pagethe overview diagram
50–80Live design clinicwe pressure-test real ones
80–90Prep for Thursdaypresent + defend
You’ll leave able to assemble six artifacts into one design document, draw the whole system on a page, anticipate a reviewer’s hard questions, and close the gaps the clinic surfaces.
It was one design all along

Your four projects were the sections.

Nothing new to invent tonight, you've been writing this document since Week 2. The capstone is the connective tissue: showing every decision fits with the others.

wk 1–2
Boundary + context
P1
wk 3
Cost + reliability
P2
wk 4
Architecture + security
P3
wk 5
Evals + ops
P4
The system design document

Seven sections, each answering one question.

SectionAnswersFrom
Problem & scopewhat does this system do, for whom, at what bar?goals
Boundarywhat's code, what's model?S2
Context & retrievalwhat does the model see?P1
Cost, latency, reliabilitywhat are the ceilings, and how does it fail safely?P2
Architecture & securitywhich pattern, and how is it defended?P3
Evals & observabilityhow do you know it works, and stays working?P4
Trade-offs & riskswhat did you choose against, and what remains?all
The overview diagram

The whole system, one diagram.

A reviewer should grasp your system in thirty seconds. This is the picture: boundary, retrieval, the model core, guardrails, and the observability wrapped around it.

flowchart LR
  IN["Request"] --> CODE["Code shell
auth, rules"]
  CODE --> RET["Retrieve + rank"]
  RET --> M{"Model core"}
  M --> GUARD["Validate + guardrails"]
  GUARD --> ACT["Action / answer"]
  TR[("Traces → evals → dashboards")] -.-> M
  TR -.-> GUARD
  classDef code fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
  classDef model fill:#EEE6FF,stroke:#1F2937,color:#0E1726;
  classDef ops fill:#DCEBFE,stroke:#1F2937,color:#0E1726;
  class CODE,GUARD,ACT code;
  class M model;
  class TR ops;
A full design, in one glance

Worked example: a billing assistant.

Problemanswer billing questions for paid users · <2s · never invent policy
Boundarycode: auth, routing, refunds · model: understand the ask + draft
Contexthybrid over billing docs + this user's invoices · rerank to 5 · cite
Cost / reliab.small model default, escalate on low confidence · fallback ladder · <$0.01
Architectureworkflow (route → retrieve → answer), not an agent · tools least-privilege
Evals / ops40 golden cases + 5% online sampling · trace every request · 3 dashboards
Top riskinjection via a support doc · mitigated: no-write tools + approval on refunds
Present like staff

Anticipate the hard questions.

A strong design names its own weak points before the reviewer does. For every major decision, be ready with the trade-off and the trigger, the S1 move, applied to the whole system.

"Why not just an agent?"
answer with the three gates and where your task landed.
"What's your worst failure?"
name it, show the fallback ladder and the runbook entry.
"What happens under injection?"
the trust boundary and least-privilege tools.
"How do you know it works?"
the eval pass-rate and the online sampling.
What the clinic usually finds

Four gaps to close before Thursday.

1
No number
"fast and cheap" with no ceiling. Add the budget from Week 3.
2
Sections don't connect
the architecture ignores the context design. Show they fit.
3
Security bolted on
injection mentioned but tools still over-privileged. Push it into the design.
4
No way to know it works
a beautiful design with no evals. It's a hope, not a system.
Live clinic · ~30 min

Assemble, then we stress it.

Draft your one-page overview and your trade-offs section now. We'll take volunteers and run the design clinic: I ask the hard questions, the group finds the gaps, you leave knowing exactly what to firm up.

1Stitch your four projects into the seven sections.Mostly copy-assemble; write the connective sentences.
2Draw the system on one page.If it doesn't fit on a page, your design isn't clear yet.
3Write the trade-offs section.Three big decisions, what you chose against, what risk remains.
Recap · then Thursday

One document. Every decision, defended.

Seven sections that connect, a system on one page, hard questions pre-answered, and gaps closed. That's the System Design Document, and it's the thing you take to work on Monday.

→
Thursday · S12
Present and defend your design to the group, then return to your Week-1 goals. Output: Project 5.
Thu 20 Aug
Tonight → Thursday
tighten the doc, rehearse the 5-minute walkthrough, bring your worst-case question.
prep