Every lab ships as runnable code
TypeScript and OpenAI, behind one provider seam you can swap. Clone the companion repo, or jump to a lab's code under its week below.
github.com/ehsangazar/maven-llms-and-agents-6-weeks ↗A field guide to the tools
Each week below ends with the common tools for that stage of the stack. Every tool has a decision-first page: what it is, the one job it does, when to reach for it, when to skip it, and where it fits your labs. The tools are a means; the architecture is the lesson.
Pre-course · set the target
Before Jul 13
Week 1 · Foundations: workflows, agents & the code/model boundary
Jul 13 – 19
S1 · Why demos die in production, and when you even need an agent
The failure modes that show up the moment a demo meets real traffic. Workflows vs agents: start simple, because most tasks want a workflow, not an autonomous agent. The five workflow patterns (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) and the course's decision map for choosing between them. Tue Jul 14.
/llms-and-agents/s01-why-demos-die 21 slidesS2 · The deterministic / model boundary
What to hand the model versus keep in code, the single decision that most shapes reliability. Workshop: place your own system on the workflow-to-agent spectrum and draw its boundary. Thu Jul 16.
/llms-and-agents/s02-code-model-boundary cheat sheet Reference · cheat sheetWorkflow Patterns Cheat Sheet
Week 1 on one page: workflow vs agent, the five patterns, the three-gate decision map, the four ways demos die, and the antipatterns. Skim before you reach for an agent.
/cheatsheets/workflow-patterns hands-on lab Lab 1 · build itBuild a Workflow Router
Classify each request and route it to a rules handler, a small model, or a big model, with validation and a fallback. Starter code, steps, acceptance criteria. Report your cost split.
/llms-and-agents/lab-workflow-routerWeek 2 · Context engineering & retrieval
Jul 20 – 26
S3 · What goes in the window, and why cramming hurts
Context engineering as the real lever, not prompt wording. RAG done properly: hybrid search, re-ranking, chunking, and agentic retrieval that decides for itself when and what to fetch. Memory and how multi-turn context degrades. Tue Jul 21.
/llms-and-agents/s03-context-engineering 23 slidesS4 · Workshop: design your context pipeline
For your system: what to fetch, how to rank it, and what to deliberately keep out of the window. Thu Jul 23.
/llms-and-agents/s04-context-pipeline artifact Project 1 · due Sun Jul 26P1 · Context & Retrieval Design
A design for what your model sees on every request: sources, ranking, chunking, and the exclusions. The first of the five artifacts.
/llms-and-agents/p1-context-retrieval-design cheat sheet Reference · cheat sheetRAG & Context Engineering Cheat Sheet
The window as a budget, lost-in-the-middle, the pipeline, hybrid search, chunking, re-ranking, agentic retrieval, memory, and a params quick-reference.
/cheatsheets/rag-context hands-on lab Lab 2 · build itBuild a Hybrid-Search RAG Pipeline
Chunk, retrieve with dense + BM25, fuse, re-rank to a few, assemble to a token budget, and refuse to answer when nothing is relevant. Starter code included.
/llms-and-agents/lab-hybrid-ragWeek 3 · Cost, latency & reliability
Jul 27 – Aug 2
S5 · Budgets, routing, and designing for failure
Per-request cost and latency ceilings, and the levers that hold them: caching, batching, streaming, model routing, and right-sizing. Then designing for non-determinism, retries, timeouts, fallbacks, and graceful degradation. The densest week. Tue Jul 28.
/llms-and-agents/s05-cost-latency-reliability 31 slidesS6 · Workshop: budget + failure-mode map
Put numbers on your system: a cost and latency budget, and a first pass at how each part fails and what happens when it does. Thu Jul 30.
/llms-and-agents/s06-budget-failure-map artifact Project 2 · due Sun Aug 2P2 · Cost & Latency Budget + Failure-Mode Runbook
Ceilings you can defend and the runbook you started for when they're breached. The runbook is finished in Week 5.
/llms-and-agents/p2-cost-latency-budget cheat sheet Reference · cheat sheetCost, Latency & Reliability Cheat Sheet
The cost formula and pricing math, the five levers, caching, routing, the fallback ladder, the retry-idempotency trap, and a request-budget template.
/cheatsheets/cost-latency hands-on lab Lab 3 · build itAdd a Budget, Cache & Fallback Ladder
Wrap a model call with a cost/latency budget, a semantic cache, a retry-and-fallback ladder, and an idempotency key so a retry can't double-charge. Starter code included.
/llms-and-agents/lab-budget-cache-fallbackWeek 4 · Agent architecture & security
Aug 3 – 9
S7 · Agent patterns and their failure modes
The loop and its four stops, tools as typed contracts, MCP, three ways to plan, the compounding maths that sets your step cap, durable runs that resume instead of restarting, and single versus multi-agent. Every pattern mapped to how it breaks, with runnable TypeScript for each. Tue Aug 4.
/llms-and-agents/s07-agent-architecture 34 slidesS8 · Securing agents + choose your architecture
Why prompt injection cannot be prompted away, the three shapes of the attack, the lethal trifecta, and zero-click exfiltration. Then five layers you can run: the trust boundary, scoped capabilities and taint, approval gates, egress allow-lists, supply-chain pinning, and a red-team corpus. Workshop: pick your architecture and threat-model it. Thu Aug 6.
/llms-and-agents/s08-securing-agents artifact Project 3 · due Sun Aug 9P3 · Agent Architecture Decision + Threat Model
The pattern you chose, why, and the threats it exposes with the mitigations you'll rely on.
/llms-and-agents/p3-agent-architecture-threat-model cheat sheet Reference · cheat sheetAgent Patterns & Prompt-Injection Cheat Sheet
Agent anatomy, tools as contracts, the ReAct loop, single vs multi-agent, prompt injection (OWASP LLM01), defense-in-depth, the trust boundary, and real cases.
/cheatsheets/agent-security hands-on lab Lab 4 · build itBuild a Tool-Using Agent with Guardrails
A ReAct agent with least-privilege tools, a step cap, an approval gate before writes, and an injected document it must refuse. Starter code + a threat model.
/llms-and-agents/lab-guardrailed-agentWeek 5 · Evals & observability
Aug 10 – 16
S9 · Trajectory evals, not just outputs
Why grading the final answer isn't enough for agents, told through the chatbot ruling that made an airline liable. Three blocks: what to assert about a trajectory (required, forbidden, budget), who grades the fuzzy half and how to score the judge against its own baseline, and when a result blocks a release. Offline sets, online sampling, and the loop where failed production traces become permanent cases. Tue Aug 11.
/llms-and-agents/s09-trajectory-evals 28 slidesS10 · Workshop: harness, tracing & runbook
The span recorder and the replay test written as code, the three dashboards and why a mean and an averaged p95 both lie, content-hashed prompt versions, and where a human stays. Finish the failure-mode runbook. Thu Aug 13.
/llms-and-agents/s10-harness-tracing artifact Project 4 · due Sun Aug 16P4 · Eval Harness Plan + Failure-Mode Runbook
How you'll catch regressions before users do, and the runbook completed for when something still gets through.
/llms-and-agents/p4-eval-harness-plan cheat sheet Reference · cheat sheetEvals & Observability Cheat Sheet
Output vs trajectory evals, the three graders, the four metrics, offline vs online, the flywheel, an eval-case and a trace in JSON, and the three dashboards.
/cheatsheets/evals-observability hands-on lab Lab 5 · build itBuild an Eval Harness
A golden set, a runner, rule + LLM-judge graders, a trajectory check, and a pass-rate report that catches a regression you deliberately introduce. Starter code included.
/llms-and-agents/lab-eval-harnessWeek 6 · Capstone: design, present, defend
Aug 17 – 23
S11 · Capstone design clinic
The end-to-end production design that pulls every decision from the six weeks together, with live instructor feedback in the room. Tue Aug 18.
/llms-and-agents/s11-capstone-clinic 11 slidesS12 · Design-review presentations + wrap
Present and defend your system design the way you would to a staff panel. Return to your Week-1 goals, and leave with what to practise next. Thu Aug 20.
/llms-and-agents/s12-design-review artifact Project 5 · due Thu Aug 20P5 · Capstone: System Design Document + Review Presentation
One document for the whole system, every decision defended, plus the review you present live. The artifact you can take to work on Monday.
/llms-and-agents/p5-capstone-design-doc cheat sheet Reference · cheat sheetProduction LLM System Design Checklist
The seven-section design document, the whole system on one page, the reviewer's hard questions, the common gaps, and a pre-ship checklist. Bring it to any review.
/cheatsheets/production-checklist hands-on lab Lab 6 · build itCapstone: Integrate & Defend
Combine Labs 1-5 into one system behind a single entry point, write the seven-section design doc, run your eval harness on it, and prepare the five-minute walkthrough.
/llms-and-agents/lab-capstone-integration