Every lab ships as runnable code
TypeScript and OpenAI, behind one provider seam you can swap. Clone the companion repo, or jump to a lab's code under its week below.
github.com/ehsangazar/maven-llms-and-agents-6-weeks ↗A field guide to the tools
Each week below ends with the common tools for that stage of the stack. Every tool has a decision-first page: what it is, the one job it does, when to reach for it, when to skip it, and where it fits your labs. The tools are a means; the architecture is the lesson.
Pre-course · set the target
Before Jul 13
Week 1 · Foundations: workflows, agents & the code/model boundary
Jul 13 – 19
S1 · Why demos die in production, and when you even need an agent
The failure modes that show up the moment a demo meets real traffic. Workflows vs agents: start simple, because most tasks want a workflow, not an autonomous agent. The five workflow patterns (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) and the course's decision map for choosing between them. Tue Jul 14.
/llms-and-agents/s01-why-demos-die 21 slidesS2 · The deterministic / model boundary
What to hand the model versus keep in code, the single decision that most shapes reliability. Workshop: place your own system on the workflow-to-agent spectrum and draw its boundary. Thu Jul 16.
/llms-and-agents/s02-code-model-boundary cheat sheet Reference · cheat sheetWorkflow Patterns Cheat Sheet
Week 1 on one page: workflow vs agent, the five patterns, the three-gate decision map, the four ways demos die, and the antipatterns. Skim before you reach for an agent.
/cheatsheets/workflow-patterns hands-on lab Lab 1 · build itBuild a Workflow Router
Classify each request and route it to a rules handler, a small model, or a big model, with validation and a fallback. Starter code, steps, acceptance criteria. Report your cost split.
/llms-and-agents/lab-workflow-routerWeek 2 · Context engineering & retrieval
Jul 20 – 26
S3 · What goes in the window, and why cramming hurts
Context engineering as the real lever, not prompt wording. RAG done properly: hybrid search, re-ranking, chunking, and agentic retrieval that decides for itself when and what to fetch. Memory and how multi-turn context degrades. Tue Jul 21.
/llms-and-agents/s03-context-engineering 23 slidesS4 · Workshop: design your context pipeline
For your system: what to fetch, how to rank it, and what to deliberately keep out of the window. Thu Jul 23.
/llms-and-agents/s04-context-pipeline artifact Project 1 · due Sun Jul 26P1 · Context & Retrieval Design
A design for what your model sees on every request: sources, ranking, chunking, and the exclusions. The first of the five artifacts.
/llms-and-agents/p1-context-retrieval-design cheat sheet Reference · cheat sheetRAG & Context Engineering Cheat Sheet
The window as a budget, lost-in-the-middle, the pipeline, hybrid search, chunking, re-ranking, agentic retrieval, memory, and a params quick-reference.
/cheatsheets/rag-context hands-on lab Lab 2 · build itBuild a Hybrid-Search RAG Pipeline
Chunk, retrieve with dense + BM25, fuse, re-rank to a few, assemble to a token budget, and refuse to answer when nothing is relevant. Starter code included.
/llms-and-agents/lab-hybrid-ragWeek 3 · Cost, latency & reliability
Jul 27 – Aug 2
S5 · Budgets, routing, and designing for failure
Per-request cost and latency ceilings, and the levers that hold them: caching, batching, streaming, model routing, and right-sizing. Then designing for non-determinism, retries, timeouts, fallbacks, and graceful degradation. The densest week. Tue Jul 28.
/llms-and-agents/s05-cost-latency-reliability 31 slidesS6 · Workshop: budget + failure-mode map
Put numbers on your system: a cost and latency budget, and a first pass at how each part fails and what happens when it does. Thu Jul 30.
/llms-and-agents/s06-budget-failure-map artifact Project 2 · due Sun Aug 2P2 · Cost & Latency Budget + Failure-Mode Runbook
Ceilings you can defend and the runbook you started for when they're breached. The runbook is finished in Week 5.
/llms-and-agents/p2-cost-latency-budget cheat sheet Reference · cheat sheetCost, Latency & Reliability Cheat Sheet
The cost formula and pricing math, the five levers, caching, routing, the fallback ladder, the retry-idempotency trap, and a request-budget template.
/cheatsheets/cost-latency hands-on lab Lab 3 · build itAdd a Budget, Cache & Fallback Ladder
Wrap a model call with a cost/latency budget, a semantic cache, a retry-and-fallback ladder, and an idempotency key so a retry can't double-charge. Starter code included.
/llms-and-agents/lab-budget-cache-fallbackWeek 4 · Agent architecture & security
Aug 3 – 9
S7 · Agent patterns and their failure modes
Tool use, planning loops, memory, and single versus multi-agent. Tools and outputs treated as contracts. Each pattern mapped to exactly how it breaks, so you choose with the failure mode already in view. Tue Aug 4.
/llms-and-agents/s07-agent-architecture 15 slidesS8 · Securing agents + choose your architecture
Prompt injection (OWASP LLM01), tool poisoning, and indirect injection through retrieved content. Tool approvals and allow-lists, least-privilege tooling, and output guardrails. Workshop: pick your architecture and threat-model it. Thu Aug 6.
/llms-and-agents/s08-securing-agents artifact Project 3 · due Sun Aug 9P3 · Agent Architecture Decision + Threat Model
The pattern you chose, why, and the threats it exposes with the mitigations you'll rely on.
/llms-and-agents/p3-agent-architecture-threat-model cheat sheet Reference · cheat sheetAgent Patterns & Prompt-Injection Cheat Sheet
Agent anatomy, tools as contracts, the ReAct loop, single vs multi-agent, prompt injection (OWASP LLM01), defense-in-depth, the trust boundary, and real cases.
/cheatsheets/agent-security hands-on lab Lab 4 · build itBuild a Tool-Using Agent with Guardrails
A ReAct agent with least-privilege tools, a step cap, an approval gate before writes, and an injected document it must refuse. Starter code + a threat model.
/llms-and-agents/lab-guardrailed-agentWeek 5 · Evals & observability
Aug 10 – 16
S9 · Trajectory evals, not just outputs
Why grading the final answer isn't enough for agents. Tool-calling, task-completion, and trajectory / trace-based evals. Offline eval sets plus online evals on sampled traffic, and the loop where failed production traces become new eval cases. Tue Aug 11.
/llms-and-agents/s09-trajectory-evals 12 slidesS10 · Workshop: harness, tracing & runbook
OpenTelemetry-style tracing, cost / latency / quality dashboards, prompt versioning, and human-in-the-loop. Finish the failure-mode runbook. Thu Aug 13.
/llms-and-agents/s10-harness-tracing artifact Project 4 · due Sun Aug 16P4 · Eval Harness Plan + Failure-Mode Runbook
How you'll catch regressions before users do, and the runbook completed for when something still gets through.
/llms-and-agents/p4-eval-harness-plan cheat sheet Reference · cheat sheetEvals & Observability Cheat Sheet
Output vs trajectory evals, the three graders, the four metrics, offline vs online, the flywheel, an eval-case and a trace in JSON, and the three dashboards.
/cheatsheets/evals-observability hands-on lab Lab 5 · build itBuild an Eval Harness
A golden set, a runner, rule + LLM-judge graders, a trajectory check, and a pass-rate report that catches a regression you deliberately introduce. Starter code included.
/llms-and-agents/lab-eval-harnessWeek 6 · Capstone: design, present, defend
Aug 17 – 23
S11 · Capstone design clinic
The end-to-end production design that pulls every decision from the six weeks together, with live instructor feedback in the room. Tue Aug 18.
/llms-and-agents/s11-capstone-clinic 11 slidesS12 · Design-review presentations + wrap
Present and defend your system design the way you would to a staff panel. Return to your Week-1 goals, and leave with what to practise next. Thu Aug 20.
/llms-and-agents/s12-design-review artifact Project 5 · due Thu Aug 20P5 · Capstone: System Design Document + Review Presentation
One document for the whole system, every decision defended, plus the review you present live. The artifact you can take to work on Monday.
/llms-and-agents/p5-capstone-design-doc cheat sheet Reference · cheat sheetProduction LLM System Design Checklist
The seven-section design document, the whole system on one page, the reviewer's hard questions, the common gaps, and a pre-ship checklist. Bring it to any review.
/cheatsheets/production-checklist hands-on lab Lab 6 · build itCapstone: Integrate & Defend
Combine Labs 1-5 into one system behind a single entry point, write the seven-section design doc, run your eval harness on it, and prepare the five-minute walkthrough.
/llms-and-agents/lab-capstone-integration