RAG & context engineering cheat sheet.

Everything competes for finite space: instructions, retrieved context, history, and room for the answer. The goal is not maximum context, it is minimum sufficient context.

Recall follows a U-curve: the model reliably uses what is at the start and end and loses the middle. The Lost in the Middle study measured accuracy sagging from the high tens into the fifties when the answer sat mid-context. A fact is not "in context" just because it is in the tokens.

"Embed the query, grab top-k, stuff it in" is the naive version, and it is why so many RAG demos feel dumb. The real pipeline has stages.

flowchart LR
  Q["Query"] --> RW["Rewrite / expand
optional"]
  RW --> H(("Hybrid
retrieve"))
  H --> RR["Re-rank
keep best few"]
  RR --> AS["Assemble window
dedup, order, trim"]
  AS --> M{"Model"}
  M --> A["Grounded answer"]
  classDef s fill:#DCEBFE,stroke:#1F2937,color:#0E1726;
  classDef m fill:#EEE6FF,stroke:#1F2937,color:#0E1726;
  class RW,RR,AS s; class M m;
    

Dense vector search finds things that mean the same. Sparse keyword/BM25 finds the exact term, product codes, names, error strings, that embeddings blur. Run both, fuse, dedup.

flowchart TB
  Q["Query"] --> D["Dense / vector
semantic"]
  Q --> S["Sparse / BM25
exact keyword"]
  D --> F(("Fuse
+ dedup"))
  S --> F
  F --> R["Ranked candidates"]
  classDef a fill:#DCEBFE,stroke:#1F2937,color:#0E1726;
  classDef b fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
  class D a; class S b;
    

You retrieve chunks, so chunking is a design decision, not preprocessing.

First-pass retrieval trades precision for recall, grab ~40. Then a re-ranker scores each against the query and you keep the top few. Cheap step, huge quality gain, and the single most commonly skipped one.

flowchart LR
  Q["Query"] --> RET["Retrieve top 40
high recall"]
  RET --> RANK["Re-rank by relevance
cross-encoder / LLM"]
  RANK --> TOP["Keep top 5
high precision"]
  TOP --> W["Into the window"]
  classDef s fill:#DCEBFE,stroke:#1F2937,color:#0E1726;
  classDef ok fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
  class RANK s; class TOP,W ok;
    

Classic RAG retrieves once, up front, blind. Agentic retrieval gives the model retrieval as a tool: it decides whether to look something up, searches, reads, and searches again for multi-hop questions. More power, more cost, so cap the loop.

flowchart LR
  Q["Question"] --> M{"Model
need to look it up?"}
  M -- yes --> SR["Search + read"]
  SR --> M
  M -- "enough" --> A["Grounded answer"]
  classDef m fill:#EEE6FF,stroke:#1F2937,color:#0E1726;
  classDef ok fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
  class M m; class A ok;
    

In a long chat the history grows every turn, and with it cost, latency, and the U-curve middle. You cannot just append forever.