Cohere Rerank

What it is

The precision pass over your candidates.

Cohere Rerank is a cross-encoder scoring API. Where retrieval compares a query and a chunk through their separate embeddings, a cross-encoder reads the query and each candidate together and scores true relevance, which is far more accurate but too expensive to run over a whole corpus. So you retrieve wide and cheap first, then rerank the shortlist. It runs on top of any vector database and any embedding model.

The one job

40 loose chunks into the 3-5 that answer.

Given a query and a pile of loosely-relevant candidates, return them reordered by how well each actually answers the query, so you can keep only the top few. It is the narrowing step that turns noisy recall into a clean context window.

Reach for it when

• Retrieval returns too much noise and your answers drift.
• You fetch wide for recall and now need precision on the shortlist.
• Your RAG answers cite the wrong chunk even though the right one was retrieved.
• You want a precision lift without changing your embeddings or store.

Skip it when

• Your corpus is tiny and top-k is already precise: reranking adds cost for no gain.
• Your latency or cost budget cannot afford a second call, a real Week 3 tension.
• You would rather self-host an open reranker like bge-reranker and avoid the API.

A minimal look

Query plus documents, ordered out.

// retrieve wide first, then rerank the shortlist for precision
const res = await fetch("https://api.cohere.com/v2/rerank", {
  method: "POST",
  headers: { Authorization: `Bearer ${apiKey}` },
  body: JSON.stringify({ model: "rerank-v3.5", query, documents, top_n: 5 }),
});
const { results } = await res.json();
// results are ordered by relevance: [{ index, relevance_score }, ...]
const top = results.map((r) => documents[r.index]);

The principle it teaches

Where it fits your labs

This is a Week 2 tool. In Lab 2 you retrieve a wide candidate set from pgvector, then rerank it down to the handful of chunks that actually answer the query before they reach the model.