Data & AI

Two-Stage Retrieval with Cross-Encoders

Data & AI Pipelines diagram · made in Skemio

Two-stage retrieval architecture diagram showing a bi-encoder querying a vector database for top-100 candidates, which are reranked by a cross-encoder down to top-5 for an LLM.
Two-stage retrieval architecture diagram showing a bi-encoder querying a vector database for top-100 candidates, which are reranked by a cross-encoder down to top-5 for an LLM.
Open it in SkemioMake your own

What it shows

  • Bi-encoders prune millions of docs to top-100 candidates in under 15ms.
  • Cross-encoders jointly score (query, doc) pairs to eliminate false hits.
  • Achieves 18-25% higher precision@5 compared to standalone vector search.

The takeaway

Use fast bi-encoders to cast a wide net, then spend compute on cross-encoders for the top 50.

More data & ai pipelines diagrams