Hard20 minAI Engineering
UpdatedAug 6, 2026
Edit

RAG: Hybrid Retrieval Trade-offs

Question Variations

  • "Why does vector-only retrieval fail for error codes?"
  • "Where would you place a cross-encoder reranker?"
  • "Design search for support queries containing both product IDs and natural-language descriptions."

Why This Is Asked

This tests whether a candidate can choose retrieval techniques based on query behavior instead of treating vector search as a universal replacement for keyword search. Interviewers look for a reasoned hybrid pipeline, ranking trade-offs, and latency-aware evaluation.

Key Concepts

  • Lexical search: Preserves exact identifiers and rare terms.
  • Vector search: Matches semantic intent beyond exact wording.
  • Fusion and reranking: Combine candidate sets before expensive ranking.
  • Latency: Measure each stage and use bounded candidate counts.

Question Variations

  • “Why does vector-only retrieval fail for error codes?”
  • “Where would you place a cross-encoder reranker?”
  • “Design search for support queries containing both product IDs and natural-language descriptions.”

Answers by Technology

+ Add Variant
System DesignImprove this answer ✏️

Expected Answer

I would use lexical and vector retrieval as complementary candidate generators. Exact identifiers, error codes, and quoted phrases favor lexical search; paraphrases favor embeddings. Run both in parallel, fuse small candidate sets, then rerank only the merged top results. Measure recall by query class and reserve a latency budget for each stage, with a fallback when reranking times out. Authorization filters apply before context assembly.

Why It Matters

Vector-only search often misses the precise token a support user needs.

Common Mistakes

  • Reranking thousands of chunks: It destroys latency and cost budgets.
  • Combining unnormalized scores blindly: One retriever can dominate for accidental numeric reasons.

Follow-up Questions

  • Why use fusion? (Answer: It gives both lexical and semantic candidates a chance.)
  • Where does reranking fit? (Answer: After cheap retrieval, before context assembly.)