Hard20 minAI Engineering
UpdatedAug 6, 2026
Edit

Vector Search: Chunking a Manual

Question Variations

  • "Why can large chunks reduce retrieval precision?"
  • "How should tables and code blocks be chunked?"
  • "How would you chunk a technical manual containing headings, tables, and code?"

Why This Is Asked

This tests whether a candidate understands that chunking determines what evidence retrieval can return. Interviewers assess semantic boundaries, metadata preservation, and an empirical method for balancing context completeness against retrieval precision.

Key Concepts

  • Semantic boundaries: Preserve sections and parent context rather than splitting arbitrary token counts.
  • Chunk size: Balance retrieval precision against sufficient answer context.
  • Metadata: Retain document hierarchy, version, and permissions.
  • Evaluation: Test evidence recall and answer quality on representative queries.

Question Variations

  • “Why can large chunks reduce retrieval precision?”
  • “How should tables and code blocks be chunked?”
  • “How would you chunk a technical manual containing headings, tables, and code?”

Answers by Technology

+ Add Variant
System DesignImprove this answer ✏️

Expected Answer

I would preserve document structure first: split at headings and coherent paragraphs, keep tables and code blocks intact, and attach parent heading, source offsets, version, and permissions. Overlap is a small continuity aid, not a substitute for meaningful boundaries. Then test chunk sizes on real questions that require tables, code, and cross-section context; choose the configuration that improves evidence recall without flooding the prompt with unrelated text.

Why It Matters

Bad chunks either omit needed context or retrieve too much unrelated text.

Common Mistakes

  • Splitting code mid-block: The retrieved example becomes unusable.
  • Dropping heading metadata: Matches lose their meaning and source context.

Follow-up Questions

  • Why use overlap? (Answer: To preserve meaning that crosses a chunk edge.)
  • How is chunking tested? (Answer: Measure evidence recall on representative queries.)