Hard20 minAI Engineering
UpdatedAug 6, 2026
Edit

RAG: Fresh Sources and Citations

Question Variations

  • "How do you prevent answers from citing superseded policy?"
  • "When should a RAG assistant say it does not know?"
  • "Design a policy assistant that cites only current, authorized documents."

Why This Is Asked

This assesses whether a candidate can make a RAG system trustworthy when source documents change. Interviewers evaluate provenance, freshness controls, access filtering, citations, and the ability to abstain rather than invent an answer.

Key Concepts

  • Metadata: Version, effective date, source URL, and access scope travel with chunks.
  • Freshness: Invalidate or re-index replaced documents and expose index lag.
  • Grounding: Require citations tied to retrieved chunks.
  • Abstention: Decline answers when retrieval confidence or evidence is insufficient.

Question Variations

  • “How do you prevent answers from citing superseded policy?”
  • “When should a RAG assistant say it does not know?”
  • “Design a policy assistant that cites only current, authorized documents.”

Answers by Technology

+ Add Variant
System DesignImprove this answer ✏️

Expected Answer

I would make provenance part of the retrieval contract, not a formatting feature. Every chunk carries a stable source ID, version, effective dates, canonical URL, and access policy. The ingestion pipeline detects replacements and revocations, marks old chunks ineligible, and reports index lag. At query time, authorization and current-version filters run before ranking, so unauthorized or superseded content never enters the model context.

The answer service should retain the chunk IDs used for each claim and render citations from those IDs. If retrieval cannot find current, authorized evidence—or sources disagree—the correct behavior is a bounded answer such as “I could not verify this from current policy,” with an escalation path. I would test document replacement, deletion, and permission changes explicitly.

Why It Matters

Freshness failures look persuasive to users because the language model can explain obsolete text fluently. Provenance and abstention make the system reviewable instead of merely plausible.

Common Mistakes

  • Filtering after generation: Unauthorized or stale context may already influence the answer.
  • Treating citations as decoration: They must resolve to the exact retrieved evidence.

Follow-up Questions

  • What causes stale answers? (Answer: Index lag, failed deletions, and missing validity filters.)
  • When should it abstain? (Answer: When no authorized, current evidence supports the claim.)