Document Version Conflicts

The Problem

Multiple versions of the same document coexist in the knowledge base, causing AI to cite outdated or conflicting information.

Symptoms

• ❌ Old and new versions both retrieved

• ❌ Conflicting information in responses

• ❌ "v1" and "v2" docs both present

• ❌ Cannot determine which is current

• ❌ Stale info mixed with current

Real-World Example

Knowledge base contains:
→ "API_Guide_v1.0.pdf" (2022): Rate limit 100/hour
→ "API_Guide_v2.0.pdf" (2024): Rate limit 1000/hour
→ "API_Guide_v2.1.pdf" (2024): Rate limit 1500/hour

Query: "What's the API rate limit?"

Retrieved chunks from all three versions:
→ AI response: "Rate limit ranges from 100 to 1500 per hour
depending on version."

Confusing - which is current?
User wants latest only

---

Deep Technical Analysis

Version Tracking Challenges

No Version Metadata:

Common problem:
→ Documents ingested without version tracking
→ Metadata: {document_id: "api_guide"}
→ No version field

New version ingested:
→ Same document_id
→ Both coexist
→ Cannot distinguish

Version Detection:

Filename-based:
→ "guide_v1.pdf", "guide_v2.pdf"
→ Parse version from filename

Metadata-based:
→ Document properties: Version 2.1
→ Last modified: 2024-03-15

Content-based:
→ "Version 2.0" in document text
→ Less reliable

Versioning Strategies

Explicit Version Metadata:

Store with each chunk:
{
  document_id: "api_guide",
  version: "2.1",
  published_date: "2024-03-15",
  is_latest: true
}

Retrieval filter:
WHERE document_id = "api_guide" AND is_latest = true
→ Only get current version

Version Lifecycle:

New version ingested:
1. Set all existing chunks: is_latest = false
2. Add new chunks: is_latest = true
3. Optionally: Delete old versions (if no archival need)

Automatic currency

Archival vs Deletion

Keep Old Versions:

Reasons to archive:
→ Compliance (retain historical docs)
→ Support legacy product versions
→ Audit trail

Strategy:
→ Keep but mark as archived
→ Filter out by default
→ Available on explicit request

Example filter:
WHERE is_latest = true OR (version = "1.0" AND user_needs_legacy)

Delete Old Versions:

Simpler approach:
→ New version → delete old chunks entirely
→ Only current version exists

Pros:
+ No confusion
+ Cleaner
+ Lower storage

Cons:
- No historical reference
- Cannot support legacy

Conflict Resolution

LLM Arbitration:

If multiple versions retrieved:
→ Prompt: "Prefer latest version"
→ AI should cite v2.1 over v1.0

But: Requires LLM to detect versions
→ Not 100% reliable

Better: Filter at retrieval

Recency Boosting:

Boost recent documents:
→ score = similarity_score * recency_boost
→ recency_boost = 1.0 + (days_since_publish / 365)

Recent docs rank higher:
→ v2.1 (2024) beats v1.0 (2022) in ranking

---

How to Solve

Track version explicitly in metadata (version number + is_latest flag) + implement version lifecycle (mark old as non-latest on new upload) + filter retrieval to is_latest=true by default + optionally delete old versions if no archival need + parse version from filename or document properties + boost recent versions in ranking + test that old versions don't appear in responses. See Version Control.