# Source Attribution Tracking

## The Problem

Cannot verify which sources the AI actually used for its response, making it impossible to validate accuracy or trace misinformation.

### Symptoms

* ❌ AI cites sources but unclear which claims from which source
* ❌ Cannot verify if citation accurate
* ❌ Multiple sources cited but unclear contribution
* ❌ Hallucination detection impossible
* ❌ No source-to-claim mapping

### Real-World Example

```
AI response:
"The API rate limit is 1000 requests per hour. Premium users get priority support.
Billing is monthly. [Sources: doc_123, doc_456, doc_789]"

Questions:
→ Which source says "1000 requests/hour"?
→ Which source mentions "priority support"?
→ What does doc_789 contribute?

No granular attribution:
→ Cannot verify each claim
→ Cannot detect if "priority support" hallucinated
→ Bulk citation unhelpful
```

***

## Deep Technical Analysis

### Granular Citation Tracking

**Claim-Level Attribution:**

```json
{
  "response": "The API rate limit is 1000 requests per hour...",
  "claims": [
    {
      "text": "rate limit is 1000 requests per hour",
      "source": {
        "chunk_id": "doc_123_chunk_5",
        "document": "API Documentation v2.0",
        "snippet": "...enforces 1000 req/hour limit...",
        "confidence": 0.95
      }
    },
    {
      "text": "Premium users get priority support",
      "source": {
        "chunk_id": "doc_456_chunk_12",
        "document": "Premium Features Guide",
        "snippet": "...priority support for Premium tier...",
        "confidence": 0.88
      }
    }
  ]
}
```

**Unsupported Claims Detection:**

```
Claim: "Billing is monthly"

Check retrieved chunks:
→ No chunk mentions "monthly billing"
→ Flag: Potential hallucination

Log:
{
  "claim": "Billing is monthly",
  "source": null,
  "flag": "unsupported_claim",
  "confidence": 0.0
}

Alert: Manual review needed
```

### Source Usage Analytics

**Source Utilization:**

```
Retrieved 10 chunks:
→ Chunk 1: Used (cited in response)
→ Chunk 2: Used
→ Chunk 3: Not used
→ Chunk 4: Used
→ ...

Utilization: 6/10 = 60%

Analysis:
→ 4 chunks retrieved but unused (why?)
→ Wasted retrieval
→ Could reduce K
```

**Chunk Contribution:**

```
Track how much each chunk contributed:
{
  "chunk_id": "doc_123_chunk_5",
  "contribution": {
    "claims": 3,
    "tokens_used": 120,
    "importance": "high"
  }
}

Identify most valuable chunks:
→ Boost in ranking
→ Prioritize similar content
```

### Citation Formats

**Inline Citations:**

```
Format response with inline sources:
"The API rate limit is 1000 requests per hour [1]. Premium users
get priority support [2]."

References:
[1] API Documentation v2.0, Section 3.2
[2] Premium Features Guide, page 5

Clear claim-to-source mapping
```

**Hover Citations (UI):**

```
User hovers over "1000 requests per hour":
→ Popup shows: Source chunk with highlighted text

Interactive verification:
→ User can check source
→ Builds trust
```

### Hallucination Detection

**Source Grounding Check:**

```
For each claim in response:
1. Extract claim
2. Search retrieved chunks for support
3. If found: Calculate semantic similarity
4. If similarity < 0.70: Flag as potential hallucination

Example:
Claim: "Premium plan costs $500/month"
Best matching chunk: "Enterprise plan costs $500/month"
Similarity: 0.75 (high)
Issue: "Premium" vs "Enterprise" (different plans!)
Flag: Substitution error
```

**Citation Validation:**

```
AI claims: "According to doc_123: Rate limit is 1000/hour"

Verify:
→ Retrieve doc_123
→ Check if it actually says "1000/hour"
→ Semantic similarity: 0.92 (confirmed)

If mismatch:
→ Flag: Incorrect citation
→ Possible hallucination
```

### Transparency Metrics

**Source Confidence:**

```
For each cited source:
→ Retrieval score: 0.85
→ Reranking score: 0.91
→ LLM confidence in using it: 0.88

Aggregate confidence: 0.88

Display to user:
"High confidence" (>0.80)
"Medium confidence" (0.60-0.80)
"Low confidence" (<0.60)
```

**Citation Coverage:**

```
Response has 5 claims:
→ 4 claims have citations
→ 1 claim uncited

Citation coverage: 80%

Target: >95% citation coverage
→ Minimize unsupported claims
```

***

## How to Solve

**Implement claim-level source attribution (map each claim to specific chunk) + track chunk utilization (which chunks actually used in response) + detect unsupported claims (no source found) + validate citations (check if source actually says what AI claims) + display inline citations in responses + monitor citation coverage percentage + flag low-confidence attributions + build source verification UI (hover to see chunk) + test attribution accuracy with eval set.** See [Source Attribution](/rag-scenarios-and-solutions/monitoring/source-tracking.md).


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://help.twig.so/rag-scenarios-and-solutions/monitoring/source-tracking.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.