Retrieval, grounding and provenance

When an answer is going to drive an action, such as rolling back a deploy or closing an incident, “the model said so” is not good enough. You need to know which source a claim came from and which passage supports it. That is provenance. Reliability depends on three things here: retrieve the right sources, ground the answer in them, and keep the link from each claim back to its source.

Retrieve less, not more

A 1M-token window makes it tempting to send everything. Don’t. Accuracy drops as context grows, and a flood of loosely related wiki pages gives the model more ways to mix up services or versions. A typical retrieval pipeline:

1. Index your runbooks, postmortems and ADRs in chunks that keep metadata: service, owner, last updated, URL.
2. Retrieve the top few chunks for the question, filtered by service or environment.
3. Send each chunk as its own document, with its metadata attached, and ask for an answer that uses only those documents.

The retrieval step is your code (search, embeddings, a vector store). Claude’s job is to answer from what you hand it and to show where each part of the answer came from.

Grounding with prompts

The docs list several prompt-level techniques that reduce hallucinations:

• Allow “I don’t know”. Tell Claude explicitly to say when the documents don’t contain the answer. This alone removes many invented answers.
• Quotes first. For long documents (the docs mention more than 20k tokens), ask Claude to extract word-for-word quotes first and then answer only from those quotes.
• Verify and retract. After drafting, have Claude find a supporting quote for each claim and remove any claim it can’t support.
• Restrict to provided sources. Say outright that general knowledge is not to be used.

These techniques reduce hallucinations. They don’t eliminate them, so validate anything high-stakes.

The Citations feature

Prompted quotes rely on Claude copying text correctly. The Citations feature does it in the API instead. You set citations: {"enabled": true} on each document, and the response comes back as text blocks with structured pointers into your documents.

# cited_answer.py - answer from two documents and print the passage behind each claim
import sys
import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

def doc(path: str, title: str) -> dict:
    return {
        "type": "document",
        "source": {"type": "text", "media_type": "text/plain",
                   "data": open(path, encoding="utf-8").read()},
        "title": title,
        "context": f'{{"path": "{path}"}}',  # metadata for the model, never cited
        "citations": {"enabled": True},
    }

resp = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": [
        doc("postmortem.md", "INC-2291 postmortem"),
        doc("runbook.md", "Payments runbook"),
        {"type": "text", "text": sys.argv[1] + "\nUse only these documents. "
         "If they don't answer it, say so."},
    ]}],
)
for block in resp.content:
    if block.type != "text":
        continue
    print(block.text.strip())
    for c in block.citations or []:
        print(f'   ↳ [{c.document_title}] "{c.cited_text.strip()}"')

How it works:

Document type	Chunked into	Citation points to
Plain text	Sentences	char_location: character range (0-indexed)
PDF	Sentences	page_location: page range (1-indexed)
Custom content	Your blocks, as given	content_block_location: block range

• Each citation carries cited_text, document_index and document_title. cited_text does not count toward output tokens.
• Citations must be enabled on all or none of the documents in a request.
• title and context are passed to the model but are never cited, so the context field is a good place for metadata such as a path or version.
• Use custom content when you want to control the units Claude can cite, such as one block per log line or bullet point.
• Source documents can be prompt-cached. The citation blocks in responses can’t.

Provenance downstream

Citations show what supports a claim inside one response. Your pipeline should also record:

• which document versions were retrieved, for example a commit SHA or last_updated,
• the model ID and request ID,
• whether every claim that leads to an action has a citation. If one doesn’t, send it to a human instead of executing it.

Recap

• Retrieve a small, relevant, metadata-rich set of chunks instead of sending everything.
• Ground answers with prompts: allow “I don’t know”, quote first, retract claims without support, and restrict the model to the provided sources.
• The Citations API returns cited_text with character, page or block locations. Enable it on all documents or none, and don’t combine it with structured outputs.
• Log document versions and request IDs so that any answer can be traced back to its sources.