Context windows and compaction

The context window is the model’s working memory for one request. It is not the training data. It is everything you send plus everything Claude writes back. A big window is useful, but more context is not automatically better: as the token count grows, recall and accuracy drop. The docs call this context rot. So the skill being tested is choosing what goes in, not only fitting it in.

What counts

Every part of a request takes up space in the same window:

• Tool definitions, the system prompt, and every message in messages, including tool results, images and documents.
• The output for this turn, including thinking tokens. Thinking is part of max_tokens and is billed as output.
• Earlier thinking blocks, on current models. Recent Opus, Sonnet and Haiku models keep earlier thinking in the history by default, where it counts as input. Older models strip it automatically.

The API never drops earlier turns for you. Each turn, the whole history is sent again. An on-call bot that pastes 200 lines of kubectl logs into every tool result can fill its window long before the incident is over.

Current Opus, Sonnet and Haiku models have a 1M-token window, and one request can produce up to 128k output tokens. Some older models have 200k. Check the model comparison table rather than assuming.

What happens at the limit

Situation	Result
The input alone is larger than the window	400 invalid_request_error (“prompt is too long”)
Input plus max_tokens exceeds the window (Claude 4.5+)	Accepted. If generation hits the limit: stop_reason: "model_context_window_exceeded"

Your code has to handle both: catch the 400 before you retry blindly, and treat that stop reason as a truncated answer, not a finished one.

Measure before you send

The token counting endpoint takes the same shape as a Messages request (system, tools, images, PDFs, thinking) and returns input_tokens:

# count_timeline.py - estimate tokens for an incident timeline before sending it
import sys
import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment
MODEL = "claude-sonnet-5-5"

timeline = open(sys.argv[1], encoding="utf-8").read()
count = client.messages.count_tokens(
    model=MODEL,
    system="You are an SRE assistant. Summarise incidents for a postmortem.",
    messages=[{"role": "user", "content": f"<timeline>\n{timeline}\n</timeline>"}],
)
print(f"{MODEL}: {count.input_tokens} input tokens")

Things to know about it:

• It is an estimate. The real number can differ slightly.
• It is free but rate-limited, and its limits are separate from message creation.
• It does not use prompt caching, even if you include cache_control.
• Counts depend on the tokenizer. Claude Opus 4.7 and later models use a newer tokenizer that produces roughly 30% more tokens for the same text. Count against the model you will actually call.

Keep long sessions small

Compaction (API, beta). The server summarises older turns so a conversation can continue past the limit. You can ask for it when you choose (on demand) or let the API do it when input reaches a token threshold you set. It is in beta for Claude 4.6 and later models.

Context editing (API, beta). Rule-based clearing instead of summarising. It is enabled with the context-management-2025-06-27 beta header:

response = client.beta.messages.create(
    model=MODEL,
    max_tokens=2048,
    betas=["context-management-2025-06-27"],
    tools=tools,
    messages=messages,
    context_management={
        "edits": [{
            "type": "clear_tool_uses_20250919",
            "trigger": {"type": "input_tokens", "value": 60000},
            "keep": {"type": "tool_uses", "value": 3},
            "exclude_tools": ["get_incident"],
        }]
    },
)
print(response.context_management)  # applied_edits: what was cleared

clear_tool_uses_20250919 replaces the oldest tool results with a placeholder and keeps the most recent ones. clear_thinking_20251015 manages old thinking blocks. If you use both, the thinking strategy has to come first in edits.

Other ways to keep context small, all in the docs: let a hook grep a 10,000-line log down to its ERROR lines, send noisy work such as test runs to a subagent so only a summary comes back, and move rarely used CLAUDE.md sections into skills that load on demand.

Recap

• Tools, system, messages, tool results and output (including thinking) all share one window, and more context can make answers worse.
• Input over the limit returns a 400. Generation that runs out of room stops with model_context_window_exceeded.
• count_tokens gives a free estimate. Count against the model you will call, because tokenizers differ.
• Use compaction to summarise and context editing to clear old tool results or thinking. In Claude Code use /context, /compact <focus> and /clear.