Lesson 2: Context Window and Non-Determinism

What this level covers

Level 1 established that Claude reads text as tokens. This level answers two follow-up questions:

1. Where do those tokens sit while Claude writes a reply?
2. Why can the same prompt give a different reply on a second run?

Both answers change how you write prompts and how you judge them.

Understand the context window

The context window is everything Claude can refer to while it produces one reply, plus the reply itself. It is measured in tokens.

Think of it as a desk of fixed size. Claude can only work with the papers on the desk. Its training is the knowledge in its head and takes up no desk space. Anything not on the desk and not learned in training does not exist for that reply.

Two consequences follow:

• It is finite. Every model has a maximum number of tokens it can hold at once.
• Bigger is not automatically better. As the window fills, accuracy and recall tend to fall. A full desk is harder to search. Later levels cover how to manage this.

List what occupies the window

In a chat, the window holds:

• Instructions the app supplies behind the scenes, plus any you have set up
• Every message you have sent in this conversation
• Every reply Claude has given
• Files, images and documents you attached
• Results from tools Claude used, such as web search results
• The reply Claude is writing now, including any thinking it does first

In an API request, the same categories appear as named parts:

• The system prompt
• Every entry in the messages list, including images, documents and tool results
• Tool definitions
• The output for this turn, including thinking

The point most people miss: a conversation is cumulative. Each new turn is processed together with the whole history before it. A short question on turn 40 still carries turns 1 to 39 with it.

Do this now: open a long chat you have had and estimate what is on the desk. Count the attachments, the pasted text and the long replies. That total is competing for space on every turn, not only your latest message.

Know what happens at the limit

Behaviour differs by surface:

• API: if the input alone is larger than the model's window, the request is rejected with a 400 "prompt is too long" error. Nothing is silently trimmed.
• Chat: the app can manage the window for you. On paid plans with code execution enabled, Anthropic's help centre says Claude summarises earlier messages when a conversation nears the limit. A summary is shorter than the original, so fine detail from early turns may no longer be in view word for word.

Action: if an early detail matters late in a long chat, restate it in your current message instead of assuming it is still in view word for word.

Look up the current size instead of remembering it

Window sizes change between releases. Any number you memorised may be stale, so treat the figures below as a dated snapshot.

Snapshot, 9 October 2026: Anthropic's model comparison table lists a 1M-token window for Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5, with some older models at 200K. The same page puts 1M tokens at roughly 555,000 words on the current tokenizer. The help centre lists 1M tokens for those four models in chat on paid plans. This check found no published figure for the free plan, so treat that as unconfirmed.

Use this procedure every time the number matters:

1. Identify the exact model you are using (the model picker in chat, or the model ID in your API request).
2. Open the Models overview page in Anthropic's Claude Platform documentation.
3. Find the comparison table and read the Context window row for your model.
4. If you use the chat apps, also open the help centre article on context window size, because chat limits can depend on plan and model.
5. Record the figure with the date you checked it.

API users have a further option: the Models API returns a max_input_tokens value for each model, so code can read the limit instead of hard-coding it.

Understand non-determinism

Claude builds a reply one token at a time. At each step it has a set of likely next tokens and one is sampled from them. Sampling involves chance, so two runs of an identical prompt can diverge at an early token and end up with different wording, structure or, occasionally, conclusions.

This is normal behaviour, not a fault. A weather forecast is a useful comparison: the same conditions give a range of outcomes, and you learn the range only by observing more than once.

A note for API users: older guidance suggested setting temperature to 0 to get repeatable output. Anthropic's documentation states that this never guaranteed identical results. On models released after Claude Opus 4.6, the API rejects any non-default temperature value with a 400 error. Plan for variation instead of trying to switch it off.

Judge a prompt on several runs

One output is one sample. It cannot tell you whether a prompt is reliable.

1. Run the prompt at least three to five times. In chat, start a fresh conversation each time or use the retry control so earlier turns do not leak into the window.
2. Keep everything else the same: model, attachments, wording.
3. Compare the outputs. Note what stays constant and what varies.
4. Decide whether the variation matters. Different phrasing is usually fine. Different facts, formats or decisions are not.
5. Only then conclude that the prompt works, or that it needs changing.

Apply the same rule in reverse: one bad output does not prove a prompt is broken, and one good output after an edit does not prove the edit helped.

Key takeaways

• The context window is everything Claude can see for one reply, plus the reply. It is finite and counted in tokens.
• In chat it holds instructions, the full conversation, attachments, tool results and the current output. In the API it holds the system prompt, messages, tool definitions and the output.
• Conversations accumulate: every earlier turn takes space on every later turn.
• Look up the window size for your exact model on Anthropic's pages and note the date. Do not rely on memory.
• Identical prompts can produce different outputs because each token is sampled.
• Judge a prompt on several runs, never on one.