A context window is how much text a model can look at in one request, and it fills faster than people expect: everything you send counts, and so does the reply.
It is measured in tokens, chunks of text that are usually a word or a piece of one. Many current Claude models have a 1M token window, and older ones such as Sonnet 4.5 top out at 200k.
What counts against it: 1) The system prompt (the standing instructions the app gives the model) 2) The whole conversation so far 3) Tool definitions, the description of every tool the model may call 4) Tool results, whatever those tools sent back 5) The reply the model is writing
If your input alone is over the limit, the Claude API rejects it with a "prompt is too long" error. If the reply hits the limit halfway, it is cut off, and the response says why with the stop reason model_context_window_exceeded.
Why have a limit at all? Partly cost. Attention, the step where every token is compared with every other token, grows with the square of the length, so doubling the text roughly quadruples that part of the work.
A bigger window is also not a better memory. Recall (how well the model finds and uses what it read earlier) gets worse as tokens pile up, even well under the limit.
Agents feel this most. An agent (a model calling tools in a loop until a task is done) appends every tool call and result to the window on each step. So trimming is part of building one: summarize old turns, clear old tool results, keep notes in files.
Quick check before you scroll: Your agent has a 1M-token window and has used only 30% of it, but its answers about early steps are getting sloppy. What is the likely cause, and what can you do?
The answer
Context rot: accuracy and recall degrade as tokens accumulate, even below the hard limit. Shrink what the model has to carry, for example by clearing old tool results or summarizing earlier turns (compaction).
New here? I post a bite-size AI / ML concept like this every day. Follow me for the daily drop, and it compounds fast. Why I do it: https://lnkd.in/gK8knHDH