dictionary

Context Window

Published Updated

The context window acts as the short-term working memory of an LLM. It limits how much information you can put in your prompt and how long the generated output can be. Modern models range from handling a few pages of text (8k tokens) to processing entire books or codebases in a single prompt (1M+ tokens).

CategoryModels
Reading time3 min read

Definition

The maximum amount of text (measured in tokens) that an AI model can process at one time.

Need this applied?

We help teams go from definitions to deployed workflows—safely and fast.

Why it matters

AnthropicGoogle

If a prompt exceeds the context window, the model will typically reject the request or truncate the earliest parts of the conversation. Larger context windows are powerful for analyzing long documents, but they increase latency and compute costs dramatically.

Needle in a haystack

Anthropic

Just because a model has a 1-million token context limit doesn’t mean it utilizes all that information perfectly. Models often struggle to recall specific facts hidden in the middle of a massive context window (the "needle in a haystack" problem).

FAQ

Is a token the same as a word?

No. A token is a chunk of characters. In English, one token is roughly 4 characters or 0.75 words. "Apple" is one token, but "Unbelievable" might be broken into two or three ["Un", "believ", "able"].

Email this summary + checklist

Get a copy of “Context Window” and an AI readiness checklist in your inbox.