Context window: how much a model sees at once
Definition
The context window is the amount of text a language model can consider at once for a request, measured in tokens. Everything together, the instruction, supplied documents and the prior history, must fit into this window. It is the model's short-term memory: what does not fit cannot be taken into account for this answer.
Why the Term Matters
Many expect an assistant to know everything at once. The context window puts a hard limit on that expectation. When it is full, older parts are pushed out or cut off, and the answer loses coherence. That is exactly why you do not supply all company knowledge, but via RAG only the fitting excerpts. Understanding the window, you plan prompts and knowledge so the important part always fits.
When It Is Relevant
- Long documents or histories should be taken into account.
- A conversation runs over many steps and should hold coherence.
- Much knowledge should flow in without overloading the answer.
- Answers feel incoherent because something important dropped out.
When Not
For short, clearly bounded requests the window size plays no role, since everything fits comfortably anyway. And a maximally large window is no end in itself: filling it indiscriminately costs more tokens and often yields worse, not better, answers.
Industries That Use It
- Law: long contracts where early and late clauses must be read together.
- Consulting: lengthy reports meant to be assessed as a whole.
- Software development: larger code sections understood in context.
- Customer service: long conversation histories whose thread should stay intact.
Typical Use Cases
- An assistant that pulls only the fitting passages into the window via RAG.
- A summary that processes a long text in stages.
- A conversation whose history is trimmed deliberately rather than carried in full.
- Choosing a model with a larger window for long documents.
- A build that puts the important part first so it surely fits.
Decision Guide
| Signal | Mind the context window | Alternative |
|---|---|---|
| Long documents | fitting excerpts via RAG | dumping all breaks the limit |
| Long conversation history | trim history deliberately | coherence gets lost |
| Short requests | barely relevant | just use it |
| Answer feels incoherent | check window and selection | choose a larger model |
Example
A team wants long contracts checked that do not fully fit the window. Instead of sending the whole document, it pulls the relevant clauses via RAG and adds a short summary of the rest. So the answer stays coherent without breaking the limit. This describes the method, not an asserted metric.
Service Connection
We plan how to handle the context window in every personal AI assistant and in the company brain, so the important part is reliably taken into account.
Sources
This framing describes common practice around the context window. The window size depends on the chosen model and must be checked alongside cost and task case by case.
