Context Window

The context window is the limit on how many tokens a language model can hold in view in a single request. It includes the system instructions, conversation history, any documents you attach, and the reply being generated.

Why it matters

Anything outside the window is invisible to the model. Long chats or large documents can exceed it, forcing truncation or summarization. Larger windows allow more material but usually cost more and can slow responses, and models may use very long inputs less reliably.

Example

Suppose a model has a 32,000-token window. If your prompt and history use 30,000 tokens, only about 2,000 remain for the answer. Applications often handle this by dropping old messages or retrieving only relevant passages with RAG.

Common misconceptions

• A bigger window means the model remembers you permanently. It only sees what is sent in the current request.
• The window counts only your input. Output tokens count too.
• Everything inside the window is weighed equally well; details buried in very long input can be missed.

Related

• Token
• Retrieval-augmented generation
• Large language model
• Prompt

This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.