Context window: how much AI can "remember" in a conversation

The context window is the maximum amount of text (in tokens) a language model can take into account at once: your messages, its own replies, uploaded files and system instructions.

The context window is the visible part of a long text

In plain words

Imagine reading a long book through a window cut in a sheet of cardboard. Only one page is visible - everything earlier is beyond the edge. A model's context window is the same kind of cutout. A small cutout and the model "forgets" the start of the conversation. A big one and you can fit a whole book under it.

How it works

Everything the model must consider is put into one long strip of tokens.
If the strip is longer than the window, the oldest parts are cut or compressed.
The model answers using only what fits in the window.
Modern models have windows from tens of thousands to a million tokens or more.
The fuller the window, the longer and more expensive the request is to process.

A small and a large context window

Where you'll see it

Context window size is listed in model and plan descriptions: "128K", "200K", "1M tokens". You notice it when you upload a big PDF or a whole project's code to a chatbot, or when a long conversation suddenly "forgets" what was agreed at the start.

Does it matter when buying

Yes, if you work with large documents - notes, books, code. Then choose a model with a big window. My tip for long chats: every so often ask the bot to briefly summarise what's been agreed and start a new chat from that summary.

Myths

Myth: A big window means the bot remembers everything perfectly. Reality: Even within the window, the model can miss a detail from the middle of a long text.
Myth: The context window is the bot's long-term memory. Reality: No, it's just the current conversation. Long-term "memory" is a separate feature of some services.

FAQ

How much text is 128 thousand tokens?

Roughly a short book - hundreds of pages, depending on the language.

Why did the bot forget what I said at the start?

Most likely the conversation grew longer than the window and old messages were cut.

Does a local model on a phone have a big window?

Usually smaller than cloud ones: it lacks device memory.