What a Context Window Is, and Why AI Forgets the Start of a Chat

You have probably noticed this. You start a chat with an AI assistant. The first hour is sharp. By message forty, it has forgotten the thing you told it at the start, or it is quietly contradicting itself, or it is answering a question you asked an hour ago as if you just asked it now.

That is not the model being lazy. It is a hard limit called the context window, and once you understand it, you will stop fighting it and start working with it.

The desk, not the filing cabinet

Picture the model sitting at a desk. The context window is the surface of that desk. Everything the model can see right now, your original question, the file you pasted, the last twenty replies, every clarification, every "actually, scratch that", is sitting on the desk at the same time.

The desk has a fixed size. It is big (modern models can hold the equivalent of a long book), but it is not infinite. When you keep adding to the conversation, you keep piling paper on the desk.

Here is the part people miss. The model does not have a filing cabinet behind it. There is no long-term memory of what you said last Tuesday unless something explicitly puts that back on the desk this time. Each new reply is the model looking at the desk, as it currently is, and deciding what to say next.

Why long chats drift

Two things happen when a chat gets very long.

First, if you blow past the window, the oldest stuff falls off the desk. The constraints you set at the start ("write in a plain style, no bullet lists, keep it under 300 words") are the first to go, because they are the oldest paper on the pile. The model is not ignoring you. It literally cannot see those instructions anymore.

Second, even before you hit the limit, long chats get noisy. Your false starts, the model's earlier wrong answers, half-abandoned tangents, they are all still on the desk, and they all pull on what the model says next. If the model got something wrong on message ten and you corrected it, both the wrong version and the correction are sitting there. Sometimes it keeps drifting back toward the wrong one, because wrong patterns on the desk are still patterns.

This is why a chat that felt brilliant for the first thirty minutes can feel foggy by the second hour. The desk is cluttered.

Why a fresh chat with a summary usually wins

The move that experienced users make, often without thinking about it, is to start over.

Not from zero. They open a new chat, and the first message is a tight summary of where things landed: "I am working on X. We decided Y. The constraints are A, B, C. Here is the current draft." Then they continue.

That is a clean desk with only the paper that matters. No false starts, no corrections to corrections, no abandoned tangents. The model's next reply is based on the good version of the state, not the entire messy history of how you got there.

If you find yourself five pages deep and the model keeps slipping, that is the signal. Summarize, open a new chat, paste the summary, keep going.

Why giving it the document beats hoping it remembers

Same principle, different situation. If you need the model to work with a specific document, a contract, a policy, a transcript, a spec, paste the relevant part into the current chat. Do not rely on it "remembering" something you discussed last week, and do not rely on vague references like "that proposal I mentioned."

If it is not on the desk right now, it does not exist, as far as the model is concerned.

This also means being selective. You do not need to paste a 200-page manual when the answer lives in two paragraphs. Give it the two paragraphs. A smaller, more relevant desk is almost always better than a huge one stuffed with everything you might possibly need.

The short version

The context window is the model's working memory for this conversation. Nothing older than that exists for it. Long chats drift because the desk gets cluttered or things fall off the edge. The fix is not a cleverer prompt. It is a cleaner desk: start fresh when a chat gets heavy, lead with a short summary of the state, and paste in the document instead of hoping the model recalls it.

If you want more plain-language explainers like this one each week, you can subscribe on the NeuroByte blog.