An AI chat doesn’t remember. It rereads.

Chat apps are designed to feel like a conversation with someone who's following along. Underneath, nothing is following along. Understanding what actually happens between your messages explains a lot of odd AI behavior — the assistant that forgets a rule you set twenty messages ago, the answer that slowly gets worse, the error that says your request is too long when your last message was one line.

The model is stateless

A language model is a function. Text goes in, a continuation comes out, and when it's done nothing about the exchange is kept inside the model. It doesn't learn from your chat, and it doesn't hold your conversation anywhere in its weights.

The feeling of memory is built by the app around it. When you send message number ten, the app doesn't send just message ten. It sends the instructions, messages one through nine, the model's own earlier replies, and then message ten — the full transcript, every turn. The model reads all of it from the top and writes the next reply. Then it forgets again, and next turn the app sends everything once more, now one exchange longer.

That's the whole trick. "Memory" in a chat is the conversation being replayed.

The window: one budget, four things competing for it

A model can only read so much at once. That limit is the context window, and it's measured in tokens — chunks of text, roughly three quarters of an English word each on average. Everything the model handles in a turn has to fit inside it at the same time:

1. The instructions. The hidden setup text an app gives the model: what role to play, what format to use. In Vault AI, each tool has its own instructions, and if you've filled in the on-device profile, that's included here too.
2. The history. Every earlier message and reply in the chat.
3. Your new message. Including anything you pasted into it.
4. The answer. The reply has to be written inside the same window, so room has to be left for it.

Only one of those four grows on its own. Instructions are fixed. Your next message is whatever you type. But the history gets longer with every turn, and it gets longer twice as fast as you might think, because the model's own replies count too — and they're usually longer than your questions.

Why long chats drift before they break

Running out of room is the obvious failure. The subtler one comes first. Models don't pay equal attention to everything in the window; material in a long context, especially in the middle of it, tends to get less weight than what's near the start or the end. A rule you set early in a long chat — "keep it under 100 words", "don't use bullet points" — is still technically in there, competing with everything said since. It loses more often as the chat grows.

Then there's the hard limit. When the transcript plus the new message no longer fits, something has to give. Some apps quietly drop the oldest turns, or replace them with an automatic summary, so the chat keeps going while the model's grip on the beginning disappears without telling you. Others refuse and say the request is too long. Either way, the conversation has outgrown what the model can see at once.

Why a phone hits the wall sooner

Large cloud models run with very large windows — enough to hold long documents and long conversations together. A model that runs on your iPhone is a different size class. Apple documents the context window of its on-device model at around 4,000 tokens: a few thousand words, shared between the instructions, the history, your message and the answer.

The reason is the same physics covered in what running AI on your phone actually costs. A bigger window means more memory and more reading time on every single turn, and a phone has neither to spare. It's the same trade as the model's size, which is why a small model forgets facts but keeps its writing skill.

Here's how that shows up in Vault AI, honestly. Starting a new chat gives the model an empty window. If a chat or a single paste gets too long for it, Vault AI tells you so rather than guessing, and asks for something shorter. It doesn't secretly summarize your history to keep going. And reopening a saved chat shows you the old messages, but the model starts that session fresh, so paste back in whatever it needs to know.

The habit that fixes it: one chat per task

Cloud chatbots taught people to keep one long thread going for everything. That habit works badly everywhere and worst on a phone. The fix is simple:

1. Start a fresh chat for each task. Drafting an email and planning a trip don't need to share a window. Unrelated history only crowds out what matters.
2. Use the purpose-built tool. Email Draft, Summarize, Rewrite and the rest already carry their instructions, so you don't spend your window explaining the job.
3. Put the rules in the message, not ten turns back. If length or tone matters, say it in the request where the model reads it last.
4. Carry a summary, not the transcript. When a task runs long, ask for a short recap of the decisions so far, start a new chat, and paste the recap in. You keep the substance and free the room.
5. Feed long documents in sections. The method is in give it the text, not the question and the offline summarizer guide.

Where the cloud genuinely wins

None of this is a workaround that makes a phone model equal to a cloud one on long work. If the job is a long back-and-forth over a big codebase, a research thread that builds over an afternoon, or a contract you need held in full while you ask questions about it, a large cloud model's window is the right tool. The honest capability map says the same.

But most everyday AI use isn't that. It's one email, one message, one page of notes — short jobs that fit comfortably in a small window and don't need yesterday's conversation. Those run on your phone, without a connection, and the transcript being replayed every turn never leaves the device.