What is context?

When a conversation with an AI gets too long for the model, one of a few things happens depending on the tool: the early part quietly stops counting, or it tells you to start a new chat, or a model writes a summary and the work carries on from it (that last one is called compaction). My last post skipped it (someone pointed it out), and it's the part of the harness I've spent more time on than almost anything else.

Every model can only look at so much at once. That's the context window...your instructions, the files, the whole conversation so far, all of it has to fit. Windows are big now, but models get worse at using them long before they're full. In a 2026 benchmark, the model that started best went from finishing 96% of the same agent tasks with 8K tokens to read to 34% with 128K.

With compaction, everything it "remembers" about the earlier conversation is a summary a model wrote...and summaries lose things, and sometimes change them. One 2026 study found compaction kept just 17% of the instructions users had set for the rest of a session. A compacted summary quietly flipped the meaning of one of my own rules, and my agent was about to "fix" the rule to match its summary. Another rule, re-read a page before changing it, stopped it. And the other day, after a compaction, it wrote up a handoff from its memory of our process instead of reading it, and skipped steps we'd agreed on. I caught that one by asking.

In the tool I use, Claude Code, the docs list what survives a compaction: what gets reloaded from files. What was only said in the conversation gets summarized. In my harness, the short rules, the ones that each cost me a mistake, live in files the tool reloads by itself after every compaction, so a summary can't eat them. After a compaction it's supposed to go back and re-read the long rulebooks, and it gets shown a small state file built from the logs, not its own notes.

And I count them...since June my harness has logged 84 compactions, and 18 times the agent just kept going and never went back to re-read. I tried a plain rule first, then a warning at the top of every turn, and one agent read that warning about forty times in a row and edited the files that hold my rules and records anyway. Now it's blocked: it can't edit those files until it has re-read them, and that block has stopped 14 writes so far.

What anyone can do, with any AI tool:
- Start a fresh conversation for a new task. One study found models did 39% worse on average when a task came in pieces over a conversation, and once they took a wrong turn they didn't recover.
- Put the instructions that matter wherever your tool keeps standing instructions, not only at the top of a long chat.
- In a long session, ask it to re-read the source before it acts on something from earlier.
- In any long chat, assume the early part is summarized or gone, whether or not the tool tells you.