Context Compaction for Long-Running Agents: What to Cut, What to Keep
Compaction is a lossy write, not a summary. A working method for deciding what survives when a long-running agent hits its context limit, and how to make the loss auditable.
Short answer: compaction is not summarization. Summarization is a task you run on a document to make a smaller document. Compaction is a state migration you run on a live agent so it can keep working after its context window fills, and it deletes information the agent may need later in ways it cannot detect. Treating it as a writing task is why compacted agents go subtly wrong an hour after the compaction instead of failing loudly at it.
Every agent that runs for hours eventually hits the wall. The session log grows, tool results pile up, and the window that was comfortable at step 20 is full at step 400. You have three options: start a new session and lose continuity, retrieve selectively and rebuild state each turn, or compact. Compaction is the cheapest and the one that quietly degrades output, so it is the one worth designing properly.
Compaction is a write, and every write is a deletion
The mental model to install: when you replace 40,000 tokens of history with 2,000 tokens of summary, you are not compressing. You are performing a write operation against the only state the agent has, and paying for it by deleting 38,000 tokens of evidence. The summary is a new artifact produced by a model that did not know what the agent would need next.
That reframing changes the engineering. A write gets a schema, a validation step, and a diff. A deletion gets an audit trail, because you need to know what was removed to explain a later failure. Most compaction implementations have none of these: a prompt that says "summarize the conversation so far, keep important details," and a hope. The failure signature is distinctive when you learn to spot it. The agent does not crash. It confidently re-does work it already did, re-asks a question it already answered, or violates a constraint that was stated in the history but not carried into the summary. You see it as a quality problem, and you go looking at the model.
For the general case of what an agent should remember versus what it should look up, the memory architecture piece covers the split. Compaction is the special case where there is no external store and the window is the whole world.
Sort the state before you compress it
The single highest-leverage move is to stop treating session history as one undifferentiated blob. Sort what is in the window into four classes, and give each class a different compaction policy.
| Class | Examples | Policy |
|---|---|---|
| Decisions and constraints | "Use the staging database." "Never touch /etc." "The client approved option B." |
Never summarize. Copy verbatim into a pinned block. Re-inject every turn. |
| Verified facts with provenance | Command output, exit codes, file paths, schema shapes, error strings | Compress to a key plus a pointer. Keep the exact string or path; drop the surrounding prose. |
| Reasoning traces | Why the agent tried approach A before approach B | Compress hard, to one line per abandoned branch. Keep the "do not retry this" signal. |
| Raw tool output | Full logs, full HTML, full JSON payloads | Drop entirely, keep the pointer and a one-line result. This is the bulk of the window and the least valuable per token. |
The asymmetry is the point. Constraints occupy a few hundred tokens and are catastrophic to lose. Raw logs occupy tens of thousands and are cheap to re-fetch. A generic "summarize and keep important things" prompt inverts that priority about half the time, because a summarizer judges importance by how much space a thing takes and how recently it appeared. Long tool outputs look important. A one-sentence constraint from turn 3 does not.
Compact at a boundary, not at the ceiling
When you compact matters as much as what you compact. Two rules earn their place in every implementation.
Compact on task boundaries. The natural moments are after a completed subtask, before a handoff, and after a verification step passes. Compacting mid-task means the agent resumes with a summary of work it was in the middle of doing, and the summary cannot represent "I was about to run the migration, do not run it twice." A boundary gives you a clean thing to summarize: finished state, open items, constraints.
Compact before you are forced to. If compaction triggers at 95% of the window, the compaction prompt competes with a full working set and the agent is already under pressure. Triggering at 60-70% leaves room for the compaction call itself, for the re-injected pinned block, and for the next few turns. The cost is more compaction events; the benefit is that each one is a deliberate write rather than an emergency.
A third rule that prevents a specific bug: never compact inside a retry loop. If the agent is on attempt three of a failing command, its history contains the two failed attempts and the exact error strings that explain them. Compacting there produces an agent that will make the same failure a fourth time with no memory of why. Defer the compaction until the loop resolves or the task is abandoned.
Make the loss auditable
You cannot fix what you cannot inspect, and a compaction is invisible by default. Four artifacts make it visible, and each one costs a few lines of code.
1. Write the pre-compaction state to disk, always. Full transcript, timestamped, before you replace it. Storage is cheaper than re-running a task. When an agent does something inexplicable at step 500, the first question is what it knew at step 300, and the only way to answer is the file.
2. Record token counts on both sides. Input tokens before, output tokens after, and the ratio. A compaction that goes from 40,000 to 30,000 tokens is not compaction, it is a rewrite, and you want to know that it happened rather than discovering it when the agent hits the wall two turns later.
3. Diff the pinned block. The constraints block should be stable and should only change when a human or an explicit decision changes it. Any diff in the pinned block between compactions is a signal that a summarizer edited your invariants. That is the highest-severity compaction bug there is, and it is trivially detectable.
4. Log the pointers you kept and test that they resolve. If the summary says "see /tmp/build.log for the failure," verify that file still exists and still contains the failure. A pointer to a rotated log is worse than no pointer, because it reads as evidence and returns nothing.
On the observability side more broadly, the telemetry piece covers what else belongs in the trace. Compaction events belong in it as first-class spans, not as a debug line.
The failure modes, ranked by damage
These are the four that account for most post-compaction regressions. They are ordered by how much they cost, not how often they occur.
Constraint amnesia. The agent forgets a boundary it was told once and violates it. Highest damage because the violation is often irreversible (a write, a delete, a send). This is the argument for a pinned block that is copied, never summarized.
Phantom completion. The summary asserts that a step succeeded when it was attempted and failed, or attempted and never verified. The agent then builds on a foundation that does not exist. Prevention: the pinned block records verified state only, with the verification command and its exit code, and "attempted" is never written as "done."
Retry loops. A dropped "do not retry this" signal sends the agent back into a branch that already failed three times. Cheap to prevent: keep one line per abandoned branch in the summary, permanently.
Voice drift. Style, naming, and formatting conventions decay because they lived in examples that were compressed away. Lower damage, high visibility, and the reason output looks subtly different after a long run. Keep two or three canonical examples verbatim in the pinned block instead of a rule that describes them.
When not to compact at all
Compaction is the right answer less often than its popularity suggests. Three alternatives are strictly better when they fit.
Externalize the state instead. If the agent's state can live in a file, a database, or a task list, it should. Then the window only needs to hold the current step plus a pointer to the state. This is the same argument as server-side memory over retrieval, applied to a single run: the durable artifact should be outside the window, and the window is a cache.
Start a clean session with a briefing. For tasks with real boundaries (a new ticket, a new day, a new build), a fresh session primed with a written brief beats a compacted one, because the brief is written by something that knows the goal rather than by something that knows the history. It is also auditable by default, since the brief is a file.
Split the task. A run long enough to need three compactions is usually two or three tasks wearing one session. Splitting gives each part a smaller working set and gives you a place to verify. This is the same instinct as constraining the problem rather than the model.
The honest framing is that compaction is a mitigation, not an architecture. It lets a session survive past its window. It does not make the session's state durable, and it does not make the loss recoverable. Build it as a deliberate, logged, schema-bound write, and treat any compaction event as a small incident you can review, because that is what it is.