Study finds context compactors retain only 17% of standing agent rules

• CompInt measured only 17% retention of injected session constraints across the tested compactors, a result summarized by rohanpaul_ai.
• The failure is selective: a compactor can preserve the task and next steps while dropping a rule such as “ask before sending,” according to rohanpaul_ai's paper summary.
• A separate constraint registry pushed retention above 90% across three long-context settings, while leaving the compactor and main model unchanged, the study summary reports.
• Coding-agent practitioners are responding with verified Markdown handoffs and background compaction, as Pragmatic_Eng and anderslie describe.

The Penn State paper gives the failure a name, Session Constraints, and tests it across chat, coding-agent, and research trajectories. Its open-source repository separates the compaction transform from the registry experiment. In parallel, HumanLayer's workflow notes turn long coding sessions into Markdown research and plan files, while a Pragmatic Engineer interview describes the same pattern as frequent intentional compaction.

The Pennsylvania State University authors define a Session Constraint as a user instruction that governs behavior for the rest of the current session, is separate from the main task, and has no intended life beyond that session. The canonical example is “show me the draft before sending anything from now on”: the email is the task, while the approval rule governs how the task is performed.

The paper groups its 15 test constraints into five categories:

• Action: what the agent may do, such as requiring approval before a command or change.
• Information: what the agent may read, reveal, or write.
• Process: how the agent must carry out a task.
• Preference: which equivalent option it should choose.
• Output: properties of the response, such as bullet-only formatting.

The distinction separates task continuity from behavioral continuity. A summary can preserve “organize the inbox” and the progress made while omitting “never send without confirmation.” The authors describe that as a silent integrity failure in their formal definition.

CompInt varies four dimensions: constraint category, strict versus preferential wording, explicit versus implicit session scope, and injection position. The authors place constraints at the top, middle, or bottom of a long history, or repeat them at multiple positions.

The long-context environments are:

• WildChat: multi-turn conversations without tool use.
• Hermes Agent: tool-calling, coding, and multi-step agent trajectories.
• OpenResearcher: long-horizon research with search, open, and find tools.

Each context is roughly 100,000 tokens, about 80% of a 128,000-token window, with 50 instances per dataset. The paper evaluates both whether the compacted text still contains the constraint and whether a downstream model follows it in a forced-choice probe.

That distinction puts a boundary around the headline number. The reported 17% is semantic retention in the compacted summary, not a direct estimate that 83% of real-world agent actions violate a rule. Retention is judged by GPT-5.4, while compliance is measured by whether the probing model selects the constraint-following answer. The repository documents the four comparison conditions, including full context with and without the constraint, compacted context, and a compacted context with the constraint reintroduced afterward in its evaluation README.

The proposed fix moves session rules into a separate running registry instead of asking the task summarizer to remember them. The paper reports more than 90% retention across all three environments without changing the compactor or the underlying LLM.

The released evaluator implements the registry as a sequential strategy layer:

• At each user turn, an extractor receives the current user message, the previous assistant turn, and constraints already in the registry.
• New constraints are appended as structured entries, with duplicate text filtered out.
• After the pass, a judge checks whether the originally injected constraint appears in the rendered registry.

The extractor implementation makes clear what the 90% figure measures: preservation in the separate registry. The repository also keeps behavioral compliance as a distinct metric, so a rule being present in memory and a model obeying it remain two separate tests.

Dex Horthy, founder of HumanLayer, describes intentional compaction as compressing noisy context into a clear Markdown artifact, verifying it, then starting a fresh conversation. His workflow turns context management into explicit handoffs rather than relying on an opaque summary.

The handoff sequence is:

1. Research: inspect the codebase and write down its current state.
2. Design: combine the ticket and the research into a document describing the desired state and unresolved design questions.
3. Planning: carry the compressed research and design into a new session to produce an implementation plan.
4. Implementation: execute the plan, then fold verified progress back into the plan document.

HumanLayer's published workflow describes keeping context utilization around 40% to 60%, using subagents for isolated research, and compacting status after each verified implementation phase. The Pragmatic Engineer's interview summary places human review at the design and architecture boundary.

Refactoring creates a second way to avoid carrying unrelated history forward. [0xblacklight] says discrete refactoring tasks can run in isolated context windows, with deterministic linter feedback providing the completion signal 0xblacklight's refactoring note.

The pattern changes the unit of work from one sprawling conversation to small tasks with their own context and machine-checkable feedback. It addresses context pollution through separation, rather than asking a single summary to preserve every intermediate detail.

Background compaction keeps the agent working while a summary is generated. [anderslie] describes a soft ceiling at 180,000 tokens for a 200,000-token context, followed by an atomic replacement once the summary finishes anderslie's background-compaction design.

The mechanism has four moving parts:

• Start compaction at the soft ceiling, before the hard context limit.
• Let new turns continue while the summary runs.
• Replace the turns before compaction with the summary, then append turns created during the summary job.
• Reuse the normal system prompt and add a strongly worded compaction instruction, avoiding a separate uncached full-context prefill.

Horthy describes another operational pattern, a queued sequence of implementation, review, compaction, testing, and another implementation pass in the 200,000 to 350,000-token range dexhorthy's compaction queue. The workflows differ in timing, but both treat compaction as an orchestration problem with explicit state boundaries, not as invisible cleanup at the edge of the window.