Echo Gap paper reports agents endorsed 31%–54% of their own wrong answers

• Self-improving memory loops can get worse without weight updates, as rohanpaul_ai's Echo Gap summary says tested models endorsed 31% to 54% of their own wrong answers as correct.
• LUCID is the mitigation to notice: rohanpaul_ai's paper screenshot includes a BIRD text-to-SQL gain to 56.9% execution accuracy, above a 54.0% self-graded memory agent and a 52.4% memory-less agent.
• Meta and UIUC treated harness use as a trainable policy, with omarsar0's EvoHarness-RL summary reporting Qwen3-8B at 96.9% on ALFWorld after learning when to read, update, and consolidate external state.
• Metis moves memory into the model path, and rohanpaul_ai's Metis summary reports LoCoMo Gold at 26.74 in a no-context setting, far above vanilla Qwen3.5-27B at 0.07 but below full-context Qwen3.5-27B at 65.03.
• Shipping harnesses are already externalizing state through inter-session messages, task boards, and skill files, with Pi's intercom post, LoopX's post, and Teknium's /learn post showing the pattern.

The Echo Gap paper frames stored self-scores as proxy rewards and points to code, data, and per-episode traces. The EvoHarness-RL paper names a compact BPE interface, while Metis ships a repo for memory states inside the backbone. A DEV post on agent memory framed ordinary agent memory more bluntly: transcript, fact store, and filesystem notes are different things that all get called memory.

The Echo Gap paper studies agents that store past episodes as (task, answer, score), retrieve similar episodes later, and write new episodes back into memory without changing model weights.

The score is the trap. Once it affects which memory gets retrieved, trusted, or imitated, it behaves like a reward signal in a frozen-weight learning loop.

The paper's Table 2 reports the self-endorsement rates that made the result pop:

• Claude Haiku 4.5 bank: 31% of wrong facts endorsed, 95% CI [26, 35].
• GPT-5.4-mini bank: 54% of wrong answers endorsed.
• GPT-5.4 bank: 41% of wrong answers endorsed.

The paper cautions that these are not clean model rankings because the gold-judging setup differs across banks. The robust finding is nastier for agent builders: wrong memories can receive high scores, then reappear as trusted precedent.

LUCID, the paper's answer-free de-inflation method, raised BIRD execution accuracy to 56.9%, compared with 54.0% for a Memento-style self-graded memory agent and 52.4% for the same architecture without memory.

The paper's correction criterion is the Error-Independence Assumption, or EIA.

A useful verifier has two jobs:

• Track truth: Corr(V, U) should be high.
• Avoid echoing the memory bias: |Corr(ν, b)| should be low.

That second condition is the useful part. The paper says stronger or different-family re-graders, judge ensembles, confidence thresholding, self-consistency pruning, and budget-matched random pruning did not supply the needed decorrelation on the tested banks.

Execution feedback, tests, retrieval evidence, invariant checks, and other non-gold signals matter here because their errors can differ from the model's own self-grading blind spots.

The EvoHarness-RL paper treats the external harness as policy-facing state rather than hand-written scaffolding.

Its BPE workspace has three parts:

• Belief: task-relevant environment facts.
• Progress: attempted, completed, blocked, and pending subgoals.
• Experience: reusable skills, failure modes, priors, and strategies.

The action protocol is deliberately small:

• track: read belief.
• commit: write a progress update.
• recall: retrieve experience.
• note: store a new insight.

Training has two stages: supervised harness fine-tuning teaches the action space, then cost-aware GRPO optimizes when state access is worth the interaction budget. On ALFWorld, the paper reports 96.9% success on the seen split and 86.6% on the unseen split.

The two named dynamics are clean vocabulary for a pattern agent engineers are already seeing. Harness annealing means repeated harness-use patterns get absorbed into the model policy; harness evolution means progress and experience updates compress the workspace into a compact task state.

Metis attacks the same problem from the model side. The Metis repo describes a native memory state that lives inside the backbone, updates through forward computation, and leaves learned weights frozen at inference time.

The mechanism has two claims:

• Persistent memory state: historical context is compressed into native memory and accessed through memory attention.
• Native memory procedures: storing and using information are learned from data, not bolted on as retrieval and prompt-construction rules.

The result is not a replacement for full context. rohanpaul_ai's Metis summary reports Metis-27B at 26.74 on LoCoMo Gold with no context, compared with 0.07 for vanilla Qwen3.5-27B and 4.24 for Temp-LoRA-27B, while Qwen3.5-27B with full context scored 65.03.

The failure mode is also the engineering problem: as more information is compressed into a fixed-size state, performance drops, and irrelevant stored memory can interfere with ordinary tasks.

HANDBOOK.md makes the same point from the long-horizon policy side: keeping the rule in context does not mean the rule controls the action.

In the cited failure, Claude Opus 4.8 found that a $7,500 expense lacked valid manager approval, then reversed itself and cleared the expense anyway. AlphaSignalAI's summary says the benchmark covers 65 tasks with policies up to 124 pages, completed trajectories average roughly 17 reasoning steps and 30 tool calls, and the strongest tested configuration passes only 36.2%.

The listed failure mechanisms are a useful checklist:

• Compounding error.
• Context competition.
• Growing context cost.
• Irreversible actions.

A larger context window can keep policy text visible, while the follow-up says checkpoints, protected constraints, action gates, and verification are the mechanisms that bind state to action.

Production harnesses are converging on external state that models can inspect, mutate, and hand off.

The pattern shows up in three different forms:

• The pi-intercom docs describe direct 1:1 messages between Pi sessions on the same machine through a local IPC broker, with send and ask flows for planner-worker coordination.
• LoopX's post describes a local control plane that remembers goals, owners, evidence, budgets, and handoffs so agents can claim tasks across days without stepping on the same file.
• Teknium's /learn post turns large sources into knowledge-base skills: a lean SKILL.md index stays loaded, chapter files load on demand, and source text is treated as data with invisible Unicode stripped.

Memory is becoming an execution substrate: messages, files, task boards, skill indexes, and model-native state all compete to decide what the agent treats as precedent.