Skip to content
AI Primer
breaking

Prime Intellect publishes Prime Agent report with 7-day Factorio evaluation

Prime Intellect’s report describes a self-improving long-horizon agent harness with persistent memory, skills, prompts, and subagent specifications. Its Factorio evaluation ran for seven days using 23.4 million output tokens across 633 trajectories.

5 min read
Prime Intellect publishes Prime Agent report with 7-day Factorio evaluation
Prime Intellect publishes Prime Agent report with 7-day Factorio evaluation

TL;DR

  • Prime Agent shifts long-task state into a persistent Python process and disk-backed store, so the model can programmatically handle information beyond its active context, as rohanpaul_ai's summary explains.
  • Its self-improvement loop updates stored prompt notes, memories, skills, and reusable subagent specifications while the model weights remain fixed, according to dair_ai's overview.
  • The authors report ARC-AGI-3 RHAE Best@1 rising from 30% to 95.5% for the same model class, plus competitive results on long-context coding, GPU kernels, emulator construction, and nanoGPT speedruns, per dair_ai's benchmark summary.
  • A seven-day Sonnet 5 Factorio run used 23.4 million output tokens, completed 24 of 196 technologies, and spawned 633 subagents, iScienceLuvr's report says.

One model-facing tool, ipython, serves as the control surface: the usage guide says file operations, project commands, skills, and recursive subagents run through Python. The technical report also documents four-character Factorio control, 19 validated nanoGPT records across 85.5 hours, and long-horizon MazeBench exploration. gdb gdb reduced the pitch to “a new way to get knowledge work done.”

State hierarchy

Addressable state is the report's sharpest idea. The technical report organizes it into four levels:

  1. L0: fixed model weights.
  2. L1: the active token context.
  3. L2: persistent REPL values and recursive subagents.
  4. L3: disk-backed event history, memories, skills, prompts, and agent specifications.

Compaction replaces a conversational prefix with a summary, while the original events remain retrievable in L3. The agent can serialize selected Python values or tool output back into L1, instead of repeatedly stuffing large logs and task specifications into the prompt.

The failure mode resembles the one in Vtrivedy10's post, where instructions from an initial user message disappear during compaction.

Continual Harness

The Continual Harness exposes four typed kinds of durable state:

  • Prompt notes for behavioral instructions.
  • Memories for facts and corrected assumptions.
  • Skills for executable procedures.
  • Subagent specifications for reusable roles and divisions of labor.

The technical report says agents can create, read, update, and delete these entries. Session-local state stays with one session by default; an explicitly requested global entry can be reused later.

An agent can request an edit directly, or invoke /refine for a background model pass over relevant events. Refinements apply at turn boundaries, retain their trigger and intended effect, and remain versioned for provenance and rollback. The source repository publishes the implementation as open source.

Recursive sessions

The RLM primitive, rlm(), creates and schedules a child session, then returns a stable handle before that child finishes. Each child has its own context, IPython kernel, history, and workspace metadata, while its parent continues local work.

The Prime Intellect launch post describes recursive delegation as programmatic tool calling. The report adds three execution details:

  • A daemon owns live root and child sessions, so client detachment does not end a run.
  • Daemon-mediated message queues let parents, children, and siblings exchange messages after one participant becomes active again.
  • The Agents View lets a human inspect a session tree, attach to a specific session, send input, or detach without interrupting execution.

The runtime also records model calls, tool use, messages, harness edits, retries, verifier outcomes, and resources. Root and descendant costs aggregate together, keeping delegation visible in the token, time, and cost accounting described by dair_ai's overview.

Evaluation accounting

Prime Agent frames long-horizon evaluation around score at a fixed expenditure and score at a practical plateau, the latter intended to show the shape of improvement over time. The technical report separates model inference, Python execution, and tool calls, then reports tokens, time, and cost independently.

Its long-running controls are explicit:

  • Autonomous mode continues within a set budget and checks a task-specific end condition after every turn.
  • Goals retain an objective across continuations until the agent marks it complete.
  • Heartbeats start timed or cron-scheduled turns.

An evaluation configuration binds task and tool interfaces to model settings, compaction and refinement policy, retries, completion gates, and resource limits. That is the paper's attempt to distinguish an exhausted model from a harness that dropped state, terminated early, or lost track of spending.

Benchmark table

The headline result is the reported ARC-AGI-3 RHAE Best@1 change from 30% to 95.5%.

The paper also says Prime Agent matches or exceeds native and popular harnesses across several long-horizon tasks, but its own table is more uneven. With Opus 5, Prime records 0.900 versus Claude Code's 0.920 on OOLONG and 0.744 versus 0.746 on LongBench v2; with GLM-5.2, it records 0.700 versus Pi-mono's 0.420 on OOLONG.

The technical report says bolded table cells mark higher point estimates, not statistical significance, and supplies no uncertainty intervals. It also says its Claude Code and Codex ARC-AGI-3 reruns fell below Anthropic's and OpenAI's self-reported public-set results, so those external figures provide context instead of isolating a harness-only effect.

Seven-day Factorio run

Factorio is the report's clearest endurance demonstration. Its environment exposes Python observations and actions against a persistent factory world, and the paper tracks technology progress alongside the agent tree's growth and concurrency.

Across seven days, the Sonnet 5 root and descendants produced 23.4 million output tokens. They completed 24 of 196 technologies, reached 71% of advanced-circuit research, and showed no signs of stalling at the report's endpoint, according to the technical report.

iScienceLuvr's report puts the run at 633 subagents, with at most seven simultaneously active. That concurrency figure sits beside the paper's core claim: refinement preserved useful work across a continuously evolving factory instead of resetting the agent after each trajectory.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 2 threads
TL;DR1 post
State hierarchy1 post
Share on X