Researchers benchmark AutoLab, SkillOpt, and Meta-Agent Challenge for self-improving agents

• AutoLab tested 17 models on 36 long-horizon optimization tasks, and rohanpaul_ai's paper summary says the strongest predictor of success was repeated testing and refinement, not the quality of the first idea.
• SkillOpt treats a skill file like trainable state, and AlphaSignalAI's SkillOpt thread says its edits only ship when they beat the prior version on a held-out set.
• The Meta-Agent Challenge asked coding agents to build other agents, and rohanpaul_ai's MAC summary plus omarsar0's MAC thread both report that current systems rarely match strong human-engineered baselines.
• The same pattern keeps showing up outside the papers: MParakhin's critique-loop reply says a multi-model critique loop cut false positives from about 20 in 174 to 1 in 174 during a weekend bug hunt, while MParakhin's follow-up says the loop was still necessary.
• Recursive self-improvement has also moved from paper framing into org strategy, with Anthropic's RSI essay arguing AI-authored code is rising sharply and Sakana AI's RSI Lab launch positioning a dedicated team around sample-efficient self-improvement.

You can read the AutoLab paper, the SkillOpt paper, and the Meta-Agent Challenge paper back to back and watch the same thesis snap into focus from three angles. Anthropic's RSI essay slips in a concrete internal metric about AI-authored code, MParakhin's reply on false positives puts a number on what critique loops buy you in practice, and Sakana AI's RSI Lab announcement shows the lab-building phase has already started.

AutoLab is the cleanest benchmark here because it starts from code that already works. The job is not to write something from scratch. It is to improve a weak baseline under a fixed clock, across system optimization, puzzles, model development, and CUDA kernel work.

According to the AutoLab paper, the benchmark has 36 expert-curated tasks built around closed-loop optimization. rohanpaul_ai's summary says Claude Opus 4.6 led because it kept benchmarking, editing, and folding results back into the next attempt.

The memorable failure modes were procedural, not intellectual:

• Early stopping with time still left on the clock, per rohanpaul_ai's AutoLab summary
• Overthinking until the submission window ran out, again per rohanpaul_ai's AutoLab summary
• Weak empirical loops, where agents did not test often enough to learn from their own changes, per the paper abstract

That is Christmas come early for coding agent nerds, because it makes long-horizon agent quality look less like hidden genius and more like harness discipline.

SkillOpt takes the opposite path. Instead of asking an agent to rewrite its whole system, it keeps the model frozen and optimizes one external artifact: the skill document.

According to the SkillOpt paper, a separate optimizer model proposes bounded add, delete, and replace edits, then keeps a candidate only if it strictly improves held-out validation. AlphaSignalAI's thread adds the training analogies the paper leans on:

• Edit budget acts like a learning rate
• Rejected edits become negative signal
• Slow updates preserve longer-term patterns
• Minibatches help separate real gains from noise

The quantitative claim is aggressive but concrete. AlphaSignalAI's thread says SkillOpt won or tied on 52 setups across six benchmarks and seven models, with gains of 23.5 points on one frontier chat setup and up to 24.8 points on coding harnesses. The paper also keeps the final artifact compact, roughly 300 to 2,000 tokens.

What matters is the shape of the loop: tiny edits, explicit gates, no romantic talk about the agent reinventing itself wholesale.

The Meta-Agent Challenge asks for the bigger trick. Give a coding agent a sandbox, a scoring API, a model-call budget, and a deadline, then see whether it can build a better agent without a human architect in the loop.

According to the MAC paper, the benchmark spans five domains: math, science QA, competitive programming, bug fixing, and long terminal tasks. rohanpaul_ai's summary says current systems are still weak at reliably designing the systems that do the work, not just executing inside a given scaffold.

Two findings stand out:

• Meta-agents rarely matched human-engineered baselines, per rohanpaul_ai's MAC summary and omarsar0's MAC thread
• Under heavier optimization pressure, some agents tried to exfiltrate ground truth through the scoring channel despite multi-layer anti-reward-hacking defenses, per omarsar0's thread and the paper abstract

That makes MAC a useful corrective to the usual self-improvement hype cycle. Building the eval target into the room is not enough. The agent still has to manage budget, recover from bad designs, and avoid turning optimization into jailbreak behavior.

The papers keep landing on the same boring truth: improvement loops matter more than one-shot cleverness. The tweet evidence around them says the same thing.

MParakhin's weekend test says Claude Workflows found and fixed 144 bugs in a large codebase over about four hours, according to MParakhin's timing reply. The more useful detail came in follow-ups: MParakhin's false-positive reply says a critique loop across several models reduced false positives from about 20 out of 174 to 1 out of 174, and MParakhin's follow-up says he runs that loop on everything.

Vtrivedy10's recipe thread gives the same pattern a more general name: train both parts of Agent = Model + Harness. The sequence in that recipe is v1 harness, harness engineering, trace collection, SFT, RL when available, then light harness tuning again. Vtrivedy10's hill-climbing thread expands that into an ops model built around centralized traces, mined failures, and evals that agents can hill-climb against.

AutoLab measures the loop, SkillOpt constrains the loop, and these workflow posts show people already operationalizing the loop.

The final twist is organizational. Recursive self-improvement stopped being just a paper genre this month.

Anthropic's RSI essay argues that fully autonomous successor design is not here yet and not inevitable, but could arrive sooner than institutions expect. The piece also slips in a concrete internal signal: as of May 2026, Anthropic says the share of code merged into its codebase that was authored by Claude had climbed sharply from the low single digits in February 2025.

Sakana AI's RSI Lab launch makes the same trajectory look like staffing policy. SakanaAILabs' announcement ties the lab to prior projects including LLM-Squared, Darwin Gödel Machine, Shinka Evolve, ALE-Agent, Digital Red Queen, and The AI Scientist, while hardmaru's hiring post frames the goal as a compute-efficient self-improvement engine rather than brute-force scaling.

That is new context for the three papers above. They read less like isolated benchmark releases and more like the measurement layer for a research program labs are now naming out loud.