ASI-Bench finds full procedures lift research-agent scores to 50.91

• Full procedures carried most of ASI-Bench's measured research-agent performance: the ASI-Bench report put the average at 50.91 with complete guidance, versus 29.10 when prompts named only a method.
• The benchmark holds the research goal, data, and evaluation fixed while reducing guidance from a complete procedure to method-only, then goal-and-data-only, as ASI-Bench's overview describes.
• Skills primarily stabilize execution rather than inject facts: the skills study attributed 65.7% of successful cases to procedural anchoring and 4.5% to explicit knowledge injection.
• Static quality gates were a weak proxy for runtime value, with the ACES study reporting a 0.14 Spearman correlation between structural scans and LLM-judge quality across 145 skills.
• Carrying forward feedback and context improved sequential task performance in 14 of 15 settings, while ContinualSkillBench found explicit skill maintenance roughly tied with in-context learning in a key ablation.

The ASI-Bench paper builds each task around an objective, data, an executable environment, and verifiable artifacts. NVIDIA's evaluation documentation draws a blunt distinction: a skill can pass safety and structure checks yet leave task completion unchanged.

ASI-Bench contains 60 project-level research tasks across 11 fields, assembled by more than 40 experts over 31,000-plus human hours. Its four conditions keep the project and scoring stable while removing increasingly consequential instruction.

• B1: complete methodological guidance.
• B2: the method name only.
• B3: only the research goal and data, leaving method selection to the agent.
• B4: the B3 setting plus task-irrelevant information to test robustness.

Across 18 agent-model configurations, the ASI-Bench report places the B1-to-B2 drop at 21.81 points, from 50.91 to 29.10. B3 averaged 26.62, a further 2.48-point decline, while method-only prompts consumed 59% more tokens than complete instructions.

The skills study analyzed 8,135 trial records to isolate why an agent skill helped, rather than merely whether it raised aggregate success. Its useful cases were overwhelmingly about preserving a sequence of actions.

• Procedural anchoring accounted for 65.7% of successful skill cases.
• Knowledge injection accounted for 4.5%.
• Distilling trajectories with visible success and failure labels reached 74.6% success, while removing those labels dropped success to 40%.
• Retrieval precision collapsed as catalogs grew from five to 100 skills, but task success moved little because agents could sometimes extract a useful procedure from the wrong retrieved skill.

The study frames a successful skill as a checklist, execution order, tool sequence, or verification plan. That matches ASI-Bench's ladder: dropping explicit steps did far more damage than dropping the named method.

NVIDIA's ACES paper treats a skill as an executable artifact and measures its marginal value through paired trials: the same task, model, harness, workspace, and scorer run once with the target skill and once without it.

Its three-tier evaluator combines:

1. Deterministic validation, including safety and structural checks.
2. Semantic deduplication against the catalog.
3. A live sandbox evaluation that compares with-skill and without-skill task runs.

The researchers scored 947 paired cases from 58 production skills across four harnesses, using the Agent Trajectory Interchange Format to normalize runs. The ACES report says the largest gains appeared in skill execution, behavior checks, and skill efficiency, metrics a static scan cannot observe. NVIDIA has released the underlying SkillEvaluator repository under Apache-2.0.

ContinualSkillBench uses five domains with 100 connected subtasks apiece, designed to create opportunities for cross-task reuse. Sequential execution improved normalized reward by 16.9% overall and beat solving each task from scratch in 14 of 15 model-domain combinations.

The ContinualSkillBench paper found that in-context continual learning performed comparably to explicit skill maintenance on average. In the GPT-5.3-Codex ablation across law, finance, and healthcare, pure in-context learning averaged 0.605 normalized reward, compared with 0.602 for explicit skill maintenance.

Kunchenguid's Backpass announcement separates hand-written user-level AGENTS.md instructions from project-level instructions learned from prior work. The open-source tool samples a repository's past agent sessions, distills learnings and losses, then produces a reviewable proposal for project instructions and skills; the post gives npx -y backpass as its entry point.

Harbor frames the parallel evaluation loop around environments: its operating thesis calls RL environments software packages that should be maintained and versioned, and agents models to be improved by hillclimbing a test set. In a team announcement, Harbor estimated more than one billion rollouts and over $10 billion of token spend this year, while its hiring call asked candidates to have built something with Harbor.