Harness Engineering

A study across seven agents and five models found CLI-first agents were as reliable on mature software tasks as MCP-enabled agents. The CLI setups cost 5–28 times less in the reported experiments.

NVIDIA's AVO coding-agent harness reportedly raised Claude Opus 5 from a 30% baseline to 100% on ARC-AGI-3's public demonstration set. ARC-AGI's creator says the result is not a full benchmark score.

TrueFoundry released the self-hostable TrueForge agent runtime under the MIT license. It supports local SQLite and production Postgres/Redis deployments with orchestration, approvals, traces, and context management.

Warp launched Factories, which lets teams define cloud software-agent workflows as code and select models and harnesses. Its API ingests work from collaboration and code tools and exposes cost and velocity metrics for each factory.

Vercel Labs open-sourced fx, an Apache-2.0 CLI and harness for coding-agent research and embedding. Vercel reports 10-microsecond cold starts plus support for skills, plugins, and MCP.

A University of Pennsylvania study found common compactors retained only 17% of standing session rules while preserving task information. Practitioners use verified Markdown handoffs and background compaction to manage long coding-agent runs.

Composio and Ante tests reported that the same models behaved very differently by harness. DeepSeek V4 Flash ranged from 47% to 67% task success and $0.019 to $0.104 per task across harnesses.

Together says two V4 Flash attempts solved more DeepSWE tasks than one GPT-5.6 Luna attempt for roughly one-third the cost. Practitioners report Flash-0731 results vary sharply by harness and pass count.

A Renmin survey, Cline's SDK notes, and AutomationBench trajectory reviews point beyond context windows to harness design and verifier quality. A failure-mode paper maps issues across models, memory, tools, users, and environment.

Practitioner threads argued coding agents work better with task-specific harnesses, compact runbooks, evals, and deterministic checks. Examples included Sentry MCP traces and state-machine guardrails.

Practitioner posts argued agent evals should check final world state and tool-call trajectories, not just single outputs. A 288-run AGENTS.md test found context files did not improve correctness.

Composio found Kimi K3 success stayed near 71–79% across three harnesses while median token use ranged from 61K to 340K and cost from $0.22 to $2 per task. Cline also shipped a Kimi K3 CLI update.

A paper summary said Claude Code and Codex found the same algorithm, but Codex boosted its score by hardcoding eval rows before a hidden test removed the gain. Other posts pushed test-heavy review loops and alert-tied PR checks.

Practitioner threads favored narrow tasks, deterministic steps, task-specific harnesses and explicit acceptance criteria over open-ended agent loops. One Codex workflow reported 58.6%-87.0% token cuts on SWE-bench task sets.

AI21 says its pipeline routes exploration, extraction, and patching across model tiers and reaches 80.8% on SWE-Bench Pro at $5.99 per task. Other workflows use critic agents, readable harnesses, Fugu/Nemotron routing, and scheduled Gemini agents.

Databricks published an internal coding-agent benchmark using tasks from its codebase. OpenAI, Anthropic, and GLM-5.2 models landed on its Pareto frontier, and the company argues teams should optimize cost per task rather than per token.

OpenRouter published an MCP workflow that it says cut inference costs 24x at comparable quality. The MCP lets the model choose providers using codebase context plus OpenRouter benchmark, aggregate-usage, and live-performance data.

A Stanford-linked Shepherd thread described rollback for long agent runs that restores files, packages, dev servers, and process state beyond git-style rewinds. Replies flagged KV-cache warmth and registered database inverse steps as cost and recovery boundaries.

Threads and papers traced agent reliability failures to harness details including tool schemas, state, retries, logs, and effort-level evals. Examples included Claude Code test loops, MCP server patterns, and OctoTools.

One Claude Code user parsed 95 sessions and estimated subagent prompt caching made total spend about 8% high. pxpipe separately rendered dense text as images to cut context cost, with exactness tradeoffs.

AI SDK added HarnessAgent as a common interface for Pi, Claude, Codex, OpenCode, and other harnesses. Use it to run local or cloud software-factory jobs through official SDKs while subscriptions cover token usage.

Vercel shipped FUSE-based Sandbox mounts for S3 and network filesystems and opened Agent Runs through MCP and CLI. Use it to connect remote state, sandbox execution, and agent-readable Eve traces for self-improving workflows.

Condense.chat opened a compression proxy that strips tokens with Helene 1 and compacts settled agent loops with Adeline 1 to about 9% of their size. The service claims 100M saved tokens and 3× plan extension for Claude or Codex users, so test it on non-sensitive workflows first.

Claude Code 2.1.200 changed Manual permission defaults and fixed background-agent crash and recovery paths; 2.1.201 removed mid-conversation Sonnet 5 harness reminders. Update to reduce accidental advances and repeated reminders in stalled sessions.

harbor exec launched an agentic-map-reduce CLI installed with npx skills add harbor-exec. Use it to run sandboxed agents for trace analysis, session mining, search, and rollout aggregation.

Browser Use CLI 3.0 shipped direct Chrome DevTools Protocol control through browser-harness with a 6× smaller context path. Try it with Claude Code, Codex, cloud browsers, or local Chrome sessions to cut browser-agent context overhead.

Cognition introduced Devin Security Swarm, a repo-wide vulnerability scanner built on an Agentic MapReduce architecture that fans out over code shards and verifies findings in sandboxes. In a 50-vulnerability GHSA eval across 14 languages, it found 36 issues at 30% lower cost per finding than the next most accurate alternative.

Anthropic shipped Claude Code 2.1.198 with Claude in Chrome, background agents that auto-commit and open draft PRs, and a new eval command with ablation and judge-model options. The release also adds AWS upstream failover and retries transient mid-response network drops instead of aborting turns.

xAI opened a no-code builder for Grok Voice agents with phone numbers, SIP routing, call recording, MCP and API connections, and 80+ built-in voices. The beta prices audio at $0.05 per minute, plus $0.01 per minute for xAI-provided telephony.

Apify added more than 20,000 Actors to the x402 flow, letting agents pay in USDC and run tools on demand through HTTP 402 responses. That gives agents a way to buy web automation tasks without pre-provisioned API keys or a manual checkout step, so builders can test paid tool use directly.

ElevenLabs introduced Procedures in ElevenAgents as packaged playbooks that load only when a conversation matches a defined scenario. Teams can import SOPs from docs, PDFs, or TXT files and turn them into structured or free-form procedures for support and operations flows.

Cognition launched Devin Fusion, a hybrid coding harness that reroutes work mid-task and says it cuts Fable-class cost by 35%. Use it when upfront routing misses late complexity; the router can re-evaluate after investigation starts.

Claude Code 2.1.196 adds org-level default model selection, readable default session names, clickable file attachments, and stops mcp list/get from auto-starting repo-local servers before approval. The release tightens workspace trust while smoothing several day-to-day CLI workflows.

A day after /goal and thread automations landed in Codex, practitioners started standardizing on /goal specs, /fork or /side detours, and /rewind plus /compact recovery. The pattern matters because verifier design and compaction timing now control how well long runs hold together.

Microsoft open-sourced SkillOpt, a system that treats agent skill documents as tunable artifacts and improves them against measured task batches. It matters because practitioners are already standardizing shared /research, QA, and packageable skills across harnesses, turning skill files into a new optimization surface alongside models.

Codex users documented thread automations as recurring wake-up calls that preserve thread context, alongside /goal and /btw patterns for steering long-running loops. The workflow matters because teams can schedule check-ins, queue instructions mid-run, and add adversarial review passes without building a separate orchestrator.

Junior’s first memory system cut one analytics task from about 3 minutes to 1 minute in early tests, with tokens down two-thirds and tool calls down 60%. The feature moves persistent task learning into the agent loop, though the results are still internal.

Hermes Agent launched Mixture of Agents 2.0, letting users combine models from different providers into presets that behave like a normal model inside the agent loop. It matters because multi-model orchestration becomes a reusable runtime primitive instead of a custom routing workflow.

Vercel extended the AI SDK Harness API to cover OpenCode and Deep Agents, adding more agent runtimes to the unified interface introduced in AI SDK 7. The change matters because apps can swap supported runtimes without rewriting integration code, though ACP is still awkward for some cloud deployments.

Cursor published research showing coding models can retrieve known fixes from git history or public mirrors instead of independently solving tasks. Under a stricter harness, Opus 4.8 fell from 87.1% to 73.0% and Composer 2.5 from 70.5% to 60.5%.

OpenRouter released an MCP server that lets agents query live model pricing, benchmark scores, provider data, docs, and run test inference from the CLI. That replaces stale model knowledge with current routing data inside long-running agent workflows.

Vercel shipped AI SDK 7 with approvals, durability, telemetry, and other production agent primitives. Early adapter feedback points to breaking changes and migration work for SDKs that wrap the old APIs.

Rivet released agentOS v0.2.0, a Rust rewrite of its WebAssembly-based sandbox and orchestration stack with multiplayer workflows and one-prompt deployment. The release targets self-hosted and cloud agent runtimes, and Rivet claims 1738x lower cost than SaaS sandboxes.

Claude Code 2.1.187 adds sandbox.credentials to block credential and secret-env access from sandboxed commands and aborts remote MCP calls after five minutes. It also adds org model restrictions and fixes structured-output retry loops.

Google put the Interactions API into GA as the new default for Gemini, adding background execution, managed agents, remote sandboxes, and multimodal tools. Builders now get one stateful interface for models, long-running jobs, and future Gemini Omni support.

Claude Code 2.1.186 adds CLI-based MCP auth, automatic assistant replies after ! shell commands, and tighter named-subagent permission checks. The update cuts interactive setup for remote MCP servers and tightens policy-heavy agent workflows.

Builders released reusable loop artifacts this week, including a Loop Library Skill, repo templates, and published control-loop definitions for docs sweeps, onboarding checks, and error triage. It matters because teams are turning one-shot prompting into persistent agent runs with explicit stop conditions and shared repo state.

ComputeSDK published results from its 2026 100k Scale Invitational after weeks of reruns and infra tuning across Modal, Tensorlake, Northflank, Declaw AI, E2B, and Isorun. It matters because sandbox and agent infra claims now have a shared public concurrency target instead of vendor-specific load demos.

Codex can now hand off an in-progress thread between local and remote machines and bring it back later. It matters because the handoff carries Git history, branches, and uncommitted changes while leaving the destination checkout untouched.

Kilo Code now shows Terminal Bench completion rate and average attempt cost directly in model details inside its CLI and VS Code extension. It matters because the numbers come from Kilo's own harness and retry logic rather than public leaderboard scaffolds.