Study finds CLI-first agents cost 5–28x less than MCP agents

• Two scaffoldings with no MCP support, Pi and Tau, completed every run of a fixed repository task and came in 5x to 28x cheaper than the five MCP-capable alternatives, according to pidotdev's report.
• MCP did not produce a stable cost advantage in strictly paired tests: the paper linked by pidotdev reports 13 MCP-to-CLI ratios from 0.43x to 29x.
• Interface instructions did not reliably constrain the agents, because pidotdev's follow-up found they kept using MCP when prompted not to; removing the option was the effective control.
• Failed MCP runs represented 12.9% of MCP spend, versus 2.2% for CLI runs, in the study cited by pidotdev.

The study verified repository state rather than trusting agent self-reports. The current MCP specification describes tools as model-controlled, a detail that makes the study's access-layer enforcement result less surprising.

The researchers ran six operations against a private online Git repository with five language models and seven scaffolds. Completion meant the repository was actually in the required state.

• No MCP support: Pi, Tau
• MCP-capable: Hermes, Codex, opencode, Claude Code, qwen-code

The split matters because the paper compares complete agent harnesses, including their context handling, tool loops, and stopping behavior, rather than comparing a shell command with an MCP call in isolation.

The headline 5x to 28x result comes from CLI runs with no MCP server attached to any setup. Pi and Tau completed the task in every run and were cheaper than all five MCP-capable scaffolds in that condition, as pidotdev's report summarizes.

The useful result is narrower than the slogan: harness choice swamped the interface in this controlled coding task.

The protocol comparison itself did not resolve to one direction. Across 13 strict MCP-versus-CLI pairs, the authors measured cost ratios from 0.43x to 29x; ratios below 1 mean MCP was cheaper in that pairing. An independent technical analysis reaches the same conclusion: a quoted multiplier belongs to a particular scaffold.

A local 27B model made the point sharply. It completed the task under every scaffold, while cost varied by 139x across them, according to the paper.

Prompting an agent to use only one interface did not make it comply. pidotdev's follow-up says the researchers had to remove MCP access to prevent MCP use.

Pi and Tau were excluded from those extra forced-interface runs because neither has MCP built in, as pidotdev's explanation notes. That preserves an important distinction between agents that can be instructed to avoid a tool and agents that cannot access it at all.

The authors found failures equally often under MCP and CLI in both the original runs and repetitions. Their paper, however, puts the share of spend that bought no completed work at 12.9% for MCP runs and 2.2% for CLI runs.

That separates task failure frequency from the cost of an unsuccessful trajectory.

The experiment covers one six-operation Git task, not every external integration. pidotdev's reply identifies the boundary directly: MCP may be stronger where the CLI ecosystem is less mature.

MCP's standard role is connecting LLM applications to external data sources and tools, per the official specification. The study establishes that a mature command surface can be enough for this particular class of work.