Databricks reports coding-agent token spend is rising exponentially

• Coding-agent spend is now an exponential cost line at Databricks, and Yuchenj_UW's post says GLM 5.2, Opus 4.8, and GPT-5.6 Sol sit on the company's quality-per-dollar frontier.
• Task-level economics beat token-price math: Yuchenj_UW's reply points to Databricks' internal coding benchmark, where Opus 4.8 looked better than Opus 5 for the same company's work.
• Hard user caps are becoming a last-resort control, while matei_zaharia's post describes AI gateways that analyze usage, route models, and push budgets down to individual tasks.
• Gateway controls are reaching coding clients: ClaudeCodeLog's changelog says Claude Code now shows the gateway cap, reset time, and operator message in usage warnings.

The Databricks cost post that Yuchenj_UW shared is unusually operational: it names Omnigent, Unity AI Gateway, and an internal coding-agent benchmark built from real merged PRs. Cursor Router says production routing over live requests can cut costs, while Databricks' Unity AI Gateway docs put models, agents, MCP servers, and tools behind the same control plane.

Databricks' useful distinction is the efficiency frontier, the set of models that deliver the best quality for a given task cost, not the highest absolute intelligence. The company's cost post says that frontier is moving faster than the intelligence frontier because new cheaper models arrive almost weekly.

The frontier points in Yuchenj_UW's chart run from low to high cost:

• GPT-5.6 Luna, about $0.50 per task and 65% average pass rate.
• GPT-5.6 Terra, about $0.75 per task and 67.5%.
• GLM 5.2, about $0.90 per task and 73%.
• GPT-5.6 Sol, about $1.60 per task and 75%.
• Opus 4.8, about $3.60 per task and 79%.

Yuchenj_UW's four operational claims are the whole story in miniature:

1. The efficiency frontier matters.
2. Opus 5.0 regressed on cost versus 4.8 in Databricks' tests.
3. Hard budgets are the wrong primitive for the highest-leverage AI users.
4. Routing, harnesses, evals, and a mix of open and proprietary models change the economics.

What matters most: the unit of analysis has moved from dollars per million tokens to dollars per accepted engineering task.

Databricks' benchmark writeup says the tasks came from actual engineering work on a multi-million-line codebase spanning Python, Go, TypeScript, Scala, Rust, Java, Bazel, Protobuf, and other internal surfaces.

The benchmark construction is more interesting than another model leaderboard:

• Databricks pulled from recent human-written PRs, filtering out bot commits, service accounts, fully AI-generated changes, and generated code.
• Tasks had associated tests, self-contained changes, and representative coverage across backend, frontend, systems code, contracts, and configs.
• The prompt removed the original solution while preserving the intended outcome and constraints.
• Correctness was evaluated with held-out tests, not an LLM judge.
• Databricks sealed git history after early traces showed agents could recover the original merged solution from the worktree.

The results explain why Databricks is talking about gateways instead of model subscriptions. GLM 5.2 landed statistically tied with Opus 4.8 on quality while costing $1.28 per task versus Opus 4.8 at $1.94, according to the benchmark post.

Token pricing misled in the opposite direction. Sonnet 5 was about 1.7x cheaper per token than Opus 4.8, but cost $2.09 per task versus Opus 4.8 at $1.94 while scoring six points lower, because it consumed 1.9x more tokens on Databricks' tasks.

In matei_zaharia's post, AI tokens are framed as another software-engineering resource, with Databricks routing usage through an AI Gateway for centralized analysis, smart routing, and per-task budgets.

Databricks' cost post splits routing into three patterns:

• Request-level routing: a stateful proxy picks the lowest-cost model capable of answering an inference request, while accounting for cache state.
• Task-level routing: a meta-harness dispatches an end-to-end task to the right model or harness based on complexity.
• Escalation and delegation: one model runs the loop while another handles either cheap worker tasks or expensive escalations.

Databricks says its AI Gateway Smart Router reduced average task cost by more than 30% while roughly matching the most expensive model in the working set. Cursor reported the same basic shape from the product side: cursor_ai's router post says its router learns from millions of in-product interactions, routes routine work away from frontier prices, and maps GPT-5.6 Sol to planning, Opus 5 to execution, and Fable 5 to debugging and visual implementation.

Databricks found that the same model at the same thinking effort could cost more than 2x as much through different harnesses, mostly because some harnesses feed the model far more context per turn. Pi sent about 3x less context per turn in Databricks' tests, according to the benchmark writeup.

Composio's DeepSeek V4 Flash eval makes the same point with a clean table:

• Most tasks passed: Oh My Pi, 17 of 30.
• Fastest median task time: Claude Code, 123 seconds.
• Lowest cost per successful task: OpenCode, $0.073.
• Same pass count, different economics: Claude Code and Codex both passed 16 of 30, but Claude Code cost $0.195 per success versus Codex at $0.081.

GitHub is making the same harness argument from another angle. pierceboggan's Copilot post says GitHub optimizes its Copilot agentic harness across model providers, tunes prompts before launches, runs offline evals and A/B tests, and dynamically builds prompts and tool schemas by provider.

Databricks says hard budget cutoffs are a last resort because high-spend developers can also be the developers getting the most leverage from AI. Its preferred sequence is visibility, spend gates, downshifting, and only then suspension.

The mechanics are concrete:

• Visibility: near-real-time spend feedback across tools, often with model-mix suggestions.
• Spend gates: warnings or approvals when spend crosses thresholds.
• Downshifting: moving a developer to a lower-cost model after a gate, rather than cutting access.
• Suspension: temporary full cutoff in the limit case.

Claude Code 2.1.225 now exposes that infrastructure in the client: ClaudeCodeLog's changelog says usage warnings can show the gateway spend cap, reset time, and operator message. LangChain is shipping the multi-tenant version of the same control surface, where LangChain's customer controls show a monthly customer spend cap scoped by an X-Gateway-Customer-ID header.

Databricks' fourth lever is reducing token overhead, because the user's typed request is often a negligible share of the context sent to the model. The cost post names compaction, less chatty harnesses, lower tool verbosity, smaller tasks, and cache tuning as active techniques.

Databricks says simple harness and caching changes cut generated tokens and related costs by almost 50% with no observed quality degradation. _avichawla's thread pushes the same pattern further, claiming Anthropic moved tool definitions and results out of the prompt path after finding more than 50,000 tokens of overhead before the agent read the user request, with one workflow dropping from 150,000 tokens to 2,000.

Web search is the stubborn case in that thread. Raw HTML brings nav, cookie banners, and footers into context, and _avichawla's reply says every fetched page stays in the conversation and gets resent on later turns.

Cache behavior is already visible across clients. thdxr's cache table put Claude Code / CLI at an 89.31% cache hit ratio on recent DeepSeek traffic, behind ZCode at 98.60%, OpenCode V2 at 97.86%, Cursor at 97.84%, Kimi Code CLI at 97.64%, Pi at 96.98%, and Codex at 96.66%.

Gateway controls are becoming observable events, not just policy files. LangChain's LangSmith trace shows a 402 error from a gateway spend-limit policy inside a LangSmith trace, with turn count, latency, and cost visible beside the blocked request.

Vercel is moving the same boundary into OpenTelemetry. vercel_dev's trace post says AI Gateway now emits an OTel trace for every request, exportable through Vercel Drains to OTLP/HTTP endpoints.

Vercel's trace docs say those traces capture routing decisions, fallbacks, retry attempts, token usage, cost, time to first token, request duration, status, project, deployment, API key, environment, and custom tags. The same docs say prompt and completion content are excluded, and exported traces cost $0.05 per 1,000 traces delivered to each drain.