Microsoft Copilot traces report 87% of LLM calls came from agents

• Copilot traces turned agent fanout into a production number: rohanpaul_ai's Copilot summary says 87% of LLM calls came from the agent, not the user.
• The cache data makes request-level serving look underspecified: rohanpaul_ai's Copilot summary says KV cache hit rate rose to 92-94% after the third call inside a turn, then fell to 55% at a same-model turn boundary and 8% after a model switch.
• Token demand is clustering around agentic work: theo's compute-crisis post framed the capacity problem around engineers spending thousands per day, while rohanpaul_ai's Altman quote put OpenAI's current top user in the hundreds of billions of tokens per month.
• Cheap inference expands workload shape: thdxr's cheap-inference thread pointed at agents reading every console.log, every video frame, and every heartbeat as prices fall.
• Coding is the leading edge because it maps cleanly to uninterrupted digital work, while levie's workflow thread says sales, legal, medicine, and life sciences need workflow rewiring before background agents drive comparable volume.

The Microsoft Research page gives the official abstract, and the arXiv PDF has the systems tables. The Together AI Cursor writeup explains why in-editor agents turn inference into a tail-latency product problem. Databricks' cost post treats AI tokens like a software engineering resource routed through gateways. The Agents in the Wild paper moves the same deployment problem into pharma and finance.

The paper's dataset is the useful part: sampled GitHub Copilot coding-agent telemetry from the first week of June 2026, covering 3.2M users, 13.5M sessions, 95.1M turns, 760.5M LLM calls, 774.7M tool calls, 44.9T prompt tokens, and 39.3B completion tokens, according to the arXiv PDF.

The paper's execution hierarchy is simple:

• Session: a coding agent session lifetime.
• Turn: one user prompt plus the agent's full autonomous response chain.
• Step: one LLM invocation or tool call.

The per-session table gives the shape behind the 87% figure:

• Median session: 3 user turns, 15 LLM calls, 13 tool invocations, 4.2 minutes.
• Mean session: 6.1 user turns, 40.6 LLM calls, 43.6 tool invocations, 62.6 minutes.
• Median turn: 4.5 LLM calls, 4 tool invocations, 227.6K prompt tokens, 1.9K completion tokens.
• Mean turn: 6.6 LLM calls, 7.6 tool invocations, 582.5K prompt tokens, 4.0K completion tokens.

The authors frame the serving mismatch directly: chat infrastructure schedules short, independent requests; coding agents create chains whose later calls depend on tool outputs from earlier calls.

KV cache is the section to bookmark. It converts "agents are chatty" into a resource-lifecycle problem.

The arXiv PDF reports:

• Within a turn, prefix caching accounts for about 90% of tokens.
• After the first LLM call, average cache hit rate rises from about 45% to 92-94% by the third call.
• At a same-model turn boundary, cache hit rate falls to 55%.
• After a model switch, cache hit rate falls to 8%.
• Context compaction appears in 7.8% of sessions, affects 44% of total tokens, drops more than 70% of prompt tokens, and resets cache state as severely as a model switch.
• Median KV-cache idle time is 1.2 seconds within a turn and 172 seconds across turns.
• A lightweight predictor using turn-level and session-level features captures 86-90% of total idle time.

The authors' systems claim is narrow and strong: KV cache becomes a session-aware schedulable resource, not a per-request optimization.

Theo's joke landed because the Copilot table made the same point with production traces: prompt counts are a weak proxy for compute once agents start looping.

Sam Altman's YC Startup School quote put the curve in consumer-scale terms: the world token leader at OpenAI used about 100,000 tokens a month six and a half years ago, the global average is now around 100,000 tokens a month, and the current leader uses hundreds of billions, according to rohanpaul_ai's Altman quote. The source video came through rohanpaul_ai's video link.

The enterprise version is already visible. Databricks' cost post says agentic coding improved velocity metrics but created exponentially growing costs, then lays out AI Gateway routing, task-level routing, and internal benchmarks as the control plane.

Matei Zaharia described the same move as making AI tokens "another resource to optimize in software engineering," routed through an AI Gateway for centralized analysis, budgets, and surprise prevention matei_zaharia's gateway note.

Cursor's workload adds a second constraint to the Copilot trace: agents inside an editor live inside the developer's active feedback loop.

The Together AI Cursor writeup says the stack was tuned across NVIDIA Blackwell GB200/B200 hardware, ARM hosts, kernels, FP4/TensorRT quantization, and serving software to keep in-editor agents responsive under production load.

That is the other half of agent capacity planning: the system has to carry fanout and preserve interaction latency while code context keeps moving.

Cheap inference changes what gets counted as reasonable input. thdxr's examples were deliberately mundane: read every console.log, process every frame of video, check every heartbeat.

OpenRouter reported a clean Jevons datapoint: GPT-5.6 Luna token volume grew 10x after its price dropped 10x, with the week projected above 10x OpenRouter's Luna volume post. Together AI's DeepSWE comparison found two DeepSeek V4 Flash attempts solved more tasks than one GPT-5.6 Luna attempt at roughly one-third the cost Together's DeepSWE comparison.

The cascade version is closer to production routing: Together's DeepSeek-first post says DeepSeek-first plus test-suite verification solved more tasks than Luna alone at 37% lower cost per task.

The same fanout pattern leaves the IDE. yacineMTB described one search-research query to personal infrastructure sending thousands of requests to systems built for human-scale access.

John Rush's version was interface-level: he said he rarely opens URLs because an AI chatbot acts as the proxy, and argued URLs may end up serving raw text while SaaS frontends become queryable through SQL johnrush's web-proxy post. Gergely Orosz connected that to booking systems, saying gym and event booking gets harder when agents swarm those flows GergelyOrosz's booking-systems note.

The security read is the same resource curve with adversarial incentives. Security through obscurity gets weaker when agent swarms can enumerate long-tail targets, according to nptacek's Shodan warning.

levie gave the cleanest boundary for why coding is first: economic value in software work maps directly to digital output, and task size can grow inside one uninterrupted session.

His contrast list is the part that matters:

• Sales work needs customer feedback before additional account work.
• Legal work needs client interaction.
• Medical work needs patient interaction.
• Enterprise agents need systems of record, data cleanup, change management, and redesigned workflows.

The Agents in the Wild paper names the deployment machinery around that shift: reasoning and planning beyond benchmarks, multi-agent coordination, verification pipelines, fallback mechanisms, and human-in-the-loop supervision. Its case studies are pharmaceutical discovery and financial systems, which makes the Copilot trace feel less like a coding-only artifact and more like the first measured version of a broader production workload.