SemiAnalysis reports TPUv7 delivers 1.67× GB300 performance per dollar on AgentX
SemiAnalysis reports TPUv7 FP8 delivers 1.67 times GB300 NVL72 FP4 performance per dollar on AgentX. Google’s four-chip vLLM submission leads across most tested operating points, with a low-interactivity exception.
TL;DR
- Google's first TPUv7 AgentX submission delivered 1.67× GB300 NVL72 FP4 performance per dollar, according to the announcement from SemiAnalysis.
- Four TPUv7 chips running FP8 led across most of the tested cost-performance frontier; GB300 FP4 retained the low-interactivity lead in the comparison.
- AgentX exercises growing conversations, long contexts, prefix reuse and subagent bursts, as SemiAnalysis explains in its workload breakdown.
The telemetry catalog identifies the model as Qwen3.5 397B-A17B and records 11 TPU operating points spanning 4–32 concurrent agent clients, dated October 13. The replay methodology removes the original prompts and code while preserving their prefix relationships through session-scoped hashes in 64-token blocks.
Four-chip FP8 result
Google submitted TPUv7 Ironwood results using the upcoming TorchTPU-backed vLLM stack. SemiAnalysis reports the headline 1.67× advantage against GB300 NVL72 running FP4.
The Qwen3.5 dashboard and telemetry catalog identify these serving configurations:
| System | Precision | Serving stack |
| --- | --- | --- |
| TPUv7 Ironwood, four chips | FP8 | TorchTPU / vLLM |
| GB300 NVL72 | FP4 | Dynamo / SGLang |
The reported advantage uses performance per total cost of ownership. It compares complete serving configurations with different precisions and software stacks, so the multiplier describes this deployment comparison.
Low-interactivity crossover
GB300 NVL72 FP4 still wins at low interactivity. TPUv7 FP8 remains ahead of GB300 FP8 in that region, according to SemiAnalysis.
AgentX defines p90 interactivity as per-client generation speed derived from p90 inter-token latency in its metrics guide. Higher concurrency can improve batching and aggregate throughput while slowing individual streams.
The crossover deserves equal billing with the headline multiplier. SemiAnalysis describes TPU leadership across most of the frontier, leaving the winning configuration dependent on the required response speed.
AgentX's four traffic patterns
AgentX represents four properties of agent sessions:
- Multi-turn: Tens or hundreds of user and assistant interactions accumulate within a session.
- Long context: System prompts, tool definitions and conversation history grow the input.
- High prefix reuse: Each turn carries forward the preceding conversation, including the previous output.
- Subagent bursts: Short-lived branches start with fresh context and create bursty KV-cache demand.
The replay runs in a closed loop: a client sends its next eligible request after its dependencies finish. Subagent fan-out can therefore put more HTTP requests in flight than the configured client concurrency, according to the methodology guide.
AgentX measures serving latency, throughput, cache behavior and cost. Task correctness requires a separate evaluation.
Native PyTorch through TorchTPU
TorchTPU exposes TPUs as native PyTorch devices through the PrivateUse1 extension point. PyTorch's dispatcher routes ATen operations directly to the TPU backend, while compiled graphs pass through StableHLO and XLA, as SemiAnalysis describes in its backend documentation.
The previous TorchAX path translated PyTorch operations into JAX. SemiAnalysis's September technical analysis described TorchTPU as the next backend for external vLLM and SGLang serving; this submission exercises that path on multi-turn agent traffic.