AA-AgentPerf: AMD MI355X delivers 15–40% more throughput per MW than B200

• MI355X delivers 15–40% more throughput per MW than B200 on DeepSeek V4 Pro at 35–75 tok/s per user, according to the DeepSeek comparison. B200 pulls ahead above roughly 80 tok/s.
• DeepSeek V4.1 Flash nearly matches Blackwell for one agent, but AMD falls behind as load rises in the published results.
• AMD lands between Hopper and Blackwell on GLM, while Kimi K3 delivers roughly a third of B300’s throughput per MW at 50 tok/s per user.
• AMD’s tests use one eight-GPU node and development software; some NVIDIA comparisons use multiple nodes, as described in the announcement.

The throughput metric counts input and output tokens, including KV-cache hits. Its speed axis excludes time to first token, even on replayed contexts exceeding 100K tokens.

AMD’s strongest result has a clear crossover:

• 35–75 tok/s per user: MI355X serves approximately 15–40% more throughput per MW than B200.
• Peak throughput: MI355X reaches about 94M tokens/s per MW at 33 tok/s per user. B200 reaches that throughput at only 13 tok/s per user.
• Above approximately 80 tok/s per user: B200 takes the lead.
• One agent: MI355X reaches 240 tok/s, versus 367 on B200.

The crossover matters more than the peak. Artificial Analysis suggests memory capacity helps: MI355X has 288GB of HBM3E per GPU, versus the 180GB it reports for B200, serving a 1.6T-parameter model.

DeepSeek V4.1 Flash comes much closer to Blackwell’s single-agent speed, but the throughput gap opens under concurrent load:

• One agent: MI355X reaches 647 tok/s, versus 652 on B300 and 669 on B200.
• At 100 tok/s per user: MI355X delivers approximately 128M tokens/s per MW, versus 193M on B200 and 220M on B300.

GLM-5.2/5.3 puts MI355X between NVIDIA’s Hopper and Blackwell generations:

• At 100 tok/s per user: approximately 26M tokens/s per MW on MI355X, versus 16M on H200, 42M on B200 and 44M on B300.
• One agent: 228 tok/s on MI355X, matching H200, versus 312 on B200 and 333 on B300.

Kimi K3 is the largest model tested, with 2.8T total parameters and 104B active parameters. Its B300 comparison uses NVIDIA’s multi-node disaggregated configurations, against AMD’s single node.

• At 50 tok/s per user: MI355X delivers approximately 8M tokens/s per MW, versus 24M on B300.
• Peak AMD throughput: approximately 13M tokens/s per MW at 12 tok/s per user.
• One agent: 152 tok/s on MI355X, versus 235 on B300.

AA-AgentPerf publishes a Pareto frontier, according to its methodology. Each point represents one serving configuration at one concurrency level; the frontier retains points unbeaten on both throughput and speed.

The measurement details put boundaries around the comparisons:

• Speed statistic: Charts default to the 25th percentile across requests, the speed three quarters of requests exceeded. Counts use each tested model’s native tokenizer.
• Power denominator: Accelerator power is measured under load, covering GPU die and HBM. The figures use measured draw rather than rated TDP and exclude the rest of the server and facility.
• Tool waits: The replay inserts fixed per-message delays sampled from real tool-duration distributions. They range from under 0.1 seconds to five seconds, with a median around one second.
• Test duration: Each load phase requires at least ten minutes of steady-state measurement, 30 completed trajectories overall and three completed trajectories per simulated agent.
• Tuning access: Participants receive 500 trajectories containing 18,997 prompts. The full test dataset remains private to prevent benchmark-targeted optimization.

All AMD runs use ROCm nightly and development builds of vLLM and SGLang, according to the announcement. The published curves therefore describe those serving stacks as well as the accelerator hardware.

For DeepSeek V4 Pro, B300’s disaggregated deployment uses two eight-GPU nodes and reaches approximately 125M tokens/s per MW at 116 tok/s per user in the comparison. That configuration sits above both single-node MI355X and B200 results.

Artificial Analysis publishes rolling updates on its live leaderboard.