Agent runtime infrastructure Model serving Inference Optimization KV Cache

ThunderAgent

KV-cache scheduling for agent workflows

A scheduler for whole agent workflows that manages KV-cache-aware execution during tool calls, intended to reduce cache thrashing in agentic inference.

Recent stories

1 linked story

releasePRIMARY2026-07-29

Together releases ThunderAgent for KV-cache scheduling in agent workflows

Together released ThunderAgent to schedule whole agent workflows instead of isolated requests during tool calls. Together reports up to 2.5x throughput and about 10x lower P50 latency under high concurrency.