TokenSpeed

TokenSpeed is an open-source, production-oriented LLM inference engine for low-latency OpenAI-compatible serving of agentic workloads, with a compiler-backed modeling layer, scheduler, KV-cache resource management, and pluggable kernel system including optimized MLA kernels.