GuideLLM
SLO-aware Benchmarking and Evaluation Platform for Optimizing Real-World LLM Inference
GuideLLM is an open source toolkit/platform for benchmarking and evaluating LLM deployment performance under production-like workloads, simulating traffic and reporting throughput, latency, TTFT, ITL, and other metrics for OpenAI-compatible and vLLM-native servers.
Recent stories
0 linked stories
No linked stories yet.