Ray Serve LLM

Ray Serve LLM is a Ray Serve-based software product for deploying and scaling LLM inference applications. It fits the evidence context because it is about LLM serving infrastructure, including routing/executor work and serving-throughput improvements rather than a model release.