Kimi K3 launches across vLLM, SGLang, Ollama, and OpenRouter

• Kimi K3's open-weight launch doubled as a runtime rollout: Moonshot released a 2.8T MoE with 1M context and native vision in Moonshot's launch post, while vLLM's thread and SGLang's thread had day-zero serving paths ready.
• The serving work centered on hybrid state: vLLM's thread says KDA uses fixed-size recurrent state instead of a growing KV cache, and SGLang's thread says K3 interleaves 69 KDA layers with 24 MLA layers.
• Speculation made the giant model feel less giant: vLLM's thread reported 118 to 370 tok/s with DSpark, SGLang's thread reported 423 tok/s on gsm8k, and Modal called K3 the most capable open model it had worked with.
• Access spread across provider and agent surfaces on day zero, including Vercel AI Gateway, OpenRouter, Cursor, Devin, Ollama, and Factory's Droid.
• Early hosted pricing converged around $3/M input and $15/M output: jaminball's pricing note said Baseten and Fireworks matched Moonshot, while mervenoyann's Hugging Face post showed Together at the same list price.

The vLLM release post is basically a cache-manager story: recurrent KDA state, paged MLA KV, P/D disaggregation, DSpark, and Kimi-specific parsers. The SGLang writeup goes deeper on 54 MB KDA state blocks, 27 KB MLA KV blocks, unified memory, ReplaySSM, and a 32x draft-window memory cut. Modal's launch post says its DFlash run used 32 B300 nodes, while Vercel's changelog exposes the product knobs: speed: 'fast', US inference region, and per-request ZDR.

Moonshot's Hugging Face model card describes Kimi K3 as an open-weight, native multimodal agentic model with Kimi Delta Attention, Attention Residuals, and a 1M-token context window.

The load-bearing specs:

• 2.8T total parameters, with 104B activated per token, according to stochasticchasm's report screenshots.
• 16 of 896 routed experts active per token, with Stable LatentMoE, according to the same technical-report excerpts.
• Hybrid KDA and Gated MLA attention, plus native vision through MoonViT-V2, according to vLLM's architecture thread.
• Roughly 2.5x scaling-efficiency gain over Kimi K2, according to Moonshot's launch post.
• Artificial Analysis placed K3 at 57 on its Intelligence Index, the top open-weight score in Artificial Analysis's launch-day chart.

The rollout hit serving engines, inference providers, and coding products within hours.

Serving engines

• vLLM: K3 support landed with prefix caching, tool calling, reasoning parsing, structured output, P/D disaggregation, and DSpark support in vLLM's thread.
• SGLang: SGLang's thread reported 423 tok/s, fused KDA decode kernels, DP attention, DSpark, P/D disaggregation, and KDA-aware prefix caching.
• Modal: Modal shipped K3 with a custom DFlash speculator, token-based Shared API access, and $30 free monthly compute.

Managed APIs

• Baseten and Moonshot's Fireworks post both announced day-zero API availability.
• Together AI positioned K3 for long-running agent workflows across code, tools, vision, and research.
• DigitalOcean added K3 to Serverless Inference, while Moonshot's Nebius post announced Nebius availability.
• OpenRouter launched K3 with third-party providers and said K3 Fast variants were coming from Wafer, Fireworks, and others.

Local-ish launchers

• Ollama exposed kimi-k3:cloud and the Claude Code command ollama launch claude --model kimi-k3:cloud.
• OpenCode put K3 into OpenCode Zen.

K3's architecture made prefix caching the hard part.

The vLLM post says most K3 layers hold fixed-size KDA recurrent state, while periodic full-attention layers still keep paged KV. vLLM's new hybrid cache manager stores both under one scheduler and decouples large physical KDA state blocks from fine-grained prefix matching.

The SGLang post gives the cleaner mental model:

• MLA KV is append-only, so prefix sharing works like ordinary radix-tree caching.
• KDA state is overwritten in place at every token, so it has to be checkpointed, copied, replayed, or donated safely.
• Under TP=8, one KDA state block is about 54 MB per request.
• One MLA KV block is about 27 KB per token across 24 MLA layers.
• SGLang's unified memory design lets KDA states grow from one end of a shared pool and MLA KV blocks from the other.

The fastest K3 numbers came from draft models and verification paths, not raw target decode.

• vLLM reported 118 tok/s without speculative decoding and 370 tok/s with DSpark, a 3.14x speedup, in the vLLM release post.
• SGLang reported about 113 tok/s before speculation and about 423 tok/s with DSpark; its launch post says ReplaySSM cuts draft-window memory by about 32x by replaying raw inputs instead of snapshotting state per step.
• Modal reported 460 tok/s on release day in its K3 launch post, with 360% faster interactivity and 88% higher throughput on agentic workloads.

Moonshot released more than weights and a report. It shipped model-adjacent infrastructure for attention, MoE communication, and agent environments.

• FlashKDA: Moonshot's FlashKDA post says the CUTLASS-based KDA kernels deliver 1.72x to 2.22x prefill speedup over flash-linear-attention on H20 and work as a drop-in backend.
• AgentENV: Moonshot's AgentENV post describes a distributed system for agent environments with fast snapshot, resume, and fork support for parallel RL workflows.
• MoonEP: Moonshot's MoonEP post describes a communication library for distributed MoE workloads, aimed at expert-parallel communication overhead.

Open weights turned the location of inference into a product choice.

Vercel's AI Gateway changelog says K3 and K3 Fast are available from US-based providers including Baseten and Fireworks, with Zero Data Retention for both. The same page says AI Gateway can route across providers for failover and throughput, request fast mode through speed, and restrict inference to US data centers with inferenceRegion at about 10% higher regional pricing.

Cursor shipped the same compliance framing: Cursor said K3 was available inside Cursor on US-based inference through Fireworks, Together, and Baseten, with ZDR support.

European routing also showed up early. teortaxesTex's Nebius screenshot showed K3 in eu-north1 with FP4 quantization, tool calling, reasoning, 1,024K context, and a listed speed of 120 tok/s.

The first hosted market did not immediately undercut Moonshot's own list price.

The first public prices clustered around:

• Input: $3.00 per 1M tokens.
• Cached input: $0.30 per 1M tokens on several provider cards.
• Output: $15.00 per 1M tokens.

Together's Hugging Face Inference Providers card showed $3/M input, $15/M output, and 54 TPS in mervenoyann's post. Ollama's model page shows the same $3, $0.30, and $15 price points for cloud usage.

Fast tiers introduced a premium instead of a discount. Vercel's changelog says Kimi K3 Fast costs about 50% more than the base model, while Wafer promoted a K3 Fast endpoint for later in the week.

Early speed screenshots were all over the place. bridgemindai's provider table showed Fireworks at 45 tps, DigitalOcean at 72 tps, Nebius at 34 tps, and Moonshot at 15 tps, while Cedric Chee said OpenRouter jumped from 19 to 49 tok/s.

K3 landed where engineers actually spend tokens: coding agents and IDE-adjacent harnesses.

• Devin: Cognition said K3 was the first open-source model it tested that approached frontier-level performance on FrontierCode 1.1, with 58.2% on FrontierCode 1.1 Extended and a 63.6% pass rate.
• Factory Droid: Factory added K3 at 50% off for two weeks; Factory's testing note said it stood out at repo context, tool use, current information, and real failure diagnosis.
• Cursor: Cursor said K3 scored close to frontier on CursorBench.
• Ollama: Ollama exposed K3 for Claude Code and said Pro or Max subscriptions were required, with extra usage credits.
• Julius: Julius said K3 was its first open-weight model and called out large document sets, deep research, and report/dashboard/PowerPoint generation.
• Morph: Morph announced K3 through API and OpenRouter.

Harness compatibility was not automatic. Hamel Husain's Codex Desktop note called coding harnesses sticky, and his follow-up said K3 through Codex Desktop got weird and sent JSON messages.

The weights are public. The hardware bar is still datacenter-shaped.

Simon Willison's Kimi K3 note sized the Hugging Face weights at 1.56 TB. The SkyPilot post describes K3 self-hosting as multi-node inference across owned GPUs, hyperscalers, neoclouds, or on-prem clusters, with autoscaling, KV-aware routing, P/D disaggregation, and observability wrapped into SkyPilot Endpoints.

Baseten's day-zero API writeup says it chose NVIDIA GB300 NVL72 systems for K3 and worked with vLLM and SGLang pre-release builds. one LocalLLaMA post put the hobbyist version bluntly: even 2 to 4 RTX 6000 Blackwell workstations made native local serving feel virtually impossible.

Kimi K3's license is permissive for ordinary use and more restrictive for large model-service businesses.

The Kimi K3 License grants rights to use, copy, modify, distribute, fine-tune, deploy, and sell products built with the software, then adds two scale triggers:

• A Model-as-a-Service business with more than $20M in aggregate revenue over any consecutive 12 months needs a separate agreement with Moonshot before commercial use.
• A commercial product or service with more than 100M monthly active users, or more than $20M in monthly revenue, must prominently display "Kimi K3" in the UI.
• Those two requirements do not apply to internal use or use through Moonshot's official products or certified inference partners.

Artificial Analysis labeled the release "Commercial Use Restricted" in its K3 license note. Simon Willison's Kimi K3 writeup noted that Moonshot consistently calls the model open weight rather than open source.