Kimi K3 benchmarks last at 53/67 in AlphaSignal repair harness

• Kimi K3 shipped as a 2.8T, 1M-context, native-vision model with open weights promised by July 27, according to Kimi_Moonshot's launch post.
• The coding story split hard by workload: K3 topped Frontend Code Arena, while AlphaSignalAI's repair run put it last at 53/67 on held-out bug fixes.
• The cost story is workload-dependent: ArtificialAnlys's index post measured $0.94 per Intelligence Index task, while Cline's real-bug run found Kimi cheaper than Fable but much slower.
• The main operational gotcha is state handling, because AlphaSignalAI's follow-up says K3 needs full thinking history each turn and Kimi's own docs currently expose only max reasoning.

Moonshot's official tech blog says K3 built a MiniTriton compiler, designed a chip in a 48-hour autonomous run, and used 2,800+ web fetches for an ASIC industry report. The API docs hide the harness contract: multi-turn callers must append the complete assistant message, not just content. Simon Willison's pelican test cost 25 cents because K3 spent 13,241 reasoning tokens on one SVG.

• 2.8T parameters, 1M context, native multimodal input, and open weights promised by July 27, per Kimi_Moonshot's launch post.
• Day-one surfaces: Kimi.com, Kimi Work, Kimi Code, and the Kimi API, according to Kimi_Moonshot's launch post.
• API pricing: $3.00 per MTok input, $15.00 per MTok output, and $0.30 per MTok cached input, according to ArtificialAnlys's model summary.
• API shape: OpenAI SDK compatibility, base_url="https://api.moonshot.ai/v1", and model="kimi-k3", per the Kimi API quickstart.
• Reasoning effort: max only at launch, with lower modes planned later, according to Kimi_Moonshot's launch post.

First-party

• Scaling efficiency: K2 baseline 1.0x -> K3 2.5x, +1.5x, according to Kimi_Moonshot's architecture note.
• Million-token decoding: baseline 1.0x -> KDA up to 6.3x, +5.3x, according to Kimi_Moonshot's launch post.
• AttnRes kernel optimization: 283.6 ms -> 114.4 ms forward plus backward, -169.2 ms, according to Kimi_Moonshot's kernel case study.

Third-party evaluators

• Artificial Analysis Intelligence Index: K2.6 44 -> K3 57, +13 points, according to ArtificialAnlys's index update.
• GDPval-AA v2: K2.6 1190 -> K3 1668, +478 Elo, according to ArtificialAnlys's model summary.
• AA-Briefcase: K2.6 815 -> K3 1547, +732 Elo, according to ArtificialAnlys's model summary.
• Frontend Code Arena rank: Kimi-k2.6 #18 -> Kimi-K3 #1, +17 places, according to the Frontend Code Arena post.
• DeepSWE: K2.7-code 31 -> K3 69, +38 points, according to rohanpaul_ai's DeepSWE summary.
• Extended NYT Connections: K2.6 91.4 -> K3 95.8, +4.4 points, according to LechMazur's benchmark note.

Customer-reported

• Cline repo bug cost: Fable $2.13 -> Kimi $0.92, -57%, according to Cline's real-bug run.
• Cline repo bug latency: Fable 3.5 min -> Kimi 12 min, +8.5 min, according to Cline's real-bug run.
• Autonomous legal work: Fable 5 14.2% -> Kimi K3 26.7%, +12.5 points, according to rohanpaul_ai's legal-work summary.
• Private cyber cost: GPT-5.6 Sol 7x Kimi's run cost -> Kimi 1x runner-up cost, ~7x cheaper than Sol, according to cramforce's cyber benchmark.

AlphaSignalAI got the loudest counter-result: K3 finished last in a repair harness even while other boards ranked it near the top.

• Signaldesk repair: Sol 100% (70/70) -> K3 79% (53/67), -21 points, according to AlphaSignalAI's repair run.
• Signaldesk latency: Grok 46s average -> K3 702s average, +656s, according to AlphaSignalAI's repair run.
• Signaldesk method: 13 planted-bug tasks, seven models, network-off Docker sandboxes, held-out tests at score time, and no test-file edits, according to AlphaSignalAI's method note.
• AA-Omniscience hallucination rate: K2.6 39% -> K3 51%, +12 points, according to ArtificialAnlys's hallucination note.
• FrontierMath Tier 4: best US models from seven months earlier 46% -> K3 39%, -7 points, according to scaling01's FrontierMath note.
• LiveBench: GPT-5.4-xhigh ahead -> K3 behind it, negative rank movement versus that reference, according to scaling01's LiveBench note.

Ofir Press, one of the ProgramBench authors, said Moonshot used an average implementation metric rather than the benchmark's recommended fully-implemented-program metric. That can turn "90% of every program" into a high score even when zero programs fully pass, OfirPress's ProgramBench caveat argued.

Moonshot also acknowledged a product gap: scaling01's screenshot of the Kimi blog quoted Kimi's own line that K3 still has a noticeable user-experience gap versus Claude Fable 5 and GPT-5.6 Sol.

K3's model card is really a systems story.

• Attention stack: Kimi Delta Attention plus Attention Residuals, according to Kimi_Moonshot's architecture note.
• Sparse MoE: Stable LatentMoE activates 16 of 896 experts, according to Kimi_Moonshot's architecture note.
• Quantization: the official tech blog says K3 uses MXFP4 weights and MXFP8 activations from SFT onward.
• Serving topology: Moonshot recommends supernode configurations with 64 or more accelerators, a detail teortaxesTex highlighted from the blog.
• Prefix caching: vLLM's project account said Moonshot contributed a KDA prefix-caching implementation and that vLLM would support K3 on day-zero release.
• Harness contract: AlphaSignalAI's reasoning-content reply and the Kimi API quickstart both point at the same issue, K3 quality depends on preserving the full thinking history across turns.

Hands-on reports mostly agreed on the shape: strong output, expensive thinking, uneven control.

• K3 was "very, very slow" in a firstmate session, burned a third of a 5-hour plan in a few prompts, and missed system-prompt instructions that other frontier models followed, according to kunchenguid's firstmate report.
• A 1.5 MB FrankenGraphDB plan review produced findings that Fable later judged substantially correct, though about a third of remedies needed correction, according to doodlestein's meta-review thread.
• A simple TUI hover-color fix went to Sol for $0.30, while Kimi reached $1.00 and started reading the database before being interrupted, according to thdxr's task report.
• Cline's traces suggested Kimi is RL-trained to spend more tokens thinking and verifying before completion, according to Cline's real-bug run.
• OpenRouter users saw the operational side immediately: one rate-limit complaint could not get two successful requests in a row, and OpenRouter's K3 page warned that upstream capacity was limited and frequent 429s were possible.
• Simon Willison's pelican writeup found the same always-on-max pattern in miniature: 16,658 output tokens for a toy SVG, including 13,241 reasoning tokens.

K3 became a day-one integration race.

• Kimi's own surfaces shipped first: Kimi.com, Kimi Work, Kimi Code, and API access, according to Kimi_Moonshot's launch post.
• Vercel AI Gateway added moonshotai/kimi-k3, and Vercel's Kimi Code plugin note said Kimi Code could draw on Vercel platform knowledge for Next.js, AI SDK, and Vercel Functions.
• Hermes Agent added Kimi through Nous Portal, Kimi Direct, and OpenRouter, according to Teknium's Hermes post.
• Cline put K3 into ClinePass, with a $1.99 promo for users installing via npm i -g cline, according to Cline's subscription post.
• OpenCode made K3 available to OpenCode Go users before negotiating a discount, according to OpenCode's rollout note.
• Pi added K3 with native multimodal support, long-horizon self-evolving workflows, and faster million-token context, according to Pi's integration post.
• AI/ML API put K3 and Claude Fable 5 behind the same key, SDK, billing, and OpenAI-compatible endpoint, according to TestingCatalog's AI/ML API post.