Thinking Machines releases Inkling-Small 276B open-weight MoE

• Thinking Machines shipped real open weights: the release thread says Inkling-Small has 276B total parameters, 12B active parameters, Tinker fine-tuning, and text, image, and audio chat in Tinker Playground.
• The compression story is unusually strong: Artificial Analysis' benchmark thread scored Inkling Small 40 versus Inkling 41, while ARC Prize put it ahead of Inkling on ARC-AGI-1 and ARC-AGI-2 at lower per-task cost.
• The wins are uneven: Artificial Analysis' breakdown found worse AA-Omniscience and τ³-Banking results than Inkling, even as it matched or beat the larger model on several coding and reasoning evals.
• Day-zero deployment was broad, with support announced by vLLM, SGLang, Modal, Vercel, Baseten, and Together AI.
• “Small” is relative: UnslothAI says quantized local runs can fit 128GB RAM, while mervenoyann noted that fitting a 2-bit model on a DGX Spark still does not make it small.

The launch post includes TFLOPs curves instead of only leaderboard tables. The model card gives the routing detail: 6 of 256 experts plus 2 shared experts per token. The vLLM recipe already has Inkling-specific tool and reasoning parsers, and ARC Prize lists ARC-AGI-3 as not yet evaluated.

Inkling-Small is a 42-layer decoder-only sparse MoE with open weights under Apache 2.0, according to the official model card.

• Scale: 276B total parameters, 12B active.
• Routing: 6 of 256 routed experts per token, plus 2 shared experts.
• Attention: hybrid local and global layers.
• Numerics: BF16, MXFP8, and NVFP4.
• Inputs: text, image, and audio.
• Output: UTF-8 text.
• Context: up to 1M tokens in the official docs.
• Access: full weights, Tinker fine-tuning, Tinker Playground chat, Hugging Face, and third-party inference providers.

the Tinker Playground note said Tinker users could test it on launch day, and TestingCatalog's availability note flagged Hugging Face and Tinker Playground access.

Inkling-Small started after Inkling, so Thinking Machines reused the bigger model as both a lesson and a teacher.

The release describes four changes from the original Inkling pipeline:

1. A revised pre-training data mix.
2. A refined ML recipe.
3. On-policy distillation from Inkling into an earlier Inkling-Small preview checkpoint.
4. Two more weeks of agentic coding RL after that preview checkpoint.

the release thread attributes the smaller model's coding and reasoning gains to that sequence. rohanpaul_ai's summary pulled out the same recipe: fewer active parameters, distillation from Inkling, and additional agentic coding RL.

The official benchmark table reports these Inkling-Small deltas against Inkling:

• SWEBench Verified: 77.6% → 80.2%, +2.6 points.
• Terminal-Bench 2.1: 63.8% → 64.7%, +0.9 points.
• SciCode: 46.1% → 48.7%, +2.6 points.
• GPQA Diamond: 87.2% → 89.5%, +2.3 points.
• HLE text-only: 29.7% → 31.6%, +1.9 points.
• CritPt: 5.4% → 8.3%, +2.9 points.
• AA-Briefcase: 839 → 917 Elo, +78 Elo.

Artificial Analysis' benchmark thread matched the broad picture: 40 on its Intelligence Index versus 41 for Inkling, with less than one third of the total and active parameters.

ARC Prize's result added the cleanest cost-performance datapoint: ARC-AGI-1 at 84.0% and $0.11 per task, plus ARC-AGI-2 at 40.1% and $0.23 per task. Its full results page compares that with Inkling at 79.5% and $0.30 on ARC-AGI-1, and 36.5% and $0.64 on ARC-AGI-2.

Thinking Machines' launch post says Inkling still has an advantage in knowledge coverage and factuality. Artificial Analysis put numbers on the caveat:

• τ³-Banking: Inkling-Small 15% versus Inkling 24%.
• AA-Omniscience Index: Inkling-Small -9 versus Inkling 2.
• AA-Omniscience Accuracy: Inkling-Small 31% versus Inkling 40%.
• Hallucination rate: Inkling-Small 57% versus Inkling 63%, so the worse Omniscience score came from lower accuracy rather than more hallucination.
• Output tokens per Intelligence Index task: Inkling-Small ~24K versus Inkling ~25K, with the full Index run using ~131M output tokens for Inkling-Small and ~128M for Inkling.

soumithchintala's reply framed the tradeoff bluntly: the model is broader and more general, so it pays for “all that extra stuff.”

Inkling-Small keeps the same natively multimodal, encoder-free design as Inkling, according to the launch post.

• Images are split into 40×40-pixel patches and transformed with a four-layer hMLP.
• Audio is represented as dMel spectrograms.
• Image, audio, and text representations are processed jointly with text tokens.
• The model can use Python to crop, zoom, and inspect images while reasoning over documents and charts.

The official multimodal table lists 74.0% on MMMU Pro, 77.4% on Charxiv RQ and 81.3% with Python, 54.9% on Audio MC, 77.0% on MMAU, and 90.1% on VoiceBench.

Public surfaces disagreed on the context window.

The official model card, vLLM, Modal, Baseten, and Together AI all advertised up to 1M tokens. Artificial Analysis' benchmark thread described Inkling Small with a 256K token context window, while its model page rounded that to 262K tokens.

That split makes the context number surface-dependent in the first-day record: official weights and several providers say 1M, while at least one evaluator profile used 256K.

The infra response was faster than the benchmark discourse.

• vLLM shipped day-zero support on the existing Inkling path, with prefix caching, LoRA, MTP, prefill disaggregation, and Blackwell optimization. The vLLM recipe includes --tool-call-parser inkling, --reasoning-parser inkling, and auto tool choice.
• SGLang reported 648 tok/s decode with DSpark and 288 tok/s without DSpark on 8× NVIDIA B200, TP 8, NVFP4, batch size 1. Its note also tied Inkling-Small to Miles for multimodal RL.
• Modal listed day-zero support with NVFP4 fitting on a single NVIDIA B300.
• Vercel added it to AI Gateway as thinkingmachines/inkling-small; Vercel's changelog says Zero Data Retention is available and gateway inference has no platform markup.
• Baseten put it on Model APIs with the same 276B total, 12B active, multimodal, 1M-context positioning.
• Together AI said its endpoint targets multimodal, coding, and agentic workloads, with a claim of Inkling-level Terminal-Bench 2.1 performance at about one fifth the cost.

The official model card gives the hardware floor for running the raw checkpoints:

• BF16 requires at least 600GB aggregate VRAM, listed as 4× NVIDIA B300 or 8× NVIDIA H200.
• NVFP4 requires at least 180GB aggregate VRAM, listed as W4A4 on 1× NVIDIA B300 or W4A16 on 2× NVIDIA H200.
• Direct GPU deployment needs SGLang, vLLM, TokenSpeed, Unsloth, or Hugging Face.

UnslothAI released GGUFs for local runs, and Unsloth's GGUF page says quantized Inkling-Small can run on roughly 128GB RAM. woosuk_k's single-B300 note called out the single-B300 angle, while mervenoyann's size joke captured the obvious caveat: 276B total parameters is still not small in the local-model sense.

The model card says Thinking Machines tested Inkling-Small across everyday interaction, dangerous-capability testing, CBRN, cyber, agentic capability, strategic deception, and sabotage potential.

It reports that Inkling-Small did not create “material uplift beyond what’s already available in the open-weight ecosystem.” The residual risks it names are more practical: occasional compliance with role-play and indirectly framed prompts about harmful topics, plus normal foundation-model failures such as hallucination, instruction-following misses, long-conversation degradation, and uneven performance across underrepresented languages or domains.

The model card also says Llama Guard-style moderation tools are compatible with Inkling-Small, because the model's own refusals are not the deployment boundary for an open-weight release.