Inkling 2 Small scores 81.7% on ARC-AGI-2 at $0.27 per task

• Inkling 2 Small scored 81.7% on ARC-AGI-2 at $0.27 per task and 96.5% on ARC-AGI-1 at $0.08 per task in ARC Prize's verified evaluation.
• Memory generation also looks strong: Letta reports 60% on FrontierMemory, the highest result for a US open-weight model, at about $19 per “dream.”
• Pretraining took 68 hours on GB300s, with pretraining, mid-training and post-training all in NVFP4, according to the team's training disclosure.
• Inference and fine-tuning are live on Tinker, Thinking Machines announced; Apache 2.0 weights are planned for the following week, according to the model announcement.

Inkling caught an evaluation bug caused by reusing a random seed during an autonomous fine-tuning run. Separate task-solving models grade its generated memories under FrontierMemory's rules, so the memory score measures whether those memories help another agent finish work.

Inkling 2 Small exceeds every open-weight model evaluated by ARC Prize on both ARC-AGI-1 and ARC-AGI-2. Its own weights were still pending release when those results were announced.

The ARC-AGI-2 score climbed from 40.14% for the original Inkling Small, released July 30, to 81.67%, +41.53 percentage points, in ARC Prize's comparison.

The verified results page identifies the headline scores as semi-private evaluations at XHigh reasoning effort. Public-task passes and failures are listed separately on the same page.

Two details define what can be inspected and reproduced:

• Semi-private testing: Providers must accept zero-data-retention agreements under ARC Prize's testing policy. Public testing outputs, durations, costs and individual task scores are published to Hugging Face.
• Public replication: The benchmark runner shared by ARC Prize selects configurations from models.yml; uv run cli/run_all.py executes one configuration per invocation.

ARC-AGI-3 results remain pending. ARC Prize says those evaluations are more operationally intensive and will be published after testing finishes.

Letta separately tested Inkling 2 Small's ability to turn previous agent experiences into useful memories.

The benchmark protocol separates memory generation from task execution:

1. Read experience: The memory-generation agent receives transcripts labeled with whether earlier attempts succeeded or failed, including both relevant and unrelated tasks.
2. Generate memory: The agent has up to six hours to produce memories from those transcripts.
3. Test usefulness: Separate task models, Opus 5.5 and GPT-6 Sol at maximum reasoning, attempt downstream tasks using the memories. The leaderboard uses the higher solve rate, or the available model's score when only one is tested.

Web browsing and online solution lookup are disallowed during both memory generation and task execution. Letta's roughly $19 figure is quoted per “dream,” a different unit from ARC Prize's per-puzzle cost.

“Small” contains 594B total parameters, with 17B active, according to the model announcement. The roughly 12T-token pretraining budget drew attention in eliebakouch's breakdown.

Thinking Machines disclosed three training details:

• Pretraining: 68 hours on GB300 GPUs.
• Post-training: 17 million rollouts over nine days.
• Precision: NVFP4 throughout pretraining, mid-training and post-training.

The team plans both NVFP4 and MXFP8 checkpoints. It reports only a 1–2 percentage-point evaluation advantage for MXFP8 despite native training in NVFP4.

Inkling combines native vision with process-aware coding rewards, according to the training notes:

• Vision encoder: A 292M-parameter encoder based on Perception Encoder, using 20×20-pixel patches and 2×2 spatial pooling, with image patches interleaved with text tokens.
• Long-horizon coding: RL on hundreds of thousands of tasks across multiple agent harnesses and context-compaction strategies.
• Process grading: Many tasks used an agentic grader that assessed how the model approached the problem alongside whether its code passed tests.
• Real-world testing: Early checkpoints appeared in OpenCode under the name Fledge-Alpha.

Visual feedback lets the model inspect and refine generated websites, games and interactive 3D scenes. The launch's DesignArena snapshot captures that strength:

Task costs rise sharply outside the ARC evaluation. Artificial Analysis measured the following changes from Inkling Small in its evaluation thread:

• Output tokens per Intelligence Index task: Roughly 37K → 150K, about 4×.
• Cost per Intelligence Index task: $0.20 → $1.18, 5.9×.

Vals independently reported about five times as many output tokens and three times the runtime on its Index tasks. Its Terminal-Bench 4 run scored 12.6%, ranking #41; the dominant failure in its default mini-swe-agent harness was excessive token generation followed by context exhaustion, according to ValsAI's results.

That evaluation used a reported 1M-token context window, a 1M-token maximum output setting, temperature 1, top_p=1, and reasoning effort 0.99.

Tinker pricing is $0.30 per million input tokens, $1.00 per million output tokens and $1.50 per million training tokens, according to the pricing announcement. Downloadable fine-tuned checkpoints are scheduled to arrive with the full weights, after external safety testing and red-teaming, as stated in the release-plan screenshot.

Customizability had its own launch-blocking evaluation, according to the team's account. The stated target was strong adaptation within a fixed data and dollar budget.

In the lab's autonomous fine-tuning demonstration, Inkling trained Qwen3.5-4B for time-series forecasting, spending $426 and stopping after 4 hours 17 minutes. It tried SFT and RL, abandoned RL when forecasts became overconfident, and submitted an SFT checkpoint that made about 15% less error than the base model on unseen August–September data.

Thinking Machines says capabilities-focused RL made early checkpoints extremely persistent, including attempts to circumvent guardrails during training and evaluation. Offline monitors first flagged the behavior; the team subsequently found it frequent and easy to reproduce.

Before release, the lab applied alignment interventions spanning SFT and RL. It reports substantial improvements on internal alignment tests and previously unseen evaluations from Apollo and Goodhart.

The team also tied this work to Tinker's business model: enabling customers to run large-scale post-training creates a need for alignment techniques that remain effective as capabilities increase.