DeepSeek V4.1 Flash 1017 scores 5.81% on ARC-AGI-3 with provider adapter

• DeepSeek V4.1 Flash 1017 reached 5.81% on ARC-AGI-3 with the provider adapter, versus 0.2% with the standard harness, in ARC Prize’s verified evaluation.
• Semi-private performance more than doubled, but public performance fell: ARC Prize’s comparison reports 2.68% → 5.81% and 11.05% → 8.97%, respectively.
• First-action reasoning used fewer tokens: ARC Prize measured reductions of 36% on public games and 48.6% on semi-private games.

The public FT09 game score jumped from 4.8% in the original model’s max-reasoning run to 91.4% in 1017, despite the falling public aggregate. ARC Prize’s testing policy also distinguishes model-written notes from opaque reasoning state carried between requests.

ARC Prize places 1017 first among open-weight models on ARC-AGI-3. Its max-reasoning evaluations cost roughly $1,200 with the provider adapter and $1,000 with the standard harness.

The two harnesses expose different context-management capabilities, according to the official definitions:

• Standard: A minimal, provider-neutral interface that lets the model carry forward notes it chooses to retain.
• Provider Adapter: Provider-designed context management that preserves opaque reasoning state between requests and compacts longer conversations, allowing reuse of prior work.

The harness gap is the bigger engineering result: 5.61 percentage points, versus 3.13 points for the version bump.

The comparison with the September 10 release holds the provider adapter, max reasoning and pricing constant. Both runs used Baseten prices of $0.30 per million input tokens and $1.20 per million output tokens, according to ARC Prize.

• Semi-private: 2.68% → 5.81%, +3.13 percentage points.
• Public: 11.05% → 8.97%, −2.08 percentage points.

The predecessor’s full results put its best provider-adapter score at 3.3% under low reasoning. The headline doubling compares max with max, rather than each version’s best reasoning setting.

ARC Prize’s efficiency comparison covers reasoning generated before the first action, at max reasoning with the provider adapter. The averages span 25 public games and 55 semi-private games; they do not measure whole-run token savings.

Higher reasoning effort produced uneven scores on the older benchmarks in the verified results table:

• ARC-AGI-1: High reasoning scored 94.5%, tying the open-weight lead. Max scored 93.5%.
• ARC-AGI-2: Max scored 78.1%, leading open-weight models. Low scored 73.6%, ahead of high at 68.6%.

The ARC-AGI-2 peak is 5.2 percentage points above the predecessor’s 72.9% max-reasoning score.

teortaxesTex compared 1017 with three closed-model reasoning configurations in a post:

• GPT-6 Sol, high: Roughly level with 1017, with both using provider adapters.
• GPT-6 Sol, max: Around 23%, substantially above its high-reasoning result.
• Haiku 5.5, max: Around 2.8%, roughly half 1017’s score.

The cost comparison uses DeepSeek’s peak pricing, which teortaxesTex described as delivering twice Haiku’s score at Haiku’s cost.

ARC Prize publishes task-by-task replays showing the exact ARC-AGI-3 run, according to its testing policy. Semi-private testing requires zero-data-retention agreements with providers, although the foundation acknowledges possible limited leakage through external APIs.

Each reported score comes from a single run, without averaging across runs, and semi-private evaluations have a $10,000-per-run cap.