Skip to content
AI Primer
breaking

ARC Prize verifies Gemini 3.7 Flash at 84.6% on ARC-AGI-2

ARC Prize verified Gemini 3.7 Flash at 84.6% on ARC-AGI-2, at a reported $0.25 per task. Artificial Analysis’ AnalystAgent benchmark placed it at 60%, ahead of Claude Opus 5 and GPT-5.5.

4 min read
ARC Prize verifies Gemini 3.7 Flash at 84.6% on ARC-AGI-2
ARC Prize verifies Gemini 3.7 Flash at 84.6% on ARC-AGI-2

TL;DR

  • ARC Prize verified Gemini 3.7 Flash at 84.6% on ARC-AGI-2 for a reported $0.25 per task, according to arcprize's verified results.
  • The model led Artificial Analysis' 80-task AnalystAgent board at 60%, with Claude Opus 5 at 53.8% and GPT-5.5 at 50.0% in WesRoth's benchmark chart.
  • The ARC-AGI-2 cost result has become the headline trade-off, which DynamicWebPaige described as a Pareto-frontier point; Google's API prices use a separate per-token unit.
  • ARC Prize is still running Gemini 3.7 Flash through ARC-AGI-3 and says it will publish the result once the evaluation is complete in arcprize's ARC-AGI-3 update.

Google's introductory API rate expires on December 31, when the published input and output rates double. Artificial Analysis also reports two deliberately different speed numbers: 1.3 minutes of weighted decode time, excluding first-token time and overhead, and 1.7 minutes of average Time per Task.

ARC-AGI-2

ARC Prize reported two verified result-and-cost pairs:

  • ARC-AGI-2: 84.6% at $0.25 per task.
  • ARC-AGI-1: 95.5% at $0.12 per task.

The verification label has a defined gate. Under ARC Prize's Verified Testing Policy, the program works with selected providers and tests public models from trusted sources. ARC-AGI-2 itself has a 120-task public evaluation set, alongside semi-private and private sets, on the official dataset page.

AnalystAgent

Artificial Analysis defines AA-AnalystAgent in its benchmark analysis as a test of answering complex questions about spreadsheets and documents. The score chart measures the share of 80 tasks solved on all five of five attempts in WesRoth's benchmark chart.

  • Gemini 3.7 Flash, high: 60.0%.
  • Claude Opus 5, max: 53.8%.
  • GPT-5.5, xhigh: 50.0%.
  • Claude Fable 5 with fallback: 48.8%; GPT-5.6 Sol, max: 47.5%, a gap also highlighted by gabriberton's benchmark summary.

Gemini 3.6 Flash baseline

ARC Prize's arcprize's comparison post added a direct 3.7-versus-3.6 view. Google's launch post supplies its first-party intergenerational deltas:

  • FrontierCode 1.1 Main: 34.4% to 43.6%, up 9.2 points.
  • DeepSWE v1.1: 49.0% to 65.3%, up 16.3 points.
  • WebDev Arena: 1538 to 1588 Elo, up 50 Elo.
  • GDP.pdf: 22.0% to 34.0%, up 12.0 points.
  • AutomationBench: 17.0% to 30.4%, up 13.4 points.

Cost, context, timing

ARC's $0.25 is a cost-per-task result, while Google's introductory API rate is $0.75 per million input tokens and $3.75 per million output tokens through December 31. The published standard rate from January 1 is $1.50 and $7.50, respectively. Artificial Analysis calculates $0.40 per Intelligence Index task at high reasoning, 30% below 3.6 Flash, and $0.26 at medium reasoning in its independent report.

The same report lists a one-million-token context window, unchanged from 3.6 Flash, and text, image, video, and speech inputs with text output. It records roughly 340 output tokens per second and a 1.7-minute average Time per Task at high reasoning.

The

labels the lower 1.3-minute figure as weighted average decode time, explicitly excluding time to first token and overhead.

Agent runs and quotas

One smaller 3-D coding measurement, as rohanpaul_ai's test summary described it, used Nous Research's Hermes Agent CLI through OpenRouter, identical prompts, and the same Three.js constraints across three voxel-city scenes:

  • Muse Spark 1.2: $0.53 total cost.
  • Gemini 3.7 Flash: $0.56.
  • DeepSeek V4 Pro: $4.57.

Three-agent voxel-city cost comparison

One fofrAI reply to a commenter says simply, “Absolutely.” doodlestein supplied a more concrete scope, saying they would trust Gemini 3.7 Flash with web front-end work and other less important tasks in doodlestein's hands-on note.

An

says a plan's baseline quota would refresh on August 25 and that Antigravity does not support higher rate limits on that Google plan.

ARC-AGI-3

ARC Prize says it is currently running ARC-AGI-3 evaluations for Gemini 3.7 Flash and will publish the numbers when they are complete.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 5 threads
TL;DR1 post
AnalystAgent1 post
Gemini 3.6 Flash baseline1 post
Cost, context, timing1 post
Agent runs and quotas4 posts
Share on X