Tools raise average VIALS-AA scores from 25% to 47%

• Tools raise average VIALS-AA scores from about 25% to 47%, a roughly 22-point gain, according to Artificial Analysis' results.
• A smaller model can beat a tool-free flagship: the comparison puts agentic Haiku 5.5 at 49% and $0.07 per task, versus Fable 5.2 without tools at 44% and $0.73.
• Cell counting shows the sharpest dependence: Fable jumps from 9% without tools to 65% with them in the domain breakdown.
• Every tested model costs more with tools, with the increase reaching 28× for Qwen3.8 Max in the cost analysis.

The benchmark's headline metric requires three correct runs per task. A Fable trajectory fits circles to curved image features, then unrolls them into straight strips for counting.

Artificial Analysis reran its implementation of Handshake AI's VIALS benchmark with one request containing the question and images, removing tool access. The default agentic setup allows models to execute code and inspect images in a sandbox, including transformed images produced during the run.

Tool-free Haiku also beats agentic GPT-6 Luna, 31% versus 27%, a cross-model reversal that teortaxesTex highlighted.

Counting is where the tools earn their keep: Claude models gain more than 50 points on the 23 cell-counting and quantification tasks, according to the domain results. Scores below run from without tools → with tools:

• Claude Fable 5.2: 9% → 65%, +56 points.
• Claude Opus 5.5: 4% → 57%, +53 points.
• Claude Sonnet 5.5: 13% → 65%, +52 points.
• GPT-6 Astra: 35% → 61%, +26 points.
• GPT-6.1 Sol: 39% → 61%, +22 points.

Code lets agents separate objects and count them using image measurements, rather than relying on a visual estimate. On the separate 23-task phylogenetics subset, Astra moves in the opposite direction: 83% without tools versus 78% with tools, according to the same breakdown.

The reported cost increases vary substantially by model:

• Fable: $0.73 → $4.20 per task, 5.8×.
• Astra: $0.21 → $0.27, approximately 1.3×.
• Sol: $0.05 → $0.06, 1.2×.
• Qwen3.8 Max (0902, xhigh): 28×.
• Gemini 3.8 Flash: 20×.
• MiMo-V2.6-Pro: 15×.

Haiku with tools also beats tool-free Opus, 49% versus 34%, while costing $0.07 rather than $0.33 per task in Artificial Analysis' comparison.

The ten models scoring below 31% without tools average approximately 42 tool calls per task in agentic runs, versus 23 for the seven higher-scoring models, according to Artificial Analysis' analysis. Their score gains and call counts differ sharply:

• MiMo-V2.6-Pro: 5% → 43%, +38 points, approximately 49 calls per task.
• Gemini 3.8 Flash (high): 16% → 41%, +25 points, approximately 68 calls.
• Astra and Sol: +13 and +16 points, respectively, with approximately eight calls each, the fewest among tested models.

Within Anthropic's lineup, calls rise from approximately 21 per task for Haiku to 37 for Fable. Most of the Opus-to-Fable increase comes from viewing images: image views rise approximately 30%, versus about 5% for code-execution calls, according to the tool-use breakdown.

On one protein-interaction task, Fable worked through 51 turns and batched tool calls on 28 of them, according to Artificial Analysis' walkthrough. Its processing sequence was:

1. Turn 4: Create crops and request views of them.
2. Turn 10: Build a color mask by thresholding color channels.
3. Turn 17: Fit a circle to each arc and unroll it into a straight strip.
4. Turn 31: Count bars as peaks in an intensity profile.
5. Turn 41: Map detected spokes back onto the original image to check the result.

Fable crops, masks, measures, and checks a protein-interaction image

That run averaged 1.75 tool calls per turn. Across the full benchmark, Fable averaged 1.60, the highest among tested models in Artificial Analysis' follow-up.

Handshake AI's VIALS repository provides the underlying benchmark. Artificial Analysis evaluates 161 tasks spanning eight artifact domains and 174 source images using Stirrup, its open-source reference harness, with Python execution and no network access, as described on the benchmark page.

Each task runs three independent times. The published metrics distinguish:

• pass^3: Correct in all three repeats, the headline score.
• Average of 3: Average correctness across the repeats.
• pass@3: Correct at least once in three repeats.

In the November 6 launch analysis, Kimi K3.1 Pro and Astra each solved roughly three-quarters of tasks at least once. Requiring all three repeats to succeed separated them to 45% and 63%, respectively.

GPT-6 Luna at medium effort grades final answers using the upstream judge prompts, with task images supplied in a second judge call when needed. A missing answer or missing ANSWER: line scores zero, including refusals, under the evaluation methodology.

The cost methodology averages token usage across the three repeats and applies representative cache-hit rates. Reported costs include contestant-model calls and exclude judge calls.