Google DeepMind launches MKA benchmark with 200 visual knowledge-work tasks

• MKA covers 200 private, expert-written tasks across 10 industry verticals, according to the launch breakdown.
• Claude Fable 5.2 (max) leads at 57% Climb WSR versus Gemini 4 Carbon (high) at 54%; average task costs are $31.97 and $6.29, respectively, in ArtificialAnlys's results.
• Agents must turn visual understanding into finished decks, documents and spreadsheets, as tulseedoshi described.

The public examples extend to RF chip design and biochemical assays. The grading agent evaluates deliverables without receiving the original input files.

Artificial Analysis partnered with Google DeepMind on MKA. Domain experts spanning 83 occupations created projects estimated to take a seasoned professional three to eight hours each, according to Google's launch post.

Artificial Analysis keeps the 200 evaluation tasks private to limit training contamination and discovery through search. A separate 10-task public showcase, licensed under Apache 2.0, exposes three components per task:

• metadata.json: the prompt, professional role, task goal and input filenames.
• rubrics.json: grading criteria and their weights.
• input/: the source files supplied to the agent.

Every model runs in the same open-source Stirrup harness. The published methodology specifies:

• Runtime: a Linux sandbox with up to 250 turns per task.
• Tools: code execution, internet access, web search, page fetching and image viewing.
• Image handling: original resolution unless provider constraints require resizing.
• Rubrics: roughly 70 yes/no checks per task, weighted 75% toward factual accuracy and 25% toward visual design.
• Judge: GPT-6 Luna (medium), inspecting submitted files and the final response in its own sandbox.
• Scoring: each criterion carries a weight from 1 to 5. Climb WSR averages the weighted share of satisfied harder criteria across tasks; Regression WSR separately measures baseline competence.

The rubric split earns its keep: the leading seven models cluster at 95% to 97% on baseline checks, while Climb scores across the field range from 28% to 57%.

Google reports 96% agreement between the autorater and domain experts grading model outputs.

Fable's lead is 2.3 points on the unrounded results, 56.6% versus 54.3%, in GoogleDeepMind's launch chart. That chart lists Fable at $35.28 per task and Carbon at $5.63, whereas Artificial Analysis's announcement reports $31.97 and $6.29: a roughly 6.3× cost ratio versus 5.1×.

Further down the leaderboard, GPT-6.1 Sol scores about 48% at $1.11 per task, roughly 29 times cheaper than Fable, according to the same announcement. Reported costs include calls to the model under test and exclude judge calls.

The public architectural task requests an exception-only coordination audit capped at 10 slides:

• Cross-check five floor plans against Sections A-A and B-B and four elevations.
• Report mismatches and “Cannot Determine” findings, supported by drawing crops and floor-level counts.

In ArtificialAnlys's side-by-side run, the two leading models produced these results:

| Model | Turns | Image views | Approximate time | Cost | Task score |
| --- | ---: | ---: | ---: | ---: | ---: |
| Fable 5.2 (max) | 224 | 203 | 3 hours | $68 | 87% |
| Carbon (high) | 76 | 118 | 1 hour | $8 | 72% |

No model leads every vertical in ArtificialAnlys's breakdown:

• Fable 5.2: Business, Engineering, Logistics & Mapping, Software & Data, and Finance jointly with Sonnet 5.5.
• Opus 5.5: Construction and Education.
• Gemini 4 Argon: Science.
• Gemini 4 Carbon: Healthcare.
• Sonnet 5.5: Law & Government, narrowly.

No model exceeds 52% on Climb criteria for Word-document or text-file outputs, according to the evaluator's file-type breakdown. Leadership across the six output formats splits three ways:

• Fable 5.2: PDF, Word documents, spreadsheets narrowly, and presentations jointly with Opus 5.5.
• Gemini 4 Argon and Carbon: images.
• DeepSeek V4.1 Flash 1017: text files.

In a separate effort, stephenhaney is developing a design benchmark focused on augmenting designers' workflows. Two proposed categories reach into different parts of design work:

• Existing design systems: staying on-brand and on-system, plus speed and conversion accuracy in both directions, feature in his proposed categories.
• Recognizable AI styling: every model performs poorly on the proposed “no slop tells” evaluations, he said in a reply.

He plans to update the benchmark whenever a new model arrives.