Google previews Gemini 4 Carbon for selected cyber defenders

• Gemini 4 Carbon starts as a restricted cyber-defense preview: Google says Fairwind participants get access first, with developers, businesses, and consumers to follow.
• General intelligence rose from Argon’s 50 to 59, +9 points, in Artificial Analysis’s evaluation. Carbon remains behind Opus 5.5 on coding agents.
• Finance and tax tasks took more than twice as long as Argon’s in ValsAI’s tests, with legal research exceeding 50 minutes per task.
• Hallucination rate rose from 4% to 11%, +7 percentage points, while factual accuracy improved from 48% to 58%, according to Artificial Analysis.

Google’s cyber benchmark scores were measured without guardrails. Its evaluation notes describe both six-hour and ten-hour ExploitGym timeouts. The launch post also credits Google’s Chrome team with a more-than-10x improvement in high-quality patch generation over an unnamed rival model.

• Preview: Carbon follows Argon by two weeks, according to Google’s announcement.
• Initial access: Selected cyber defenders enter through Fairwind this week, per Google’s rollout thread.
• Broader availability: Developer, business, and consumer access remains forthcoming, without a date in Google’s access announcement.
• Introductory pricing: $2 per million input tokens and $10 per million output tokens, unchanged from Argon, according to the pricing announcement.
• Standard pricing: $4 input and $20 output after the 50% introductory discount; its end date is unconfirmed, per Artificial Analysis.

Comparisons below are Argon → Carbon. Google’s full launch table also includes science, multimodal understanding, computer use, and knowledge-work results.

First-party

• SEC-bench Pro, vulnerability discovery: 83.4% → 88.2%, +4.8 percentage points, in Google’s cyber results.
• ExploitGym, exploitation: 42.0% → 54.8%, +12.8 percentage points, in Google’s cyber results.
• SWE-sweep: 4.3% → 9.6%, +5.3 percentage points, in Google’s comparison.
• GraphWalks, 256K–1M-token BFS F1: 84.2% → 96.2%, +12.0 percentage points, in Google’s comparison.

Third-party evaluators

• Artificial Analysis Intelligence Index v5: 50 → 59, +9 points, per Google’s report of the result.
• Artificial Analysis Coding Agent Index: 53 → 59, +6 points, with Antigravity CLI, per Artificial Analysis.
• AA-Omniscience v1.1: 46 → 54, +8 points, per Artificial Analysis’s results.
• AA-Briefcase: 1483 → 1609 Elo, +126 Elo, per Artificial Analysis’s knowledge-work results.
• Collinear’s CWE-bench, patching: 68.0% → 71.0%, +3.0 percentage points, in Google’s comparison.
• Vals Index v2.2: 69.2% → 70.4%, +1.2 percentage points, reaching #2 overall in ValsAI’s results.
• Text Arena: 1520 → 1528, +8 points, reaching #1 in Arena’s launch results.
• Code Arena: WebDev: 1679 → 1768, +89 points, reaching #4 in Arena’s launch results.
• Agent Arena net improvement versus baseline: 11.0% → 16.2%, +5.2 percentage points, per Arena’s comparison.

Customer-reported

• No external customer before-and-after benchmark series appears in Google’s launch post, which cites its own Chrome team and Wiz.

• Calibration: AA-Omniscience hallucination rate rose 4% → 11%, +7 percentage points, despite accuracy rising 48% → 58%, +10 percentage points, in Artificial Analysis’s tests. Carbon has the second-lowest hallucination rate, behind Argon.
• Research latency: Finance Agent and Tax Agent took more than 2x Argon’s time; Legal Research exceeded 50 minutes per task, according to ValsAI. Carbon was faster and more accurate on EMB and Terminal-Bench 4.
• Token use: Reasoning consumption reached 2.3x Argon’s in ValsAI’s evaluation. Output increased 64%, concentrated in finance, tax, and legal research, while EMB, Terminal-Bench 4, and Harvey used roughly the same or fewer tokens in ValsAI’s follow-up.
• Coding-agent ceiling: Carbon’s 59 trails Claude Code with Opus 5.5 at 62, 3 points behind, in Artificial Analysis’s comparison.
• Desktop-use ceiling: OSWorld’s offline subset scored 81.5%, versus Opus 5.5’s 87.2%, 5.7 percentage points behind, in Google’s table.
• Shell reliability: Arena’s comparison records −0.3 percentage points in Bash reliability versus Argon in the published chart.

The AA-Omniscience definition calculates hallucination rate as incorrect / (incorrect + partial answers + not attempted). Its denominator includes refusals and partial answers, while accuracy uses all questions.

Context and development

• Limits: ValsAI reports a 1M-token context window and a 1M-token maximum output in its configuration details. Artificial Analysis lists text, image, video, and speech inputs, with text output, in its model details.
• Training lineage: Google describes Carbon as “building on” Argon in its launch post, without specifying whether that means new pretraining or continued training of Argon.
• User feedback: Coding usability and output-quality changes incorporated feedback on Argon, Google said.
• Development automation: Agents and automated feedback loops helped accelerate releases, according to mirrokni. Separately, lmthang credited reasoning-team contributions and referred to an “RSI recipe,” without describing that recipe.
• Long-running work: Google reports internal runs lasting days, including root-causing and patching subtle bugs across large codebases and low-level systems. Its preview announcement describes persistence through complex tasks without getting stuck or taking shortcuts.

Evaluation harnesses

The published methodology discloses several settings beneath the headline scores:

• Terminal-Bench discrepancy: Google’s table lists 65.2%; Artificial Analysis’s Antigravity CLI results report 64%, a 1.2-percentage-point difference. Google attributes its row to Artificial Analysis, but the published methods do not reconcile the figures.
• SEC-bench Pro: The 183-task May 2026 JavaScript-engine split uses Antigravity, with pass@1 averaged over three independent attempts per task. A proof of concept must crash the vulnerable build but not the patched build, or pass source review by an LLM judge if both crash.
• ExploitGym: Antigravity gets one attempt per task without internet access. Success requires capturing the flag and an LLM judge confirming exploitation of the intended vulnerability.
• Agents’ Last Exam: Google uses the 122-task October 17 version, without the October 26 update, through ALE-Claw with a five-hour window and safety filters enabled.
• OSWorld: The score covers 82 offline tasks and takes the maximum over three single-attempt runs. Its screenshot-based harness permits parallel batch tool calls, retains screenshot history with compaction, and allows 500 steps.

Safety controls

Google reports five alignment changes in its launch post:

1. Shortcut resistance: Successful Terminal-Bench cheating fell to 0.0%, with attempts at 0.4%; high-severity misalignment alerts in internal live coding fell almost 80%.
2. Transparency: Unsupported claims roughly halved versus Argon in internal testing, with missing evidence and blocked actions disclosed explicitly.
3. Ambiguity handling: Carbon retrieves sources and surfaces discrepancies before acting.
4. Scope control: Overreach fell 26% and under-service fell 29%.
5. Prompt injection: Gray Swan indirect prompt-injection attack success fell 0.7% → 0.0%, −0.7 percentage points, across the evaluated attacks.

Task economics

• Artificial Analysis: Cost per Intelligence Index task changed from Argon’s $7.68 to Carbon’s $7.76, +$0.08, at standard prices. Carbon cost approximately 23% less than Opus 5.5 and 24% more than GPT-6 Astra, according to Artificial Analysis’s comparison.
• Agent Arena: Its leaderboard lists Carbon at the introductory $2/$10 rates. Arena’s launch results reported a $1.80 median task cost, 72% below Opus 5.5; this uses a different task population and pricing basis from Artificial Analysis.

• Carbon kept andrew_n_carr almost exclusively in Antigravity for his last three working days. He still rated Opus 5.5 higher in a follow-up.
• Constraint-following improved and output contained less “slop,” according to legit_api, who also asked for xhigh and max reasoning levels.
• Peter Gostev published side-by-side 3D, game, visual-explainer, and UI tests in Arena’s preview. The video includes a chapter on missing detail at 6:28, and the prompts are public.

Peter Gostev’s side-by-side Carbon tests, including 3D scenes, visual explainers, and UI generation.