Skip to content
AI Primer
update

Anthropic engineer claims Claude Code defaults can hit 0% prompt-injection success

Benjamin Cherny said Anthropic trains Claude against prompt injection and that Claude Code’s model, probe, and auto-mode layers can reach 0% attack success in next week’s defaults. The claim comes from an Anthropic engineer rather than an independent benchmark.

7 min read
Anthropic engineer claims Claude Code defaults can hit 0% prompt-injection success
Anthropic engineer claims Claude Code defaults can hit 0% prompt-injection success

TL;DR

  • Claude Code auto mode becomes the default on August 14 for Pro, Max, and Team users, and Anthropic says the classifier caught 89% of dangerous commands versus 14% for manual approval in its testing, according to Claude Developers' rollout thread.
  • The strongest prompt-injection claim is layered: bcherny's reply says Claude Code can reach 0% when model training, probes, and auto mode are combined, while another bcherny reply says the last step uses multiple classifiers.
  • Anthropic's published prompt-injection test reported no successful attacks against Claude models in auto mode across 720 attempts per bar, but bcherny's footnote reply notes the Codex comparison used Codex v0.144.5 before OpenAI's latest Auto-review update.
  • The workflow bet is longer unattended coding runs: Claude Developers' overhead note says auto mode's classifier tokens no longer count against Pro, Max, and Team usage limits, and bcherny's usage note says his team has used auto mode exclusively for months.
  • The safety claim has a boundary: bcherny's credential-leak reply says prompt injection is not the only route to credential leaks, and bcherny's permissions reply says blocked commands can be tuned with /permissions.

Anthropic's launch post hides the operationally spicy bits: 97% of permission prompts are approved, 25% of interactive sessions start in bypass mode, and broad python:*-style allow-rules are set aside while auto mode is active. The auto-mode docs add the enterprise vocabulary: trusted repos, buckets, domains, hard denies, and an autoMode config block. The backdrop is Simon Willison's lethal trifecta: private data, untrusted content, and external communication in the same agent loop.

Auto mode default

Claude Code's new default replaces routine permission prompts with a classifier that reviews shell commands and actions before execution. Starting August 14, Anthropic says new Pro, Max, and Team sessions will run in auto mode unless a user or admin has already pinned another default in the launch post.

The rollout mechanics are concrete:

  • Pro, Max, Team: default auto mode for new sessions on August 14.
  • Existing user default: one-time prompt may ask whether to switch.
  • Pinned admin default: unchanged.
  • Enterprise, API, AWS, Bedrock, Google Cloud Agent Platform, Microsoft Foundry: opt-in for now, with default rollout planned in the coming month.
  • Mode switching: Shift+Tab in the CLI or mode dropdown in desktop surfaces, per Claude Code permission docs.

Boris Cherny, creator and Head of Claude Code according to his LinkedIn profile, gave the trust version in four words: bcherny's trust reply said Anthropic "fully" trusts auto mode.

Prompt fatigue

The human baseline is rough. In Anthropic's controlled study with 1,053 paid professional testers, human reviewers caught 13.6% of dangerous commands, while auto mode blocked 89%, according to Claude Developers' tester-study post.

Anthropic's launch post gives the sharper breakdown:

  • Human testers caught 143 of 1,053 dangerous prompts.
  • Auto mode blocked 937 of the same 1,053 commands.
  • Auto mode blocked 800 commands that humans approved.
  • Humans blocked 6 commands that auto mode allowed.
  • Human blocking fell from about 17% early in a session to about 5% after 50 or more prior prompts.
  • Auto mode's block rate stayed flat across session length.

Anthropic also says Claude Code users approve 97% of permission prompts, reject only 3% of individual permission requests, and reject 39% of plan-approval dialogs. The numbers make manual review look like a ritual, not a control surface.

Prompt injection zero

Cherny's broad claim was that Anthropic has "largely solved" prompt injection in practice for Claude models. The operational claim was narrower: bcherny's 0% reply says Claude Code reaches 0% when the model, probe, and auto-mode layers are stacked.

The public method lives inside Anthropic's launch post, not in a standalone Trajectory Labs report. Anthropic says it commissioned Trajectory Labs to test 72 held-out indirect prompt-injection scenarios, 10 attempts each, against Claude Code and Codex as of July 17, 2026.

The result Anthropic published:

  • Claude Fable 5, Opus 5, Sonnet 5 in auto mode: 0 successful attacks across 720 attempts per model.
  • Claude's latest models in bypassPermissions: 0.09% average attack success without the extra auto-mode safeguards.
  • GPT-5.6 Sol in Codex Auto-review: 5.83% attack success.
  • GPT-5.6 Sol in Full Access: 19.03% attack success.

The comparison has a version caveat. bcherny's Codex footnote reply says the test used Claude Code v2.1.205 and Codex v0.144.5, and that OpenAI had released a newer Auto-review version the prior week.

Defense-in-depth

Cherny described the stack as "a combination of model, probes, and harness." In a separate reply, bcherny on classifiers said the models do most of the work, while multiple classifiers are layered on to reach 0%.

Anthropic's implementation breaks into a few parts:

  • Model training: Claude is trained not to follow malicious instructions from untrusted content, per bcherny's prompt-injection post.
  • Input probes: when Claude pulls external content from web pages, files, or tool outputs, an API-side probe screens for hijack attempts, according to Anthropic's launch post.
  • Intent classifier: auto mode checks whether proposed actions match user intent before a tool call runs.
  • Hard denies: data exfiltration and similar actions can be placed in categories the classifier never approves.
  • Environment context: the auto-mode docs let organizations define trusted repositories, buckets, and domains so the classifier can distinguish internal operations from external leaks.

The system still falls back to manual approvals after three consecutive blocks or 20 blocks in one session, according to Anthropic's launch post. bcherny's open-source reply called open-sourcing auto mode "a cool idea," but no open-source release was announced.

Production agents

The practical reason for the default flip is unattended runtime. Cherny said his team has used auto mode exclusively for months, and trq212's rollout note said classifier overhead is no longer charged when auto mode rolls out by default.

Anthropic's production post and launch post name four production users:

  • Adobe: an agentic loop builds UI, verifies it against intended design, and sends finished PRs for review across Adobe.com merchandising work.
  • Nuro: research agents run overnight, hill-climb evaluation metrics, and return PRs by morning.
  • Gusto: about 10% of sessions since mid-May include a classifier denial.
  • Garner Health: auto mode was pushed as the default to 550 employees through managed settings.

Anthropic also says Teams and Enterprise adopters using auto mode ship about 25% more PRs. Long-running agent work gets a lot less magical when every shell command asks for a click.

Allow-rules and admin knobs

Auto mode changes how broad allow-rules behave. Anthropic says 49.5% of active CLI users had manually created a Bash allow-rule as of June 2026, including 5% who allowed any shell command and another 43% with interpreter rules like Bash(python:*) or Bash(node:*) in the launch post.

In auto mode, permission rules still fire before the classifier, except broad rules that grant arbitrary code execution. Those rules are temporarily set aside so they cannot skip the classifier, and they apply again when the user switches modes.

The admin surface is now part of the product. Anthropic's docs describe defaultMode, disableAutoMode, trusted infrastructure lists, custom hard-deny rules, and CLI inspection commands in the auto-mode config docs. If a legitimate command gets blocked, Cherny said /permissions can tune the classifier in his permissions reply.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 6 threads
TL;DR5 posts
Auto mode default1 post
Prompt fatigue1 post
Prompt injection zero3 posts
Defense-in-depth2 posts
Production agents2 posts
Share on X