Skip to content
AI Primer
release

Claude Code adds computer use in research preview for Pro and Max

Anthropic put computer use directly into Claude Code, letting the CLI open apps, click through GUIs, and verify work on screen. Try it if you want Claude Code to handle end-to-end UI tasks beyond file edits, but note it is rolling out as a research preview on Pro and Max plans.

5 min read
Claude Code adds computer use in research preview for Pro and Max
Claude Code adds computer use in research preview for Pro and Max

TL;DR

  • Anthropic added computer use directly to Claude Code, so the CLI can now open apps, click through GUIs, and verify what it built on screen instead of stopping at file edits and shell commands Anthropic launch tweet.
  • The feature is a research preview for macOS users on Pro and Max, and the official docs add two important caveats: it needs Claude Code v2.1.85 or later, and it only works in interactive sessions, not -p print mode Anthropic launch tweet.
  • Anthropic positions screen control as a last-resort tool. The docs say Claude will prefer MCP servers, Bash, or browser automation first, then fall back to computer use when the task needs a real desktop Anthropic thread.
  • The permission model is tighter than the launch clip suggests: users must grant macOS Accessibility and Screen Recording, then approve specific apps per session, with extra warnings for terminals, IDEs, Finder, and System Settings Wes Roth summary.
  • One small extra reveal came from independent analysis of the new computer-use MCP: it appears to include a teach mode with tooltip overlays and step-by-step guided tours, a clue that Anthropic is thinking beyond testing into interactive UI walkthroughs Teach mode analysis.

You can read Anthropic's new CLI computer use docs, the broader Desktop docs, and the original 2024 computer use launch post. The docs also hide a few useful specifics that are not in the tweet: /mcp is where the built-in computer-use server gets enabled, only one Claude session can hold the desktop lock at a time, and the terminal window stays visible but excluded from screenshots.

Computer use inside the agent loop

Anthropic's launch video is short, but the thread spells out the pitch: one prompt can write code, compile it, launch the app, click through the UI, find a bug, fix it, and verify the result.

That closes a gap Claude Code has had since launch. It could already read files, edit code, and run commands. Now Anthropic is threading GUI execution into the same conversation loop, so native apps, Electron builds, simulators, and other GUI-only tools no longer sit outside the agent.

The official docs frame the feature in concrete workflows:

  • build and validate native macOS apps
  • run end-to-end UI testing without a Playwright harness
  • reproduce visual bugs by resizing windows and taking screenshots
  • operate GUI-only software that has no CLI or API

The fallback order matters

The most interesting sentence in the docs is not about clicking buttons. It is the tool routing rule.

According to Anthropic, Claude tries more precise interfaces before it reaches for the desktop:

  1. MCP server, if one exists for the service
  2. Bash, if the task is a shell command
  3. Claude in Chrome, for browser work
  4. Computer use, if nothing else can reach the target

That makes computer use the broadest tool in the stack, but also the slowest and least structured. Anthropic's docs say screen control is reserved for native apps, simulators, and tools without an API. The product story here is not "the agent got eyes." It is that Claude Code now has a last-mile tool for workflows that still break when everything else works.

Permissions, lock, and escape hatch

The rollout is gated harder than the demo implies. The docs say users first have to enable the built-in computer-use MCP server from /mcp, then grant macOS Accessibility and Screen Recording.

After that, Claude still does not get blanket access. App approvals are granted per session, and Anthropic adds special warnings for high-reach apps:

  • terminals and IDEs are labeled equivalent to shell access
  • Finder is labeled able to read or write any file
  • System Settings is labeled able to change system settings

Anthropic also built in a machine-wide lock. Only one Claude Code session can control the desktop at once. While it is active, other visible apps are hidden, the terminal stays visible but is excluded from screenshots, and pressing Esc aborts the action immediately. Those details read like the real preview boundary, not the headline clip.

Desktop parity, but not everywhere

The docs make clear this is not a universal Claude Code feature yet.

Computer use currently requires macOS, a Pro or Max subscription, and authentication through claude.ai. Anthropic explicitly says it is unavailable on Team and Enterprise, unavailable with third-party providers like Bedrock, Vertex AI, or Foundry, and unavailable in non-interactive CLI mode.

The Desktop app uses the same underlying computer-use engine, according to Anthropic's docs, but the CLI and Desktop surfaces are not identical. Desktop has a graphical settings path, a denied-apps list, and some adjacent features like app preview and PR monitoring. The CLI path is narrower: enable computer-use in /mcp, approve apps, then let the agent drive.

Teach mode

The strangest detail did not come from Anthropic's launch materials. It came from Marcel Pociot, who inspected the newly announced computer-use MCP and found commands for a guided overlay system.

His screenshot lists three teach-mode actions:

  • teach_start, begin a guided UI tour with tooltip overlay
  • teach_step, show a tooltip, wait for the user's Next click, then execute actions
  • teach_batch, queue multiple teach steps in one call

If that reading is accurate, the same machinery Anthropic just exposed for testing apps can also be used to explain them. That is a different product surface from autonomous debugging, and it is the most revealing hint in the whole rollout.

Share on X