Vals reports DeepSeek completed 72.8% of simulated cyber objectives

• DeepSeek V4.1 Flash 1017 completed 72.8% of simulated cyber objectives, eight percentage points behind GPT-6 Sol Daybreak's 80.8%, according to ValsAI.
• Nova's near-parity result cost under $3 per DeepSeek run versus about $22 for Daybreak, as ValsAI's accompanying thread reports.
• Short runs missed DeepSeek's eventual capability: teortaxesTex found it still at zero Lighthouse objectives after 44.6 million cumulative tokens, before it ultimately completed 21.

Meta Wonder quietly topped Lighthouse with 28 of 31 objectives, despite a weak Nova result. The token accounting includes cached inputs and reasoning tokens, so the enormous totals include repeatedly reading existing context.

Vals evaluated seven models across two isolated enterprise-network simulations. DeepSeek has downloadable weights; GPT-6 Sol's cyber access came through OpenAI's restricted Daybreak program.

Each range contributes half the overall score under the published scoring rules:

| Model | Nova, 21 objectives | Lighthouse, 31 objectives |
| --- | ---: | ---: |
| GPT-6 Sol Daybreak | 15.7 | 27 |
| DeepSeek V4.1 Flash 1017 | 16.3 | 21 |
| Kimi K3.1 Pro | 13 | 20 |
| Meta Wonder | 7 | 28 |
| Grok 4.7 | 12 | 16 |
| Kimi K3 | 9 | 10 |
| GLM 5.3 Flash | 5 | 5 |

For Daybreak and DeepSeek, Nova entries average three runs: Daybreak ranged from 12 to 19 objectives, DeepSeek from 15 to 18. The table reports one Lighthouse score per model.

Every model ran inside the open-source Pi agent on an isolated Kali Linux machine, according to the methodology.

The execution and scoring controls were:

• Starting information: task instructions, a public web-application address and an authorized scope, with no human help during the run.
• Network access: only the model provider and the simulated range, with a fresh range copy for every run.
• Execution: run a command, read its output, choose the next step.
• Scoring: search recorded command output for hidden markers from a private answer key. The model's own success claims earn no credit.
• Limits: objectives count within 500 million cumulative tokens; runs also have a 48-hour ceiling.

Nova contains connected Windows Active Directory domains. Lighthouse spans two company networks, with information from the first providing the only route into the second.

Nova cost under $3 per DeepSeek run versus about $22 for Daybreak in Vals's published comparison. That price gap accompanied slightly higher average objective completion for DeepSeek on this range.

On Lighthouse, Daybreak completed 27 objectives in under three hours, including administrator access, password theft and movement into the second company. DeepSeek completed 21.

Daybreak reached 25 of Lighthouse's 31 objectives before 100 million cumulative tokens. DeepSeek made most of its progress after that threshold, spending two to four times as many tokens, according to Vals's analysis.

An early cutoff would have badly understated DeepSeek's reach. Its inexpensive tokens made that long cold start affordable.

Additional budget did not rescue every model. Vals reports several limits in its run analysis:

• Every run stopped on its own before reaching either the token or time ceiling.
• Grok 4.7, Kimi K3 and GLM 5.3 Flash stalled and never recovered.
• GLM spent roughly 36 hours and a thousand turns on one range, much of it sleeping and polling a background job that never returned, and finished with five objectives.
• No model completed Lighthouse's final objectives. Only one run reached Nova's final stage.

Observed criminal and state-backed activity still often uses older, less capable models to conserve resources, according to WIRED's reporting. These experiments measured autonomous capability in controlled networks, not the frequency of comparable attacks in the wild.