DeepSeek V4 Flash benchmarks show cheaper tokens but 3x SWE-Bench task cost

• DeepSeek shipped a real checkpoint, weights, a technical report, and an MIT license, and kimmonismus's weights note says the 0731 build supersedes the V4-Flash preview.
• The headline benchmark move was agentic: Artificial Analysis put V4 Flash 0731 at 50 on its Intelligence Index, up 10 points, while Cline's changelog catch reported a Terminal-Bench score of 82.7.
• Per-token cheap did not always mean per-task cheap: Ramp's SWE-Bench follow-up measured 3x the cost per task versus GPT-5.6 Luna, nearly 3x longer harness runs, and more turns.
• Cache hits are DeepSeek's price weapon: Artificial Analysis lists $0.0028 per million cached input tokens, while teortaxes's pricing table compared that with Luna's $0.02 cache-read price.
• The rollout hit agent tools fast: Cline's rollout, Ollama's rollout, and Unsloth's local guide put the model into IDE-agent flows, cloud runs, and local GGUFs.

The Hugging Face model card has the weird implementation bits: no Jinja chat template, a dedicated encoding folder, three reasoning_effort levels, and DSpark speculative decoding through one vLLM flag. Artificial Analysis found the omniscience gain came from fewer hallucinations, not higher accuracy. Hacker News was already arguing about the unreleased DeepSeek Harness. Unsloth says its lossless GGUF is 162GB because the routed experts are natively MXFP4.

DeepSeek's model card calls DeepSeek-V4-Flash-0731 the official release of V4-Flash, replacing the preview, with the same DSpark-style model structure and a speculative decoding module attached. The repository and weights are MIT licensed, according to the official model card.

The official benchmark table is the cleanest way to read the jump:

• Terminal Bench 2.1: 61.8 → 82.7, +20.9 points.
• NL2Repo: 39.4 → 54.2, +14.8 points.
• Cybergym: 38.7 → 76.7, +38.0 points.
• DeepSWE: 7.3 → 54.4, +47.1 points.
• Toolathlon-Verified: 49.7 → 70.3, +20.6 points.
• DSBench-FullStack: 37.0 → 68.7, +31.7 points.
• DSBench-Hard: 25.8 → 59.6, +33.8 points.

DeepSeek's caveat is buried under the table: public Code Agent tasks used the minimal mode of DeepSeek Harness, which the model card says is still "to be released," with max reasoning effort, temperature = 1.0, and top_p = 0.95.

Artificial Analysis measured the new model at 50 on its Intelligence Index, up from 40 for the previous V4 Flash and 6 points ahead of DeepSeek V4 Pro. The same analysis put GPT-5.6 Luna at 51, one point higher, but said DeepSeek's first-party cost per task was about 60% lower.

The component moves were lopsided toward agents:

• GDPval-AA v2: 1189 → 1559 Elo, +370 Elo.
• Terminal-Bench 2.1: +17 points to 79%.
• τ³-Bench Banking: +8 points to 31%.
• CritPt: +9 points to 17%.
• SciCode: +5 points to 50%.
• HLE: +5 points to 37%.
• AA-LCR: +3 points to 66%.
• GPQA Diamond: +1 point to 91%.

The hallucination result is the non-obvious one. Artificial Analysis said AA-Omniscience improved from -23 to -16 because hallucination rate fell to 84%, while accuracy stayed unchanged at 37%.

Ramp's coding benchmark cut against the cheap-token story. Ramp's follow-up said V4 Flash cost 3x more per task than GPT-5.6 Luna on Ramp SWE-Bench, spent nearly 3x longer in the harness, and took more turns on every task.

The behavior profile was also different. Ramp said DeepSeek burned many turns before its first code change, then persisted through repeated tests and edits to the same files; Ramp's SWE-Bench page describes the benchmark as 80 production-grounded tasks derived from merged pull requests and run through a shared mini-swe-agent harness.

Task economics, not token pricing alone, decided the split here.

DeepSeek's published first-party prices are $0.14 per million input tokens, $0.28 per million output tokens, and $0.0028 per million cached input tokens, according to Artificial Analysis. Luna's prices in teortaxes's comparison were $0.20 input, $1.20 output, and $0.02 cache hits, with higher prices above 272K context.

The same table listed DeepSeek cache writes as free for V4 Flash and V4 Pro. The Luna screenshot in teortaxes's pricing source showed cache writes billed at 1.25x the uncached input rate and long prompts above 272K priced at 2x input and 1.5x output.

That is the split between long-context agent sessions with stable state and Ramp-style runs where extra turns can swamp the token discount.

vLLM described V4 Flash 0731 as a sparse MoE with 256 routed experts, six active per token, a 1M-token context window, and three reasoning-effort levels. The DSpark speculative decoder ships inside the weights, so vLLM enables it with a --speculative-config flag rather than a second draft model.

Local running is possible but not small:

• Unsloth said lossless 4-bit runs on 168GB RAM, while 3-bit runs on 110GB RAM.
• Smaller quants later expanded to 1-bit on 96GB RAM and 3-bit on 128GB RAM, according to Unsloth's quant update.
• Daniel Han Chen listed UD-Q8_K_XL at 162GB, UD-Q4_K_XL at 155GB, UD-Q3_K_XL at 129GB, UD-IQ3_S at 117GB, UD-Q2_K_XL at 97GB, and UD-IQ1_S at 83GB.

Hosted serving also appeared immediately. Ben Burtenshaw pointed to Hugging Face Inference Endpoints at $10/hour, while his follow-up called 2x H200 the ideal setup at about 90 tps with 64 concurrent requests.

Arena put DeepSeek-V4-Flash-High at #7 overall in Frontend Code Arena with a 1,586 score, #3 among open models, and a +154 point jump over the Flash preview. Arena's Pareto chart framed it as the best performance-per-dollar model in its class at $0.14/$0.28 per million tokens.

Other scorecards landed in the same rough shape:

• Vals AI ranked it #3 among open-weight models, with 63.95% accuracy, $0.06 per test, and a +7 point advantage over DeepSeek V4 Pro on the Vals Index.
• Lech Mazur scored it 89.6 on the 940-puzzle Extended NYT Connections benchmark, above Gemini 3.6 Flash at 89.0 and Qwen 3.7 Plus at 74.8.
• Mazur's caveat said older NYT Connections ratings were slightly higher because partial credit scoring had changed.
• Arena added the model to Agent Arena, Frontend Code Arena, Text, and Vision Arena, with Agent Arena scores still pending.

Cline made the updated model free, called it the first flash model it had found performing at SOTA levels, and exposed it through the Cline provider as deepseek-v4-flash.

The rest of the rollout was broad:

• Baseten added it to Model APIs with a 1M-token context window and an 80% to 98% cheaper-than-frontier claim.
• OpenHands made openhands/deepseek-v4-flash free for OpenHands Cloud users for a limited time.
• dabit3 ran it inside Devin Desktop through ACP with OpenCode.
• Ollama added deepseek-v4-flash:0731-cloud and showed a command for using it with Claude Code.
• Venice added the model with a private-access pitch.

Simon Willison got a weak pelican SVG at default reasoning, then a much better one after switching OpenRouter to reasoning_effort high. his cost reply said the default run cost 0.44 cents and the high-effort run cost 0.22 cents, likely because cache helped on the second attempt.

Other hands-on reports clustered around cheap persistence rather than polish:

• kimmonismus said a working game cost 7 cents to create.
• rishdotblog saw data-extraction recall rise from 71% to 79%, versus Luna-medium at 67% at the same cost.
• omarsar0 called the high model unusually strong at front-end artifacts for its price.
• teortaxes reported 475K tokens into a context with no clear sign of fatigue.
• rishdotblog's capacity note also hit repeated 503 "service too busy" errors during the release surge.

The OpenRouter and Nous Portal rollout hit a privacy-policy snag. Teknium first said the model was available in Hermes Agent through Nous Portal and OpenRouter, then said Nous Portal did not allow models whose providers train on user data.

Teknium later said the issue was resolved because more providers were up. In replies, Teknium said the policy did not enable paid endpoints that train on user data through OpenRouter, and another reply said DeepSeek explicitly trains on user data, which was why it had been blocked on Nous Portal.