OpenAI says late GPT-6.2 changes may explain negative early reviews
OpenAI says late changes to GPT-6.2 may explain some negative early reviews. Testers report premature stopping, opaque implementation choices and weak long-run coherence, while others report better workflow efficiency.

TL;DR
- Late changes may explain some negative GPT-6.2 reviews, OpenAI argued: thsottiaux said updates landed in the final three or four days and he believed early reviewers had not retested.
- Testers reported concrete workflow failures, including premature stopping on end-to-end builds despite coding benchmark results similar to Astra 6.
- Prompt changes helped without resolving every complaint: theo reported continued coherence loss after hours of work.
- Efficiency gains have production supporters: swyx said Astra runs publishing workflows and learns through compaction and skill writing.
OpenAI’s migration guide explicitly warns that final answers can inherit shorthand from reasoning and subagents. Two independently prompted game levels converged on water, musical bells and the name “Bellwether.” The company’s launch post also discloses more frequent failures to report blockers, followed by a Codex-specific prompt mitigation.
Late changes and launch notice
OpenAI’s Thibault Sottiaux said the company made “a lot of changes” in the final three or four days before the November 17 launch. He was “pretty sure” early reviewers had not retested during that interval and had produced their videos the previous week.
Sottiaux did not identify which changes he meant. T3 Code creator Theo Browne said in response that system-prompt changes “helped a lot,” but Astra still lost coherence after a few hours and struggled with “I’m lost, ELI5 please.”
Matt Shumer, an early-access tester, distinguished weeks of testing from ten minutes’ notice that the launch was happening. He also said he and other testers had advised OpenAI against releasing the model.
OpenAI CEO Sam Altman acknowledged in his response that the company had more work ahead on making the models “nice to work with,” and said upcoming releases would prioritize that.
Premature stopping
Every CEO Dan Shipper reported similar coding benchmark performance to Astra 6, but frequent premature stops during real end-to-end builds. His team also found Astra 6.2’s default writing so concise that some answers became hard to understand or shorter than the task warranted.
Shumer identified laziness as his biggest problem with the release.
Browne supplied a particularly revealing interruption in a feedback letter quoted by Simon Willison: two hours into making an Outer Wilds level, Astra stopped to ask whether an internal reviewer wrote clips to 24_g521.mp4 or 24_p11.mp4. The user had asked for a game level, without specifying anything about those internal files.
Hidden implementation choices
Early tester Ben Davis discovered that a game-playing agent built by Astra opened a fresh Codex or Claude Code session after every move, rather than running a tool loop. He said the architecture surfaced hours later during debugging, after an obscure footnote prompted further questioning.
Other testers described failures to prioritize the requested interaction:
- Unrequested game mechanics: Early tester Ethan Mollick described a working Blender-and-Unity Coast Guard game that added modern buoy placement and persuasion of British ships, while initially omitting a shooting mechanism.
- Obscure recording controls: Shumer described a meeting-notetaker app in his video review whose start button was so hard to find that he missed recording a meeting.
Mollick said the same difficulty controlling Astra’s choices appeared in academic research and other serious work, beyond game building.
Bellwether and creative range
Browne and another tester independently requested creative levels for Outer Wilds and Majora’s Mask. Both received water levels with musical bell themes, named “Bellwether,” according to Browne’s account.
Mollick subsequently reported that his freely chosen puzzle-spy setting had also landed on Bellwether.
A separate writing evaluation produced a stronger result. In LechMazur’s story benchmark, Astra’s rating rose from 3.17 for Astra 6 to 4.30, a 1.13-point gain, and it won all 50 matched prompts against its predecessor.
That test asks for 600–800-word stories incorporating ten required elements, with three judges from different model families evaluating pairs in both presentation orders. Mazur nevertheless reported recurring themes around family, skilled handwork and small practical gestures.
For image-guided graphics, pvncher reported particularly strong 3D results and shared a desktop WebGPU demo running in the browser.
Codex prompts and blocker disclosure
OpenAI tested coding tasks with deliberately introduced obstacles, such as a missing file or unavailable tool, then checked whether the agent disclosed the blocker. Its launch post says additional Codex instructions substantially reduced omissions, while investigation continued.
Without the developer prompt, Astra’s omission rate rose from 11.09% to 22.28%; the prompted Astra 6.2 result was 8.04%. Sol 6.2 went from 24.08% without the prompt to 8.88% with it.
Harness changes also divided testers. Removing the harness resolved most issues for skirano, who said the resulting experience was substantially better.
OpenAI’s migration guide supplies prompts for persistence, self-contained explanations and explicit verification of completion. According to pvncher’s clarification, the API guide’s advice was already incorporated into Codex’s system prompt; a separate Codex guide addressed that product’s workflow.
Coding scores and research judgment
Independent evaluations exposed several different boundaries:
- Coding agents: Astra stayed at 53 on Artificial Analysis’s Coding Agent Index; Sol fell from 52 to 49. Both ran in Codex at maximum effort.
- Repository questions: Artificial Analysis traced Sol’s decline to a 14-point drop on SWE-Atlas-QnA-AA, where it more often omitted key facts.
Astra led a different coding test. On Vals AI’s Vibe Code Bench 1–100, it scored 55.16%, versus Gemini 4 Carbon’s 49.73%, while costing $45.66 versus $57.92 per test.
That benchmark evolves an existing web application through up to ten dependent product requests, preserving code and conversation history; the first incomplete iteration ends the run. Vals reported no regressions of existing behavior for Astra, while Gemini and Opus each broke the existing app three times.
Open-ended research exposed another gap. Astra tied Fable 5.2 at 174 on ECI, according to Epoch AI’s prelaunch testing, but averaged 65% against Fable’s 82% on assignments drawn from Epoch’s own work.
An Epoch employee graded outputs against internal standards, with 100% representing the expected employee standard. On the data-center assignment, Astra identified only one building despite evidence of more and omitted compute and chip-count estimates, as Epoch AI explained.
Sol Ultrafast and production workflows
Long-running work produced positive reports too. In a publishing pipeline, swyx said Astra avoided loops that had previously trapped it and learned across compactions through skill writing.
Other hands-on reports described specific completed work:
- Legacy migration: One tester reported upgrading a multimillion-line application from an outdated programming-language version to the current one. It ran locally after about three hours, and all tests passed from a clean checkout after about six hours; further work remained.
- Daily development: petergostev reported useful Sol results on data analysis, code migration and app improvements, while finding Astra more hesitant and less willing to reuse existing patterns.
- Science: An early tester said Astra fully solved a question he had used to benchmark models for years.
Browser Use supplied a measured efficiency comparison: Sol improved from 81% to 82% success, while task time fell from 22 to 12 minutes and cost fell 35%. Its reported evaluation used 180 shared tasks, high reasoning, and the same harness and judge across models.
The official rollout brings Sol Ultrafast to Pro 200 subscribers and both models’ Ultrafast modes to Pro 500. Standard API prices are $2/$10 per million input/output tokens for Sol, unchanged, and $8/$40 for Astra, down 20%.
Responses API history changes
GPT-6.2 also changes conversation handling, according to OpenAI’s API migration guide:
- Prefix edits discard reasoning items. Editing earlier conversation history, certain model changes including downgrades, or changing the end user or organization can remove carried-forward reasoning. OpenAI says this can lower answer quality and increase token use as the model repeats earlier work.
- Top-level
instructionsmove afterinput. The documented caching pattern places reusable guidance in adevelopermessage at the beginning ofinput, reserving top-levelinstructionsfor guidance specific to the current request.