Magic says its pretraining recipe matches DeepSeek V4 Pro with 50x less compute

• Magic says a scaling-law estimate reaches DeepSeek V4 Pro Base's heldout-loss level with roughly 50 times less pretraining compute, per Magic's announcement.
• A 10x larger run cost about $4 million on GB200s and outperformed publicly available open base models on Magic's perplexity evaluations, according to Magic's scale-up post.
• The comparison rests on private heldout code, math, and research-text loss, with tokenizer-normalized bits-per-byte as the central metric, as Magic's evaluation note describes.
• Magic attributes the gain to tens of combined architecture, optimizer, objective, data, and bug-fix changes rather than a single technique, as Magic's methods post puts it.
• Its next stated work is long-context RL for agent test-time learning, latent-knowledge alignment training, and more pretraining, according to Magic's roadmap.

Magic built evaluation sets with a different parser and OCR pipeline from the one used for training, according to its September 8 research update. The DeepSeek V4 technical report describes the baseline as a 1.6T-parameter MoE with 49B active parameters and more than 32T pretraining tokens. Magic also says on its recruitment page that it has thousands of GB200s and $515 million in funding.

Magic's 50x figure is a scaling-law estimate from heldout bits-per-byte loss curves, not a public task benchmark. Its post says lower bits-per-byte normalizes away tokenizer differences and fits a curve for the compute required to reach a given capability level.

The accounting uses the common 6·N·D approximation, where N is activated parameters and D is pretraining tokens. Magic says it used checkpoint tensor headers to estimate competitors' active parameters, and left sequence-dimension costs out because their exact contribution depends on undisclosed sequence-length distributions.

Magic says it then scaled the recipe 10x to about $4 million of GB200 compute and beat the publicly available open base models it tested on perplexity. The post lists its Current_e24 run at 1.63e24 under 6·N·D, while listing DeepSeek V4 Pro at 9.67e24 under the same approximation.

The headline and the larger run answer different points on the fitted curves. Magic does not disclose the parameter counts or training-token counts for its own runs in the table.

The result is measured on loss rather than prompted generations, because Magic says base models are prompt-sensitive and loss offers a smoother pre-RL comparison. Its reported test material includes:

• its own codebase and private codebases acquired from startups
• private competition math problems, with Kimi K3-generated chain-of-thought walkthroughs filtered for correct answers
• recent, low-citation research papers

Magic says it removed vendored open-source code and documents matching its corpus through a 96-character normalized-text window or a sensitive Jaccard threshold. It also says it cannot decontaminate the open models' internet-based test exposure, a condition it argues would favor those baselines.

The company calls the improvement a multiplicative outcome of changes across model architecture, optimizer, training objective, and data curation. It says fixing minor bugs also compounded into a compute gain.

Magic tested each candidate change by training three models across two orders of magnitude of compute, retaining changes whose power-law fit suggested a benefit at scale. It says it ran one-tenth-scale experiments every few weeks and full hero runs every few months.

An excerpt shared in dwarkesh_sp's post draws a useful boundary around this kind of claim: model advances can make larger runs feasible by removing stability, memory, bandwidth, and speed constraints, even where their direct fixed-capability FLOP savings are limited. Magic's post does not separate the contribution of individual changes.

Magic describes itself as likely the smallest team training trillion-parameter models. Its public careers page separately says it has thousands of GB200s, while giving no headcount.

The company frames the program around code models and autonomous AI research, combining frontier pretraining with domain-specific RL, ultra-long context, and inference-time compute.

Magic says its pretraining and long-context work is mature enough to shift emphasis to long-horizon RL. Its stated research list is:

1. Exploration and credit assignment for long-horizon RL, plus the systems work to scale it.
2. Alignment training against narrowly elicited latent knowledge.
3. Further pretraining improvements.

The blog describes the first item as training agents to continue learning after deployment through long-context interaction. It says a model release is planned, without giving a date.