German consortium releases SOOFI base model trained on 27T tokens

• SOOFI S is a German-English MoE base model trained on about 27T tokens, with eliebakouch's read pointing to a near-Nemotron recipe rather than a new architecture.
• The architecture table is Nemotron-shaped: scaling01's screenshot shows 31.6B total parameters, 3.2B active parameters, 52 layers, 23 Mamba-2 layers, 23 MoE layers, and 6 GQA layers.
• The sharpest criticism targets the sovereignty and benchmark framing, since eliebakouch's follow-up says only about 20% of the data mix changed and JJitsev's benchmark critique attacks the Capability Index.
• The useful artifact is transparency at pretraining scale: eliebakouch's post says SOOFI releases training logs and data-mixture details.
• Access is still preview-shaped, and scaling01's follow-up says a Nemotron 3 Super-equivalent SOOFI model is coming.

The project site says SOOFI S is the first block in an open European model family, aimed at industrial AI on sovereign infrastructure. The Hugging Face tech report puts the model at 31.6B total parameters and roughly 26.68T pretraining tokens, with German up-weighted to as much as 15.3% of the mix. NVIDIA's Nemotron 3 Nano docs are the comparison point: 25T-token pretraining, the same hybrid Mamba-Transformer-MoE family, and an open recipe that made this kind of sovereign replication possible. Fraunhofer's press release frames the target workloads as industrial processes, technical and regulatory document analysis, code generation, and agentic AI systems.

SOOFI S is a small active-parameter model with a big total-parameter shell. The official SOOFI S page lists 30B parameters, roughly 3B active MoE parameters, Mamba architecture, German benchmark wins, 3.3x higher inference throughput per GPU, and beta tests with selected industrial partners from June 2026.

The arXiv pretraining report gives the full shape: 31.6B total parameters, 3.2B active per token, 3.6B including embeddings, 52 layers, 23 Mamba-2 layers, 23 MoE layers, 6 GQA layers, model dimension 2688, 128 routed experts, 6 activated experts per token, and RMSNorm.

For engineers, the interesting artifact is a public full-stack pretraining run on European infrastructure using a known efficient-model recipe.

SOOFI's most debated detail is how close it sits to NVIDIA's Nemotron 3 Nano. eliebakouch's read says SOOFI uses the same architecture and most hyperparameters as Nemotron 3 Nano, trains for roughly 26T tokens, and overlaps with Nemotron's mixture by about 80%.

NVIDIA's own Nemotron 3 Nano training recipe describes an open, reproducible pipeline with pretraining, SFT, and RL stages. Its pretraining page says the base model is trained from scratch on 25T tokens with Megatron-Bridge, a 52-layer hybrid Mamba-Transformer-MoE architecture, open-source-only recipe data, and a two-phase curriculum.

That makes SOOFI a valuable replication exercise, not a clean architecture advance. giffmana's reply put it bluntly: the model basically is Nemotron 3 Nano, but it is a good exercise for getting started.

SOOFI changed the data, not the core model shape. eliebakouch's follow-up argued that calling the model sovereign is overstated because the change is roughly 20% of the data mixture, although training on a German cluster is still a real sovereignty distinction.

The mixture chart shared by JJitsev compares SOOFI and Nemotron across diverse pretraining and high-quality annealing phases. It also lists SOOFI-added sources such as FinePDFs, ClimbMix, UltraData-Math, HPLT-3 DE, Genios, German Commons, FineWiki, QA-base, Multilingual Reasoning, and OlmoOCR.

JJitsev's follow-up raised the unresolved technical question: the small delta over Nemotron 3 Nano may come from about 2T extra tokens rather than a better mixture.

The public disagreement moved quickly from model quality to eval construction. scaling01's post called SOOFI small, still worse than Qwen3.5, and comparable to Nemotron 3 Nano.

The official side claims German strength. The SOOFI S page says the base model ranks first on all German benchmarks, while the arXiv report says its Capability Index averages five benchmark groups: code, mathematics, knowledge, reading and commonsense, and reasoning and science.

Critics focused on that index. JJitsev's benchmark critique said the Capability Index makes Qwen 3 32B look weaker than Teuken, and JJitsev's reply questioned whether that score should be trusted when it ranks Qwen below models he described as mostly broken.

The sovereignty claim is strongest on infrastructure. teortaxesTex quoted SOOFI's claim that the model was trained end-to-end on Deutsche Telekom's Industrial AI Cloud in Munich.

Deutsche Telekom's Industrial AI Cloud launch coverage described the platform as one of Europe's largest AI factories and named the SOOFI project as a sovereign European language-model effort on that cloud. DFKI's SOOFI project page lists the project goal as an open-source larger AI language model, followed by a reasoning model and initial AI-agent use cases.

The consortium footprint is broad. The arXiv report names KI Bundesverband as coordinator, BMWE as funder, and affiliations including DFKI, Fraunhofer IAIS, Fraunhofer IIS, TU Darmstadt, Universität Würzburg, Berliner Hochschule für Technik, L3S, Lamarr, ellamind, hessian.AI, and Merantix Momentum.

SOOFI's release surface is still uneven. The project site says the current focus is practical testing with industrial partners and that a general release for direct use has not yet happened.

The Instruct Preview card requires users to share contact information, while the Rhine Preview and Isar Preview cards label the checkpoints as preview/internal, say weights and metadata may still change, and list custom license terms with TODO text.

Serving artifacts already exist around those previews. The Rhine GGUF card says SOOFI-S is a custom hybrid Mamba-2/MoE architecture with its own modeling code, and that GGUF conversion requires a llama.cpp build that understands the architecture. scaling01's follow-up adds the roadmap tease: a Nemotron 3 Super-equivalent SOOFI model is next.