Nanbeige releases Apache 2.0 Nanbeige4.3-33B with a reported 512K context

• Nanbeige4.3-33B brings 33B dense parameters, Apache 2.0 weights and a reported 512K context window, as kimmonismus reported.
• Coding results include a 7.9-point Terminal-Bench 2.1 lead and a 23-point DeepSWE lead over Qwen3.8-27B in the release benchmark chart.
• The wins have exceptions: the full comparison puts Qwen ahead on SWE-Bench Pro and GDPval, and Gemma4-31B ahead on GPQA Diamond.
• Looping concerns come from earlier Nanbeige experience: teortaxesTex expected broadly credible results while warning about fragility and “doom looping.”

The model card budgets up to 262,144 generated tokens for chat and retains previous reasoning during benchmark runs. Its serving examples configure four-way tensor parallelism, and its tool template favors XML over JSON.

Nanbeige published an agent-tuned checkpoint alongside Nanbeige4.3-33B-Base. The agent checkpoint uses BF16 weights, putting raw weight storage at approximately 66 GB for 33 billion parameters, before runtime overhead.

The configuration specifies:

• LlamaForCausalLM, with 52 layers and a hidden width of 7,168.
• 56 query heads and eight key/value heads, with a head dimension of 128.
• max_position_embeddings and seq_length both set to 524,288.

Those dimensions imply 104 GiB of KV cache for one full-length sequence using a conventional, unquantized BF16 cache: 2 × 52 × 8 × 128 × 524,288 × 2 bytes. That calculation excludes model weights and runtime buffers.

The lab says it evaluated every model at its highest thinking effort, keeping the harness and inference hyperparameters consistent across models within each benchmark. These are release-team measurements.

Scores below run Qwen3.8-27B → Nanbeige4.3-33B, following the published comparison:

• Terminal-Bench 2.1: 73.0 → 80.9, +7.9 points.
• Terminal-Bench 4.0: 6.3 → 23.8, +17.5 points.
• DeepSWE: 42.5 → 65.5, +23.0 points.
• SWE-Bench Pro: 61.7 → 60.3, −1.4 points.
• SWE-Atlas-QnA: 41.9 → 50.8, +8.9 points.
• NL2Repo: 48.3 → 53.0, +4.7 points.
• Scicode: 46.7 → 52.6, +5.9 points.
• SkillsBench: 56.4 → 61.1, +4.7 points.

Outside coding, Qwen leads GDPval at 86.4 versus 84.8. Gemma4-31B leads GPQA Diamond at 91.3 versus Nanbeige’s 89.5 in the same comparison.

JobBench evaluates 35 occupations through Claude Code equipped with a search tool, according to the evaluation notes.

Nanbeige names the harnesses and budget exceptions in its benchmark footnotes, a welcome level of detail:

• Reasoning history: Nanbeige runs with preserve_thinking=true, retaining reasoning from previous assistant turns.
• Terminal tasks: Terminal-Bench 2.1 uses Terminus 2; Terminal-Bench 4.0 and DeepSWE use mini-swe-agent v2.4.6.
• Repository generation: NL2Repo runs through Claude Code.
• Codebase questions: SWE-Atlas-QnA uses mini-swe-agent v2.4.6, with GPT-6 Sol at medium reasoning effort judging answers against 1,304 task-specific rubrics.
• Toolathlon-Verified: Individual model requests have no max_tokens limit.
• Claw-Eval: Results cover only the General subset, comprising 157 single-turn user-interaction tasks.
• GDPval: Scoring uses revised rubrics.
• YC-bench: Each model call receives only the most recent ten turns of user dialogue as historical context.

The quickstart exposes separate controls for generating reasoning and keeping it in conversation history:

• enable_thinking defaults to enabled and controls reasoning for the current response.
• preserve_thinking controls retention of earlier reasoning. Nanbeige recommends False for general chat and question answering, and True for multi-turn tools, office tasks and coding agents.
• Passing tools activates the tool-use template.
• tool_call_format="xml" is the lab’s preferred format; json remains available for compatibility.

The vendor’s sampling settings also differ by workload:

| Workload | Temperature | Top-p | Maximum new tokens |
| --- | ---: | ---: | ---: |
| Agentic and tool use | 1.0 | 0.95 | 65,536 |
| Reasoning and chat | 0.6 | 0.95 | 262,144 |

The vLLM and SGLang examples both specify qwen3 as the reasoning parser and qwen3_coder as the tool-call parser.

Drawing on earlier Nanbeige experience, teortaxesTex expected the benchmark results to be largely accurate but warned that the models could be fragile and at risk of “doom looping.” The accompanying “best dense model” assessment was explicitly conditional on those results holding up.

A future Nanbeige4.3-3B is planned, as testingcatalog reported, and the model card positions the 33B checkpoint as a teacher for distilling smaller agents.

In an official architecture reply, the lab said upcoming Nanbeige4.3-3B and Nanbeige4.5-3B models will use a Looped Transformer architecture. It attributes that choice to experiments finding a better quality–compute trade-off at smaller model sizes.