Tencent releases 1-bit and 4-bit GGUF weights for 295B Hy3 single-GPU runs

• Tencent shipped 1-bit and 4-bit GGUF builds for Hy3, and TencentHunyuan's launch post says the 295B model now runs through llama.cpp with MTP on dramatically lower hardware.
• The practical 1-bit download sits in the high-80s to low-90s GB range: teortaxes saw 91 GB, Emad Mostaque called it 88 GB, and the Hy3-GGUF README lists IQ1_M at roughly 83 GiB before MTP overhead.
• Quality did not fall off a cliff in the posted agent chart: Emad Mostaque's benchmark post highlighted 75.4% SWE-Bench Verified and 53.9% SWE-Bench Pro for the 1-bit build.
• A local Atomic Chat run beat the cloud API on elapsed time, with Rohan Paul's post reporting 76.9K tokens in 15.5 minutes locally versus 75.1K tokens in 34.3 minutes through the cloud API.
• The llama.cpp path is already code, not just a model card: the LocalLLaMA post points to PR #25395 for hy_v3 support and MTP speculative decoding.

The Hy3-GGUF README includes exact llama-server commands, a hardware matrix, and four quantization recipes. The llama.cpp PR wires Hy3's MTP head into --spec-type draft-mtp. The base Hy3 model card says the model is Apache 2.0, 256K context, 295B total parameters, 21B active parameters, and a 3.8B-parameter MTP layer.

Hy3 is a 295B-parameter MoE with 21B active parameters and a 3.8B-parameter MTP layer, according to Tencent's base model card. The same card lists 80 non-MTP layers, one MTP layer, 192 experts with top-8 routing, 64 attention heads, 256K context, and Apache 2.0 licensing.

Tencent's original deployment path for BF16 Hy3 was still server-class: the model card recommends 8 GPUs for vLLM or SGLang serving. The GGUF release changes the local path, not the base architecture.

Sina's reprint of Tencent's announcement says the BF16 weights are 598 GB, while the new IQ1_M build is 85.5 GiB and the Q4_K_M build is 169.9 GiB. The Hy3-GGUF page currently shows larger user-facing file sizes because MTP and packaging overhead matter.

• BF16 original: 598 GB, per Tencent's announcement reprint.
• IQ1_M: 85.5 GiB in the announcement, about 83 GiB in the README's deployment notes, and 89.4 GB to 91.8 GB in the Hugging Face file card.
• Q4_K_M: 169.9 GiB in the announcement, about 166 GiB in the README's deployment notes, and 182 GB to 185 GB in the Hugging Face file card.
• GPTQ Int4: Tencent also describes a vLLM-oriented Int4 path for server deployment in the same announcement reprint.

The gap between 85.5 GiB and 91.8 GB is not a contradiction in practice. GiB versus GB, MTP variants, and file-card packaging explain the range that teortaxes and Emad Mostaque rounded to 91 GB and 88 GB.

The llama.cpp PR adds Hy3 as hy_v3 / HYV3ForCausalLM, with support for a 299B MoE, 80 layers, and one MTP layer. The PR reuses llama.cpp's existing speculative driver instead of adding a separate one.

The useful implementation pieces are concrete:

• LLM_ARCH_HY_V3 registration, including n_layer_nextn and six NEXTN tensor entries.
• src/models/hy-v3.cpp, with sigmoid routing, expert-bias MoE, ungated shared expert, q/k norms, and the MTP graph.
• conversion/hunyuan.py, with --mtp and --no-mtp conversion paths.
• Runtime use of --spec-type draft-mtp for the MTP head.

The Hy3-GGUF README lists two server commands: plain serving, and MTP self-speculative serving with --spec-draft-n-max 3, q8 KV flags, flash attention, and -c 65536 context.

The chart attached to Emad Mostaque's benchmark post gives the cleanest public comparison between BF16, Q4_K_M, GPTQ Int4, and IQ1_M on agent tasks.

| Benchmark | BF16 | Q4_K_M | Q4 delta | IQ1_M | IQ1 delta |
| --- | ---: | ---: | ---: | ---: | ---: |
| nl2repo | 57.9 | 54.7 | -3.2 | 53.1 | -4.8 |
| mcp_atlas | 79.1 | 78.8 | -0.3 | 76.9 | -2.2 |
| SWE-Bench Verified | 78.0 | 77.2 | -0.8 | 75.4 | -2.6 |
| SWE-Multilingual | 75.8 | 75.7 | -0.1 | 72.7 | -3.1 |
| SWE-Pro | 57.9 | 55.1 | -2.8 | 53.9 | -4.0 |
| AA_LCR | 73.4 | 72.4 | -1.0 | 72.8 | -0.6 |
| Frontier | 74.8 | 72.8 | -2.0 | 71.8 | -3.0 |

Christmas came early for local-LLM people: the 1-bit model gives up only 2.6 points on SWE-Bench Verified in this chart while moving a 300B-ish model into single-96GB-card territory.

The Hy3-GGUF README describes the release as mixed-precision quantization, not one flat bit-width across every tensor. Tencent/AngelSlim is the broader compression toolkit behind the work.

The recipes spend bits where the README says low precision hurts most:

• Attention tensors and token embedding / output head stay higher precision, often q8_0, q4_K, or q6_K.
• Shared experts stay around q5_K to q6_K because they are active on every token.
• Routed experts carry the aggressive low-bit compression because they dominate file size.
• IQ1_M uses iq1_m for most layers, with ffn_down and sensitive layers lifted to higher precision.
• MTP recipe variants add the nextn block, while MTP experts stay in K-quant types because the imatrix covers trunk layers.

The four shipped recipe targets are hyv3_q4km_recipe.txt, hyv3_q4km_mtp_recipe.txt, hyv3_iq1m_recipe.txt, and hyv3_iq1m_mtp_recipe.txt.

Atomic Chat's local run used 4 RTX 5090 GPUs with 128 GB VRAM against the Hy3 cloud API, according to Rohan Paul's post. Both runs generated Flappy Bird, Arkanoid, and Snake from one-shot prompts, with no crashes or obvious visual gap reported in the post.

The timing was the part that traveled:

• Hy3 1-bit local: 76.9K tokens in 15.5 minutes.
• Hy3 cloud API: 75.1K tokens in 34.3 minutes.
• Reported speedup: 2.2x faster locally.

The linked Atomic Chat GitHub repo presents the app as an offline local ChatGPT alternative with an OpenAI-compatible server on localhost:1337/v1. Rohan Paul also flagged the next test question: whether reasoning degrades before coding as 1-bit compression gets pushed harder.

Hy3 hit the top of OpenRouter's weekly LLM leaderboard before the GGUF release cycle finished. TencentHunyuan's post shows Hy3 free at 6.13T tokens for the week, ahead of MiMo-V2.5 at 5.95T and DeepSeek V4 Flash at 5.22T.

The OpenRouter screenshot is usage, not quality. It still explains why the GGUF drop landed with unusual local-LLM interest: developers had already been hammering the cloud route.

Tencent's cloud UI screenshot in TencentHunyuan's reply lists Hy3 with 256K context, deep-thinking and text-generation tags, 1,000,000 TPM, and 60 RPM. The same screenshot shows billing at 1.0 yuan per million input tokens, 0.25 yuan per million cache-hit tokens, and 4.0 yuan per million output tokens.

The reply adds one operational detail: post-paid billing is off by default, and calls fail after free resources are exhausted unless post-paid billing is enabled.