llama.cpp

A LocalLLaMA post ran a 122B Qwen model on an older laptop with llama.cpp. The run had very long load and generation times, while another report put Qwen 3.6 35B at 21 tok/s on a Radeon 7600 after ROCm tuning.

A LocalLLaMA benchmark on Qwen 3.6 27B and RTX 6000 PRO reports near-6x speedups from MTP, DFlash, and n-gram drafting. Related tests cover remote prefill, NUMA offload, GPU clock tuning, and llama.cpp Gemma 4 support.

Tencent released 1-bit and 4-bit GGUF builds for its 295B Hy3 model with llama.cpp support and MTP. Posts cite 88–92GB local runs and SWE-Bench scores of 75.4% Verified and 53.9% Pro.

llama.cpp now has an official website and a single-line installer that provides one `llama` entrypoint for running, serving, and agent integrations. The packaging change simplifies local setup while reusing GGUF models already on disk.