llama.cpp

C/C++ inference and serving toolkit for LLMs

Open-source C/C++ software for running and serving large language models locally, with support for GGUF models and efficient inference on CPU and other accelerators.

Recent stories

1 linked story

releasePRIMARY2026-05-18

llama.cpp adds MTP for Qwen3.6 and pushes local 27B decode to 70-160 tok/s

llama.cpp added multi-token prediction for the Qwen3.6 family, and Unsloth published MTP GGUFs claiming about 1.4-2.2x faster local generation. The update moves Qwen3.6 closer to daily-driver speeds on commodity hardware, though results still vary by ROCm build and quant.