llama.cpp ships its first stable major release

• Both libraries reached their first major release: ggerganov's follow-up confirms v1.0.0 for llama.cpp and ggml.
• Public API and ABI incompatibilities will require major-version bumps under the announced compatibility policy.
• Large MoEs gained offloading and smarter expert caching, which danielhanchen highlighted alongside v1.
• Cohere separately confirmed llama.cpp support for North Micro Vision.

Speculative decoding's NextN API moves from llama-ext.h into the public llama.h. Official release tags also must contain ggml sources identical to a standalone ggml release.

The November 13 announcement defines two library surfaces:

• libllama: LLM inference in C/C++, with public headers in llama.cpp/include/.
• libggml: The underlying tensor library, with public headers in ggml/include/.

Version numbers follow three rules:

• Major: Breaking changes to the public C API.
• Minor: Backward-compatible features, backend updates, model support or API additions.
• Patch: Bug fixes without API changes.

The compatibility contract deserves top billing. Its boundary remains important: the versioning discussion explicitly excludes experimental llama-ext.h interfaces from the public API guarantee.

For models that exceed VRAM, llama.cpp's v1 implementation adds a host-memory expert cache with:

• CPU execution of cache misses: A configurable moe_cache_cpu_share determines how much missed-expert work runs on the CPU instead of being uploaded to the GPU.
• Replacement policies: LRFU and ClassLRU govern expert eviction.
• Independent CPU work: The CPU portion no longer waits for cached experts.

These controls expose more of the trade-off between CPU computation and moving expert weights onto the GPU.

ComputingForGeeks writer Philip Ndungu tested the new cache format on a four-vCPU AMD EPYC VM with 16 GB RAM and no GPU, using Gemma 3 1B in Q4_K_M.

For throughput, he enabled FlashAttention and ran three repetitions of 512-token prompt processing and 128-token generation. Changing both K and V caches from f16 to f8_e4m3 produced:

• KV-cache memory at 32K context: 183 MiB → 91.5 MiB, −50%.
• Prompt processing: 73.44 → 21.43 tokens/s, −70.8%.
• Generation: 22.06 → 19.55 tokens/s, −11.4%.

All three comparisons hold the v1.0.0 executable constant and change the cache format. The new format is selected with -ctk f8_e4m3 -ctv f8_e4m3; these CPU measurements do not describe performance on GPUs with native FP8 support.

Moving from v0.9.0 to v1.0.0 changes binary compatibility and several server defaults, as Ndungu's migration tests document:

• Library identity: libllama.so.0 → libllama.so.1. Exported llama_* functions fell from 255 to 237, 18 fewer net exports.
• API removals: The release deletes 35 deprecated functions plus llama_set_warmup. Legacy names such as llama_load_model_from_file disappear; their replacements include llama_model_load_from_file.
• Removed flag: --defrag-thold changes from a deprecation warning to a fatal argument error.
• Unauthenticated tools: Enabling tools or MCP on a non-loopback address without an API key changes from a warning to startup refusal. The explicit override is --allow-unauthenticated-tools.
• Private media URLs: Local and private-address media fetches are blocked by default. The new opt-in is --media-allow-private-urls.

The security changes can therefore interrupt existing LAN-hosted tool servers and vision pipelines that fetch images from internal storage.

Each ggml backend can compile as a dynamic library and load at runtime. The release notes describe hardware-specific work across:

• CUDA and Metal: Few-row quantized matrix multiplication and MoE fusion.
• Vulkan: A prefill FlashAttention path.
• SYCL: A device-side MoE scheduler.
• OpenCL: Decomposed FlashAttention for Adreno.
• WebGPU: Additional kernels.
• Hexagon: Expanded operation coverage.

The launch announcement counts more than 150 model architecture variants and support for all mainstream speculative decoding methods. Local execution on phones, laptops and workstations is central to the accompanying announcement.

Beyond the compatibility milestone, v1's model and multimodal additions include:

• Text: Kimi-K3.1-Pro, Ripple-65B-Preview and xing4_0.
• Speech generation: drama-3 TTS, plus an OpenAI-compatible /v1/audio/speech endpoint.
• Vision: MageVL-Cowork and Kimi-K3.1 vision.
• Streaming audio input: Gemma 4 support.

Multimodal subcontexts now share a compute buffer to reduce memory use in models combining vision, audio and generation. The release also removes clip.h and consolidates that infrastructure into mtmd.

Prebuilt binaries selected by the stable tag can still print 1.0.0-dev. Ndungu traced the release's nightly-tag.txt to build b12359, whose nightly pipeline supplies that suffix.

The llama.cpp team at Hugging Face describes two CI investments:

• HF Buckets and HF Jobs: Scaling test capacity for changes landing on master.
• Self-hosted runners: Expanding hardware coverage with support from NVIDIA, AMD, Qualcomm, Arm, IBM and Intel.

Cohere says its recent North Micro Vision integration completes llama.cpp support across its latest architectures. The company also promised more developer support without specifying the next additions.