Qwen Benchmarks SGLang Model serving Inference Optimization GPU Infrastructure llama.cpp

DFlash

Diffusion-based speculative decoding for faster LLM inference

Speculative-decoding inference acceleration software for large language models. It uses trained draft/speculator models and is integrated with serving and local-inference stacks including SGLang and llama.cpp to accelerate token generation without changing output.

Recent stories

3 linked stories

workflowSECONDARY2026-07-16

LocalLLaMA users report near-6x Qwen 3.6 27B speedups with MTP

A LocalLLaMA benchmark on Qwen 3.6 27B and RTX 6000 PRO reports near-6x speedups from MTP, DFlash, and n-gram drafting. Related tests cover remote prefill, NUMA offload, GPU clock tuning, and llama.cpp Gemma 4 support.

releaseSECONDARY2026-06-15

SGLang adds DFlash and Spec V2 with 4.3x Qwen3.5-397B-A17B throughput

LMSYS and Modal shipped DFlash plus Spec V2 in SGLang, claiming 4.3x baseline throughput and 1.5x native MTP on Qwen3.5-397B-A17B. It cuts latency and serving cost for very large open models.

releasePRIMARY2026-05-10

DFlash adds Qwen3-8B speculator with 82.2% first-token acceptance

Posts said Qwen3-8B now has a DFlash speculator with 82.2% first-token acceptance and 3.74 accepted tokens per step, alongside broader DFlash claims of over 6x lossless acceleration. It matters because the release turns a decoding paper into a concrete speculative-inference artifact engineers can test against existing Qwen stacks.