Qwen Benchmarks SGLang Model serving Inference Optimization GPU Infrastructure llama.cpp

DFlash

Block Diffusion for Flash Speculative Decoding

DFlash is an open-source speculative-decoding tool and draft-model framework for accelerating autoregressive LLM inference. It uses lightweight block-diffusion drafters to propose a block of tokens in parallel, then verifies them with the target LLM, with integrations or usage paths for SGLang, vLLM, Transformers, and MLX.

Recent stories

3 linked stories

workflowSECONDARY2026-07-16

LocalLLaMA users report near-6x Qwen 3.6 27B speedups with MTP

A LocalLLaMA benchmark on Qwen 3.6 27B and RTX 6000 PRO reports near-6x speedups from MTP, DFlash, and n-gram drafting. Related tests cover remote prefill, NUMA offload, GPU clock tuning, and llama.cpp Gemma 4 support.

releaseSECONDARY2026-06-15

SGLang adds DFlash and Spec V2 with 4.3x Qwen3.5-397B-A17B throughput

LMSYS and Modal shipped DFlash plus Spec V2 in SGLang, claiming 4.3x baseline throughput and 1.5x native MTP on Qwen3.5-397B-A17B. It cuts latency and serving cost for very large open models.

releasePRIMARY2026-05-10

DFlash adds Qwen3-8B speculator with 82.2% first-token acceptance

Posts said Qwen3-8B now has a DFlash speculator with 82.2% first-token acceptance and 3.74 accepted tokens per step, alongside broader DFlash claims of over 6x lossless acceleration. It matters because the release turns a decoding paper into a concrete speculative-inference artifact engineers can test against existing Qwen stacks.