Qwen 3.8 27B reaches 3,200 TPM at 262K context on two RTX 3090s

• A two-RTX-3090 vLLM stack reported roughly 3,200 tokens per minute across concurrent coding-agent sessions at the full 262,144-token context window, in the dual-3090 vLLM test.
• Prefix caching carried most of that workload. The same test reported a 96.8% cache-hit rate, leaving about 3% of prompt tokens to reach GPU prefill.
• Long-context speed varied sharply by workload: a Strix Halo plus RTX 3090 Ti test reported 153.32 tok/s for code-shaped output at 32K, versus 35.8 tok/s for prose at 200K.
• Application anecdotes now include a DGX Spark deployment that used the model as an implementer agent and a team coding and OCR comparison that described it as competitive in both workloads.
• A mechanically graded local benchmark found a failure mode concentrated in a small slice of hard problems: the benchmark report said reasoning stalls accounted for 3% to 9% of items and disproportionately damaged its scores.

A single-RTX-3090 vLLM repository documents a separate consumer-GPU serving stack. A Strix Halo plus 3090 Ti field note found that a terser chat template cut output tokens 51% without a reported accuracy change. The Unsloth Qwen3.8 model page also lists developer-role support and nested-object tool-call parsing, two features that show up in the agent-focused tests.

The author of the dual-3090 test served a W4A16 AutoRound checkpoint through vLLM 0.27.1 with tensor parallelism set to two, no NVLink, fp8 KV cache, MTP-3 speculative decoding, and prefix caching.

• Context: 262,144 tokens, with about 130K prompt tokens per agent turn.
• Interactive path: 80 to 90 tok/s for one session, against a 106.9 tok/s benchmark ceiling.
• Time to first token: 6.5 seconds mean, with p50 at five seconds or less, versus 140 seconds on a cold prompt.
• Concurrent path: about 3,200 tok/min sustained across chats. Two sessions split decoding to about 42 tok/s each rather than doubling single-session output.

The dual-3090 test attributes its prompt behavior to the cache hit rate: it says decoding stayed at roughly the same rate for a 200K-token prompt as for a short one, while prefill absorbed the context-length cost.

A 159-experiment hardware test offers a more granular explanation. It describes Qwen3.8 as 64 layers with a 3:1 split between Gated DeltaNet and full-attention layers, leaving only 16 layers with context-scaling KV cost. Switching K and V from q8_0 to q4_0 freed 2,176 MiB, enough to place every full-attention layer on the RTX 3090 Ti. That test reported 28% faster prefill and 20% faster generation.

The Strix Halo experiment measured native MTP-3 moving its target-only baseline from 9.474 tok/s to 25.999 tok/s. Its author then reported a further 72% to 140% code-generation gain from adding n-gram drafting, with about a 1% decline on prose and no added VRAM cost.

A modified RTX 2080 Ti post demonstrates how quickly local benchmarks need auditing: its title says 45 tok/s, while its body lists about 456 tok/s for MTP-3. The post uses the official W8A16 artifact, Q8 KV, and a 22 GB modified card with room for roughly 128K context, but the conflicting headline and body leave that throughput figure unresolved.

A custom RTX Pro quant takes a different route, combining MXFP6 weights with MXFP8 cache support and a quantized DFlash2 component. Its author reported daily 256K-context use, contingent on a custom llama.cpp fork.

One DGX Spark deployment placed Qwen3.8-27B behind Claude Code as an implementer agent and claimed it caught bugs created by an Opus 5 orchestrator. Separately, a team report from r/LocalLLaMA said its developers compared the model with GPT Luna for coding and used it on an OCR pipeline, where they judged the OCR output stronger than Gemini 3.5 Flash Lite.

Harness configuration can discard context before the model does. An OpenCode compaction report traced compaction with roughly 30K tokens left to model limits that omitted input and output; its posted configuration sets all three of input, context, and output to the full 131,072-token window.

The local benchmark report ran 2,483 items on a Q8_0 GGUF through LM Studio with two RTX 3090s and 16K context, grading code with unit tests and math with symbolic equivalence rather than an LLM judge.

• MATH-500: 86.4% measured overall. Items where reasoning completed scored 94.3%, while the 45 stalled items scored 6.7%.
• GSM8K: 98.1% for completed reasoning, 52.4% for stalled reasoning.
• ACT: 98.4% for completed reasoning, 57.7% for stalled reasoning.

The harness re-asked blank stalled generations with thinking disabled. Its author also treated the 95.5% HumanEval score as the softest result after a continuation probe found one near-verbatim docstring completion, while calling the 80% MBPP result more trustworthy.