LocalLLaMA post reports Qwen 3.8 27B reasoning loops caused most errors

• A 2,483-item local run of Qwen 3.8 27B on two RTX 3090s reported 96.7% on GSM8K, 86.4% on MATH-500, 95.5% on HumanEval, and 80.0% on MBPP, as on_line187's benchmark report records.
• Between 3% and 9% of inputs triggered a non-terminating reasoning path, and on_line187's result split shows that this small slice produced most of the failures.
• Local deployment results depend sharply on the serving stack: TrifleHopeful5418's llama.cpp tests measured 153.32 tokens per second for code-shaped output at 32K context, while a separate V3 GGUF report documented first-token crashes on one dual-AMD Windows setup Equivalent-Ear-8016's rollback note.

Qwen's official model card describes a compact dense vision-language model with flexible thinking control. Qwen's thinking documentation says a capped thinking_budget should stop reasoning and generate an answer; in this LM Studio run, some requests instead exhausted reasoning_content without an answer, so on_line187 re-ran them with thinking disabled.

In on_line187's benchmark report, Qwen 3.8 27B ran as a Q8_0 GGUF in LM Studio with full offload, 16K context, and two RTX 3090s. The 2,483 tests took 9.1 hours and used executed unit tests for code plus SymPy equivalence for math, with no LLM judge.

• GSM8K: 1,275 of 1,319, 96.7%
• MATH-500: 432 of 500, 86.4%
• HumanEval: 95.5% mean across three runs
• MBPP test split: 400 of 500, 80.0%

on_line187's result split separates each evaluation by whether the fallback fired. The stalled subset was tiny, but its accuracy collapsed:

• GSM8K: 98.1% when reasoning completed, 52.4% after a stall
• MATH-500: 94.3% when reasoning completed, 6.7% after a stall
• ACT, a separate run: 98.4% when reasoning completed, 57.7% after a stall

The MATH-500 fallback returned three correct answers from 45 stalled items, versus 429 from 455 completed ones. The author estimates that behavior reduced the aggregate MATH-500 score by about eight points, while stalled GSM8K requests consumed 60 of the run's 245 minutes.

on_line187's contamination probe gave the model the first 35% of an item, asked it to reconstruct the rest, then compared that continuation with endings from unrelated items. The comparison was intended to separate ordinary shared phrasing from suspiciously exact recall.

• HumanEval continuation similarity was 0.387 against a 0.080 cross-item baseline, a 0.307 gap. One item reached 0.896 similarity.
• MBPP measured 0.187 against a 0.129 baseline, a 0.058 gap.
• Three HumanEval runs produced 94.5%, 97.6%, and 94.5%, a 3.1-point span on 164 problems.

The author calls the HumanEval number the softest result, while noting that prefix conditioning inflates the continuation test and does not alone establish memorization.

on_line187 found four harness defects that depressed the reported scores in the detailed notes: omitted prompt helpers caused NameErrors, an eval blocklist rejected a task designed to build and evaluate an expression, leading-whitespace stripping broke Python indentation, and math_verify failed on Windows with Python 3.14 and on equivalent fractions.

The first three surfaced through grader self-tests. The same report says reasoning_effort: "none" disabled thinking in LM Studio, while its attempted chat_template_kwargs: {"enable_thinking": false} setting had no effect.

A 159-experiment llama.cpp account split one 27B model between a Strix Halo system running Vulkan and an RTX 3090 Ti running CUDA. It measured 153.32 tokens per second for code-shaped generation at 32K context and 87.74 at 200K; prose at 200K ran at 35.8 tokens per second.

Moving K and V cache from Q8_0 to Q4_0 freed 2,176 MiB, enough to move all full-attention layers to the 3090 Ti. The author measured 28% faster prefill and 20% faster generation, and attributed the leverage to Qwen 3.8's 16 full-attention layers out of 64.

xrailgun's NInfer post headlines 45 tokens per second for MTP3 on a modified 22GB RTX 2080 Ti, but its body lists about 456 tokens per second for the same mode. The linked NInfer repository says the groupwise-int profile is supported but lacks a published benchmark campaign.

Equivalent-Ear-8016 reported that Unsloth's V3 Qwen 3.8 27B quants crashed at the first generated token on Windows with an RX 9070 XT plus RX 7800 XT under Vulkan. Changing the context limit and trying other V3 quants did not resolve the issue; the author said the older V2 revision 408fcc1807ab worked on that configuration.