Skip to content
AI Primer
release

Unsloth Desktop adds AMD NPU inference for Strix, Krackan and Gorgon

Unsloth Desktop added AMD NPU support for Strix, Krackan and Gorgon devices. The team claims 2–4× lower energy use per token than GPUs and demonstrates local coding-agent endpoints.

4 min read
Unsloth Desktop adds AMD NPU inference for Strix, Krackan and Gorgon
Unsloth Desktop adds AMD NPU inference for Strix, Krackan and Gorgon

TL;DR

AMD's 35B-model demo needs about 28 GB available to the NPU at full context, translating to a 64 GB Windows machine under the default 50% shared-memory allocation. Unsloth's settings documentation exposes four NPU power modes, from Power saver to Turbo, plus image-resolution controls for vision models.

XDNA 2 hardware

The supported processor families are:

  • Ryzen AI 300: Strix Point and Krackan Point.
  • Ryzen AI Max: Strix Halo.
  • Ryzen AI 400: Gorgon Point.

Unsloth also promises support for future AMD devices with integrated NPUs. The current requirement is an XDNA 2 or newer NPU, running Windows 11 or Linux x64.

Linux has additional prerequisites, listed in the installation guide:

  • Kernel 7.0 or newer with amdxdna, or an older kernel with amdxdna-dkms.
  • Updated NPU firmware.
  • Unlimited memlock.

The model picker's Enable NPU flow downloads FastFlowLM and checks the driver. NPU builds appear alongside GPU models.

Energy and model speeds

Similar generation speed to GPUs is part of danielhanchen's pitch. Unsloth's published model table gives these figures:

| NPU model | Recommended system RAM | Generation | Time to first token |
| --- | --- | --- | --- |
| Qwen3.6-35B-A3B | 64 GB | 17.2 tok/s | 3.13 s |
| Qwen3.8-27B (MTP) | 48 GB | 2.0 tok/s | 3.80 s |
| Gemma4-12B-IT | 32 GB | 6.8 tok/s | 1.70 s |
| Qwen3.5-9B | 32 GB | 9.5 tok/s | 1.11 s |
| Qwen3-VL-4B-Instruct | 16 GB | 20.5 tok/s | 1.05 s |
| LFM3-1.7B | 16 GB | 40.7 tok/s | 0.12 s |

The guide does not specify the benchmark machine, prompt length, or a matched GPU energy baseline. Its default context setting sizes the context window to the RAM available when the model loads.

Coding-agent endpoint

The endpoint is the headline feature for coding-agent users. OpenCode and Pi are among the clients named in danielhanchen's follow-up, while AMD describes the serving path as Unsloth Desktop API → Lemonade and FastFlowLM → XDNA 2 NPU.

AMD says the OpenAI-compatible interface connects more than 100 compatible agentic applications and workflows, including Cline, Open WebUI and Hermes Agent.

The documented server invocation is:

  • Client connection: /v1/chat/completions, using the base URL and API key printed by the command.
  • Local access: the managed Lemonade server accepts connections only from the PC and generates a fresh authentication key at each start.
  • Runtime lifecycle: unloading releases the model from the NPU; quitting Unsloth stops the managed runtime.

Unsloth's broader training and image/video-generation workloads remain on supported Radeon GPUs, according to the guide.

Q4NX export

Only selected models work today, according to danielhanchen's reply. The NPU runtime consumes 4-bit Q4NX builds, and supported GGUF models can be converted through Unsloth's Export page.

The conversion UI breaks the process into four steps:

  1. Pick a GGUF model from Hugging Face or the local device.
  2. Select its file. The UI lists Q4_K_M as the standard path, with Q4_0 and Q4_1 available where compatible.
  3. Confirm the original model and output directory.
  4. Convert to write the Q4NX files.

The conversion documentation says conversion runs on the CPU, so the preceding training can use any GPU. It describes loading converted models under Trained, while its FAQ still says loading custom conversions directly from the NPU model list is not yet supported.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 4 threads
TL;DR2 posts
Energy and model speeds1 post
Coding-agent endpoint1 post
Q4NX export2 posts
Share on X