Unsloth Desktop adds AMD NPU inference for Strix, Krackan and Gorgon
Unsloth Desktop added AMD NPU support for Strix, Krackan and Gorgon devices. The team claims 2–4× lower energy use per token than GPUs and demonstrates local coding-agent endpoints.

TL;DR
- Unsloth Desktop now runs supported models on AMD XDNA 2 NPUs, covering Strix, Krackan and Gorgon devices, according to danielhanchen's launch post.
- Energy efficiency is the pitch: UnslothAI's announcement claims 2–4× less energy per generated token than GPUs.
- Local coding agents can use the endpoint: danielhanchen's follow-up names OpenCode and Pi among the clients.
AMD's 35B-model demo needs about 28 GB available to the NPU at full context, translating to a 64 GB Windows machine under the default 50% shared-memory allocation. Unsloth's settings documentation exposes four NPU power modes, from Power saver to Turbo, plus image-resolution controls for vision models.
XDNA 2 hardware
The supported processor families are:
- Ryzen AI 300: Strix Point and Krackan Point.
- Ryzen AI Max: Strix Halo.
- Ryzen AI 400: Gorgon Point.
Unsloth also promises support for future AMD devices with integrated NPUs. The current requirement is an XDNA 2 or newer NPU, running Windows 11 or Linux x64.
Linux has additional prerequisites, listed in the installation guide:
- Kernel 7.0 or newer with
amdxdna, or an older kernel withamdxdna-dkms. - Updated NPU firmware.
- Unlimited
memlock.
The model picker's Enable NPU flow downloads FastFlowLM and checks the driver. NPU builds appear alongside GPU models.
Energy and model speeds
Similar generation speed to GPUs is part of danielhanchen's pitch. Unsloth's published model table gives these figures:
| NPU model | Recommended system RAM | Generation | Time to first token |
| --- | --- | --- | --- |
| Qwen3.6-35B-A3B | 64 GB | 17.2 tok/s | 3.13 s |
| Qwen3.8-27B (MTP) | 48 GB | 2.0 tok/s | 3.80 s |
| Gemma4-12B-IT | 32 GB | 6.8 tok/s | 1.70 s |
| Qwen3.5-9B | 32 GB | 9.5 tok/s | 1.11 s |
| Qwen3-VL-4B-Instruct | 16 GB | 20.5 tok/s | 1.05 s |
| LFM3-1.7B | 16 GB | 40.7 tok/s | 0.12 s |
The guide does not specify the benchmark machine, prompt length, or a matched GPU energy baseline. Its default context setting sizes the context window to the RAM available when the model loads.
Coding-agent endpoint
The endpoint is the headline feature for coding-agent users. OpenCode and Pi are among the clients named in danielhanchen's follow-up, while AMD describes the serving path as Unsloth Desktop API → Lemonade and FastFlowLM → XDNA 2 NPU.
AMD says the OpenAI-compatible interface connects more than 100 compatible agentic applications and workflows, including Cline, Open WebUI and Hermes Agent.
The documented server invocation is:
- Client connection:
/v1/chat/completions, using the base URL and API key printed by the command. - Local access: the managed Lemonade server accepts connections only from the PC and generates a fresh authentication key at each start.
- Runtime lifecycle: unloading releases the model from the NPU; quitting Unsloth stops the managed runtime.
Unsloth's broader training and image/video-generation workloads remain on supported Radeon GPUs, according to the guide.
Q4NX export
Only selected models work today, according to danielhanchen's reply. The NPU runtime consumes 4-bit Q4NX builds, and supported GGUF models can be converted through Unsloth's Export page.
The conversion UI breaks the process into four steps:
- Pick a GGUF model from Hugging Face or the local device.
- Select its file. The UI lists
Q4_K_Mas the standard path, withQ4_0andQ4_1available where compatible. - Confirm the original model and output directory.
- Convert to write the Q4NX files.
The conversion documentation says conversion runs on the CPU, so the preceding training can use any GPU. It describes loading converted models under Trained, while its FAQ still says loading custom conversions directly from the NPU model list is not yet supported.