Older RTX GPUs reportedly deliver roughly 50% faster matrix multiplication

• Matrix multiplication gained roughly 50% throughput across BF16, FP16 and TF32 in an independent test, with identical outputs.
• Local MoE inference improved much less: the tester’s follow-up reported under 10% faster prefill and essentially unchanged decode.
• Separate open-source projects target Turing, Ampere and experimental Ada GPUs, as documented in the project roundup.

The Ampere README lists an RTX 3080 Ti variant that completes the unlock but keeps half-rate BF16. The Ada fork reports nearly doubled RTX 4090 instruction throughput, yet only 12–34% faster FP16 cuBLAS GEMMs in a matched run. The Turing port was tested on a Dell RTX 2080 Ti with 22 GB of memory.

John Whitaker reported roughly 50% higher TFLOPS across three floating-point formats, with identical outputs in his matmul test.

An RTX 3090 Ti benchmark in Zagitta’s README supplies more specific test conditions: dense 8192³ GEMM, a 450 W power limit, five-sample medians and FP32 accumulation.

| Input format | Before | After | Throughput gain |
| --- | ---: | ---: | ---: |
| FP16 | 87.46 TFLOPS | 118.39 TFLOPS | 35.4% |
| BF16 | 87.50 TFLOPS | 157.41 TFLOPS | 79.9% |
| TF32 | 42.58 TFLOPS | 82.61 TFLOPS | 94.0% |

The README also says owner reports that compared outputs found them bit-identical before and after the unlock.

Whitaker’s local mixture-of-experts test produced two different outcomes:

• Prefill: a small improvement, below 10%.
• Decode: about the same speed.

Other owner reports collected in the Ampere README put prefill gains at 14–36%. The project attributes largely unchanged decode performance to memory-bandwidth limits.

The Ada maintainers recorded another narrow result on a modified 48 GB RTX 4090: a quantized MoE serving workload delivered 0.983× its original 8K prefill speed. They attributed the lack of improvement to quantized matmuls not using the affected FP32-accumulate Tensor Core path.

Zagitta’s boot-flow documentation describes a Linux kernel module that runs before NVIDIA’s driver binds the GPU:

1. Early loading: the package places the module and firmware in the initramfs.
2. Selector writes: the unlock writes SS0 as 0x88888888 and SS1 as 0x8.
3. Reset: the Ampere implementation performs a PCIe Function Level Reset.
4. Driver handoff: the unmodified NVIDIA driver binds the card.

Keeping the stock driver is the clever packaging choice: it avoids manually re-patching NVIDIA’s driver for each new release.

The module remains inactive without a kernel-command-line opt-in. gpu_unlock.targets= can select all accepted cards or specific PCI addresses; loading it after NVIDIA has already bound a card does nothing to that card.

The linked implementations have different hardware coverage and test conditions.

• Turing, TU102: the port reports FP16 matmul with FP32 accumulation rising from 55.7 to 91.4 TFLOPS at a 2048³ matrix size and 250 W, approximately 64% higher. Its reported test used NVIDIA driver 575.51.03; the README says both proprietary and open drivers worked.
• Ampere: user-reported BF16 GEMM gains span 25–67% on RTX 3090, 70–73% on RTX 3080 10 GB, and 73–81% on RTX 3070. These reports use different benchmarks and power configurations, so the figures are not directly comparable.
• Ada, RTX 4090: the fork reports FP16/FP32-accumulate instruction throughput rising from 165.6 to 331.6 TF/s, and BF16/FP32 from 165.5 to 330.6 TF/s. In a separate matched owner run, BF16 cuBLAS GEMM improved 16–20% at two shapes but was essentially unchanged at 4096³.

Experimental Ada targets have separate opt-in flags. Several also require an exact PCI address and cannot be selected through targets=all; the fork cautions that results from a few cards do not establish compatibility across other boards, memory configurations or laptops.

The Ampere compatibility table’s lowest reported RTX 3090 gain, 25%, came from cards capped at 225 W. In the Hacker News discussion, project author Zagitta said heavier compute can push the GPU down its frequency curve to stay within its power limit.

One owner in that discussion reported approximately 6% faster nanochat training on an RTX 3060 capped at 170 W. The Turing project explicitly warns that full-speed Tensor Cores draw high power and may damage hardware.

Several boundaries remain:

• Other arithmetic: the Ada fork’s INT8, dp4a and scalar FP32 measurements stayed near baseline.
• Instruction availability: one commenter flagged instruction-set restrictions as additional work still left to do in a reply.
• Gaming: Whitaker expected little benefit beyond a possible effect on DLSS.

The Ampere module’s resume handling accounts for the selectors being reset when suspend-to-RAM removes GPU power:

• Suspend-to-RAM: it reruns the unlock before NVIDIA resumes the card, adding about a second.
• Suspend-to-idle: a card that retained power remains unlocked and is only read.
• Hibernation: the project says it is not covered.

A separate failure path retains DMA buffers if the GPU firmware does not stop, because the firmware may still access them. The documented recovery requires a power-off before another attempt.