Parakeet Redux cuts NVIDIA's Parakeet from 1.2 GB to 178 MB

• Parakeet Redux shrinks NVIDIA's 1.2 GB Parakeet checkpoint to 178 MB by setting every encoder weight to -1, 0, or +1, according to vikhyatk's launch post.
• The claimed 113x real-time throughput on eight x86 CPU cores comes from moving far fewer weight bytes, a bandwidth-bound advantage that vikhyatk's CPU explanation puts at 2.5x over the fastest other Parakeet runtime measured.
• Accuracy improves on the 25-language FLEURS set and on long-form talks, while business speech and background noise worsen in the scorecard linked by vikhyatk's model-card post.
• CPU inference can run slightly slower when a GPU is visible to the process, a release-day bug that vikhyatk's bug report says has a temporary CUDA_VISIBLE_DEVICES= workaround.

The Redux model card includes segment and word timestamps, plus a model-owned VAD head that cuts long recordings into segments of at most 30 seconds. A same-day Kestrel commit routes kernel calls by tensor device, directly addressing an awkward CPU-in-a-GPU-process edge case.

Redux keeps the architecture and tokenizer of NVIDIA's Parakeet TDT checkpoint, a 600M-parameter multilingual ASR model that automatically detects among 25 European languages. The author identifies that upstream model in vikhyatk's link to the original model.

The compression is confined to encoder weights, which now take one of three values. Vikhyat says the quantization recipe minimizes forgetting, then a post-training pipeline recovers most of the expected accuracy loss in vikhyatk's quantization note.

The model card's performance table defines real-time factor as audio seconds transcribed per wall-clock second, processing one utterance at a time. Its x86 run used eight physical cores of an AMD EPYC 9575F, DDR5-6000, Ubuntu 22.04, and LibriSpeech test-clean.

• x86 CPU: Photon with Redux reached 113x at 178 MB, against parakeet.cpp at 45x with 0.94 GB q8 weights, sherpa-onnx at 42x, and onnx-asr at 28x.
• MacBook Air M2 CPU: Photon reached 38x, against onnx-asr at 33x, sherpa-onnx at 28x, and parakeet.cpp at 12x.
• MacBook Air M2 GPU: Photon reached 43x through Metal, against 39x for parakeet.cpp f16 and 37x for parakeet-mlx fp32.

The x86 comparison spans 2,620 utterances, while the Apple tests use a 50-utterance slice with cool-downs between runs. The card says every Redux speed result uses Photon, while competing runtimes ran at their defaults.

The scorecard swaps multilingual and long-form gains for penalties on some English, business, and noisy material. The Redux benchmark table scores both models with the Open ASR Leaderboard pipeline, but combines a Photon run for Redux with a bf16 NeMo run for the original.

• Open ASR Leaderboard, seven English sets: 6.26% WER to 6.55%, +0.29 points.
• FLEURS, 25 languages: 11.62% to 10.56%, -1.06 points.
• Business speech: 6.15% to 6.96%, +0.81 points.
• MUSAN background noise: 6.72% to 9.04%, +2.32 points.
• TED-LIUM long-form: 2.71% to 2.51%, -0.20 points.

The model card attributes the low-SNR gap to a thinner acoustic margin in the ternary encoder. Vikhyat described the observed behavior in a reply on noisy audio as substituting similar-sounding words rather than exact ones.

Photon reads the packed weights directly and uses AVX-512 VNNI on x86, NEON on ARM, and Metal on Apple GPUs, per the Redux model card. Its minimal API exposes plain text, sentence-level timestamps, or word-level timestamps:

The weights are CC-BY-4.0, matching the source checkpoint. Vikhyat confirmed there are no additional restrictions while responding to a WebAssembly port in vikhyatk's licensing reply.

A CPU-placed Redux model could take a slower path when CUDA was visible anywhere in its process. Vikhyat's reported workaround was to hide the GPU from that process, with a fix planned for the next release in vikhyatk's bug report.

The Kestrel issue attributes the behavior to choosing a runtime from process-level CUDA visibility instead of the model or tensor device, which selects PyTorch fallbacks rather than fused CPU kernels. It also reported transcript differences between the two settings with weights and thread counts held constant.

A verified September 21 commit changed kernel routing by tensor device across 29 files. The author pointed to that next-release fix in vikhyatk's follow-up.