EdgeRock-4B fixes its attention KV cache at 72 MiB

• EdgeRock-4B caps its attention KV cache at 72 MiB per conversation, using local attention and fixed-capacity memory anchors, according to testingcatalog's architecture breakdown.
• AIME 2024–2026 mean pass@1 reached 82.9%, versus 18.8% for the strongest evaluated bounded-state baseline, as testingcatalog reported.
• Distant-memory query accuracy reached 94.1% with anchors enabled and 0.4% with them disabled in Rock Matrix's test, described in testingcatalog's follow-up.
• A local Mac demonstration generated roughly 9 tokens/s, according to testingcatalog's laptop report.

An intermediate training checkpoint scored 96.1% on the memory-query test, above the final release. Rock Matrix also bundles a Transformers 5.7.0 development wheel with the reference runtime.

Total persistent history reaches 97.76 MiB per sequence in the batch runtime using FP16 recurrent matrices, according to Rock Matrix's memory accounting:

| Component | MiB per sequence |
| --- | ---: |
| Committed memory anchors | 32.00 |
| Pending anchor candidates | 8.00 |
| Local-window KV | 31.97 |
| Recurrent matrices | 24.00 |
| Convolution state | 1.50 |
| Metadata | 0.29 |
| Total persistent history | 97.76 |

The local ring stores 1,023 preceding positions; the decoder supplies the current token. The headline 72 MiB attention-KV capacity compares with 8 GiB for Qwen3.5-4B at 256K context, a 99.1% reduction with BF16 KV.

Model weights, buffers, CUDA Graphs and workspace consume additional memory. Measured peak GPU allocation reaches 16.02 GiB for 128K input plus 128K output at batch size one.

EdgeRock-4B preserves Qwen3.5-4B's 24 Gated DeltaNet layers and feed-forward blocks, while converting its eight full-attention layers to sliding-window attention paired with Rock Memory Anchors. Rock Matrix's technical report describes three operations:

1. Write: A learned writer produces dedicated key–value records. Pending candidates occupy a fixed 256-position buffer.
2. Admit: Every 256 tokens, existing anchors compete with new candidates for up to 1,024 records in each of four KV groups. That learned competition is the clever bit.
3. Retrieve: Queries read the committed bank, and newly selected records become visible in the next block. A query-dependent signed gate combines retrieved information with attention over the 1,024-position local window; retained records remain separate and unchanged.

Fixed component capacities give O(1) persistent history in sequence length and O(N) generation work for N output tokens, at fixed model and memory dimensions.

Rock Matrix's memory evaluation uses 256 contexts, 1,024 queries and inputs ranging from 16K to 130,000 tokens. The ablation switches off all eight RMA modules in the final model while leaving recurrence and local attention active.

Each task has 64 contexts. Complete-context success requires all four answers to be correct:

| Task | Complete contexts | Success rate |
| --- | ---: | ---: |
| Exact retrieval | 58/64 | 90.6% |
| State-update interpretation | 56/64 | 87.5% |
| Cross-document combination | 57/64 | 89.1% |
| Variable binding | 34/64 | 53.1% |
| Overall | 205/256 | 80.1% |

The 18.8% AIME comparator is RWKV-7 G1j at 13.3B parameters, according to Rock Matrix's benchmark tables. Against its Qwen3.5-4B backbone, EdgeRock improves eight of 29 text benchmarks.

The source-to-converted-model comparison exposes substantial variation:

| Benchmark | Qwen3.5-4B | EdgeRock-4B | Change |
| --- | ---: | ---: | ---: |
| AIME three-year mean | 86.1% | 82.9% | −3.2 points |
| LiveCodeBench v6 | 75.7% | 61.6% | −14.1 points |
| GPQA-Diamond | 77.3% | 64.6% | −12.7 points |
| HellaSwag | 80.6% | 64.9% | −15.7 points |
| MMLU-Pro | 77.7% | 79.8% | +2.1 points |
| SuperGPQA | 52.8% | 60.0% | +7.2 points |
| HumanEval | 86.0% | 96.7% | +10.7 points |
| MBPP | 71.0% | 87.8% | +16.8 points |

Across six visual benchmarks, equal-weight answer-level accuracy falls from 89.1% to 84.9%, a 4.2-point decline. All these scores are Rock Matrix's reported evaluations.

Rock Matrix measured its runtime benchmarks on one Hopper 80GB GPU with BF16 weights and KV:

• Best-batch serving, 32K input plus 32K output: Qwen3.5-4B delivered 1,889 output tokens/s at batch 64; EdgeRock delivered 12,124 at batch 512, 6.42× higher. Elapsed time includes prefill.
• Single-request decoding, 128K input plus 128K output: Qwen3.5-4B delivered 151.3 tokens/s; EdgeRock delivered 233.7, 1.54× higher. Decode timing excludes prefill.
• Concurrency sweep: Across 16 input/output length pairs, EdgeRock reached 12,497 weighted decode tokens/s at batch 512, or 24.4 tokens/s per sequence, with 61.26 GiB peak allocated memory.

Both models use FP32 Gated DeltaNet updates. EdgeRock stores recurrent state in FP32 at batch one and FP16 at larger batches; Qwen3.5-4B keeps FP32 state storage.

The laptop example runs 17 × 23 with step-by-step reasoning through Rock Matrix's PyTorch runtime on the Mac GPU.

Weights and inference code are public, with Hugging Face and ModelScope downloads linked in testingcatalog's follow-up. Packages in the official model directory have different runtime requirements and validation scopes:

• Original BF16: Supports text and image inputs with text generation.
• NF4, INT8 and FP8: Use the included compact loader. Validation covers text inference on Hopper; the original model's benchmark scores do not establish quantized-package accuracy or throughput.
• GGUF: Requires the custom EdgeRock RMA backend. Stock llama.cpp and Ollama are unsupported.
• MLX: Uses the included EdgeRock runtime. Functional checks ran on Linux CUDA; Apple Metal has not been tested, separately from the PyTorch Mac demonstration.