NVIDIA puts Groq 3 LPX into Vera Rubin production

• NVIDIA has put Groq 3 LPX, a rack-scale accelerator for token generation, into full production alongside Vera Rubin NVL72. WesRoth's launch summary captured the announced 3,400-token-per-second headline.
• Artificial Analysis measured 3,431 output tokens per second on Gemma 4 31B with a 100,000-token input, according to ArtificialAnlys's benchmark report.
• LPX uses 256 LP30 processing units, compiler-planned chip traffic, and SRAM to make tensor parallelism work at tiny batches, details visible in and NVIDIA's technical walkthrough.
• Nebius is the first cloud adopter, while GroqLLC's deployment announcement says Groq will deploy LPX and Rubin NVL72 with Dell in its inference cloud.

NVIDIA's technical post specifies a compiler that schedules chip-to-chip traffic before a workload runs, down to the clock cycle. Its 100K-context result came from a private endpoint and 50 sequential requests, while the system's documented serving layouts range from simple prefill-decode handoff to splitting attention and FFN layers across different racks.

The benchmarked LPX configuration has 256 LP30 local processing units, 128 GB of aggregate SRAM, and 96 chip-to-chip links per LPU, each rated at 112 Gbps, according to NVIDIA's hardware description.

The compiler sees the compute units, memory, and network topology before execution, then schedules transfers rather than arbitrating them in real time. NVIDIA says it can start moving a result after a 320-byte vector is computed, overlapping communication with the rest of a matrix operation.

At small batches, tensor parallelism becomes dominated by first-bit latency: selecting a link, synchronizing endpoints, and resolving contention can take as long as moving a small tensor. LPX's deterministic schedule targets that fixed coordination cost.

NVIDIA granted Artificial Analysis access to a private demonstration endpoint serving Gemma 4 31B. The benchmarker ran its standard 1K, 10K, and 100K input prompts, and ArtificialAnlys's benchmark report records a 3,431-token-per-second median at 100K across 50 sequential, single-concurrency requests.

NVIDIA's technical account puts the 10K median at 3,382 tokens per second, only 49 tokens per second below the 100K result. Its separate SPEED-Bench run, conducted by NVIDIA rather than Artificial Analysis, reported a 4,767-token-per-second median on coding problems and a 5,520-token-per-second P80.

The 4x comparison in NVIDIA's launch blog compares that private, pre-release endpoint with public serverless endpoint measurements. It is a single-concurrency output-speed measurement, not a pricing or high-concurrency throughput test.

The agent loop can split work among Rubin GPUs, LPX, and Vera CPUs, as rohanpaul_ai's hardware map describes: heavy model computation, token generation, and surrounding tool or code-execution work each have a different home.

NVIDIA's co-execution guide describes three layouts:

1. Prefill-decode disaggregation: Vera Rubin NVL72 processes prefill and sends the KV cache once per turn; LPX retains weights in SRAM and performs decoding.
2. Attention-FFN disaggregation: Rubin computes attention and retains the KV cache in DRAM; LPX runs FFN layers, with intermediate tokens crossing between racks for each full-attention layer.
3. External-drafter speculative decoding: LPX runs a smaller draft model; Rubin verifies and commits proposed tokens from the larger target model. Each rack keeps its own KV cache, and draft tokens cross the link.

NVIDIA illustrates these combinations with a projected model scale above two trillion parameters. The public third-party measurement remains the Gemma 4 31B endpoint.

NVIDIA says the rack is in full-scale production and will enter operation later this year, a timeline also reported by ArtificialAnlys's benchmark report. Nebius calls itself the first AI cloud to adopt LPX for its Token Factory.

Groq's announcement says it will be among the first adopters, pairing LPX with Vera Rubin NVL72 in its inference cloud and working with Dell on deployment.