GPU Infrastructure

A Microsoft–Tsinghua study attributed 60.33% of 300 sampled issues across seven GPU kernel libraries to compiler, build or runtime environments. It also identified hidden correctness constraints introduced by specialization.

Developers report higher Tensor Core throughput on older RTX GPUs, and an independent test found identical outputs. John Whitaker's local MoE tests showed under 10% faster prefill and essentially unchanged decode speed.

SemiAnalysis identifies 8.7 GW of Texas datacenter campuses needing air permits under the expanded freeze. That is total campus capacity, not delayed capacity; site work and already permitted generation can proceed.

Modular says Mojo and MAX are fully open for development, extension and commercial use, including compiler and runtime internals. Support for NVIDIA, AMD and Apple hardware is already public.

DeepSeek releases DeepInfer under Apache 2.0 for V4.1 Flash on Blackwell or Ascend hardware. The engine exposes techniques such as speculative decoding but omits some serving components.

FastVideo's FastH3-QAD-v2 and NVFP4 VAE decoders cut weights from 118 to 30 GiB. The team reports generating a five-second 480p clip in 52.7 seconds on one DGX Spark, down from 141 seconds.

vLLM reports 842 tokens per second at batch one on GB300 NVL72 with DSpark, plus up to 1.8× better P90 interactivity. Engineers dispute describing the implementation as a megakernel.

NVIDIA-submitted AA-AgentPerf results show Rubin delivering up to 5.7× GB300’s throughput per watt across four agentic coding workloads. The results detail per-user speed, power efficiency and serving stacks.

AA-AgentPerf finds MI355X delivers 15–40% more throughput per megawatt than B200 on DeepSeek V4 Pro at moderate user speeds. AMD’s relative results are weaker on Kimi K3 and GLM.

Zach Mueller explains how passive and active MCIO cabling differ as PCIe signals degrade over longer runs. He details a switch-based expansion option using retimers and GPU-side MCIO hosts.

Zach Mueller compares direct and cascaded PCIe connections for an eight-GPU expansion setup. His updated explanation corrects CPU-bypass claims and details bandwidth, airflow, noise, and maintenance constraints.

SemiAnalysis reports up to 10× better inference performance per dollar on Rubin NVL72 than GB300 NVL72 using vLLM. These are preview results; the full public InferenceX submission is still pending.

SemiAnalysis reports outages, storage failures, link flaps and GPU errors at IREN's Canadian sites. It disputes managed-cluster claims while describing better redundancy at newer US facilities.

SemiAnalysis found vulnerable libraries, login-pod setup errors and storage configuration problems that left pods stuck in Vultr's ClusterMAX test. It also reported strong Lustre throughput alongside networking reliability problems.

Modal made multi-node, RDMA-connected clusters globally available. The runtime also adds full-kernel VM sandboxes, sandbox sidecars, and sticky sessions that reconnect WebSockets to the same container.

NVIDIA's Open Agent Safety Platform uses OpenShell to run agents in isolated environments. A technical report describes a BlueField-4 Sentry layer that can quarantine agents outside the sandbox.

Google and Planet plan to launch a prototype satellite carrying four TPUs. The mission will collect data on whether ML hardware can operate under orbital conditions.

Perplexity launched Portable Computer in its Windows app for supported NVIDIA RTX systems. Local inference requires at least 24GB of VRAM, while the update also adds local MCP support and scheduled tasks.

Together AI says its HUMAIN partnership will provide 250 MW of data-center capacity for open-source models over the next year. The deployment exceeds 100,000 chips and could produce more than $5 billion in annualized AI.

OpenAI says Jalapeño delivered 1.5–1.9× more work per watt and 1.7–3.6× lower end-to-end latency than NVIDIA systems in its tests. The company plans to deploy the inference chip in its compute infrastructure by year-end.

Groq 3 LPX adds dedicated token generation to NVIDIA Vera Rubin systems, with Groq and Nebius among planned deployers. Artificial Analysis measured about 3,400 output tokens per second on Gemma 4 31B.

Marin has begun an open training run for a 535B-parameter mixture-of-experts model with 23B active parameters. The team plans to train on 18.75T tokens across 11 GB200 NVL72 systems over about three months.

Reports say OpenAI agreed to secure roughly 8 GW of capacity at Ohio’s PORTS-Pike campus. The initial 800 MW is expected in 2028, with SB Energy funding grid and transmission upgrades.

Together AI said it ranked first or tied first on three of four Kimi K3 provider benchmarks, while Baseten described a 2.8T-parameter Blackwell GB300 serving stack. Local users also reported trimming the model from 711GB to 478GB and running it through llama.cpp RPC across clusters.

Posts quoting Liang Wenfeng said DeepSeek is targeting low positive API margins while constrained by GPU supply. They also said early-June capacity was about 20,000 H100-equivalent units and that Huawei capacity remains below Nvidia.

A LocalLLaMA benchmark on Qwen 3.6 27B and RTX 6000 PRO reports near-6x speedups from MTP, DFlash, and n-gram drafting. Related tests cover remote prefill, NUMA offload, GPU clock tuning, and llama.cpp Gemma 4 support.

Meituan released LongCat-2.0 weights and inference code under MIT, with Hugging Face, GitHub, ModelScope, GPU, and NPU paths. Analysts noted the ~48B-active MoE keeps attention shape while reducing zero-communication experts from 256 to 128.

Wafer reported GLM-5.2 serving at 2,626 tok/s per MI355X node, and Together put it at 80% of Sonnet 5 capability for 20% of the price. Critics questioned whether public benchmark gains were overfit.

Meituan disclosed LongCat 2.0, a 1.6T-parameter MoE with about 48B active parameters, 1M context, and 35T training tokens on domestic hardware. The release ties a near-frontier open model to a Chinese domestic compute stack and a custom sparse-attention design.

Snowflake open-sourced Arctic RL and said its ZoRRo optimization delivers up to 6x actor-update speedup and 3.5x end-to-end gains. The repo packages those gains into VeRL and SkyRL integrations plus open Text2SQL and multi-hop QA recipes.

The FT reported that Google capped Meta's Gemini usage after Meta asked for more model capacity than Google could supply, affecting internal safety, support, ad, and coding projects. The restriction matters because model access is now constrained by chip, memory, and networking capacity as much as by API contracts.

Practitioners published concrete GLM-5.2 self-host numbers, from Mac Studio and 4090-class setups to annualized power and hardware costs. That matters because open weights now offer privacy and rate-limit control, but quant quality, electricity, and latency still keep hosted APIs cheaper for many teams.

Ollama said it doubled GPU capacity for GLM-5.2 cloud usage and said the model is currently hosted only in the US. The rollout adds capacity as open-model demand climbs, so users should check hosting and privacy details before deploying.

Posts citing Korean reporting said NVIDIA qualified HBM4 from Samsung, SK hynix, and Micron for Vera Rubin and expanded memory co-design with SK hynix. The supply detail matters because HBM4 availability is the constraint behind next-generation AI systems.

NVIDIA shipped Nemotron 3 Ultra, a 550B/55B-active hybrid Mamba-Transformer MoE with open weights, data, and recipe, plus broad runtime and host support. It matters because the model pairs frontier open benchmarks with immediate agent-serving options, though local use still needs heavy quantization or large-memory hardware.

Microsoft and NVIDIA unveiled RTX Spark systems, including Surface Laptop Ultra and DGX-class Windows hardware, with 128GB unified memory and 1 PFLOP FP4 local AI. Day-one support from Hermes Agent, vLLM, Ollama, and Unsloth makes the launch useful for local inference and fine-tuning, not just a PC refresh.

Huawei outlined a τ scaling framework and LogicFolding design that shifts chip progress from node shrinkage toward shorter signal delay. The proposal matters because it targets performance, density, and yield gains without relying only on EUV-era process shrinks.

Anthropic said it is expanding onto Colossus 2 GB200 capacity, while a SpaceX S-1 excerpt pegged the contract at $1.25 billion per month through May 2029. The deal shows long-term chip and datacenter supply is becoming a gating factor for frontier-model deployment.

Nous Research published Lighthouse Attention, a hierarchical selection layer that keeps the standard attention kernel while cutting end-to-end pretraining wall clock by 1.4-1.7x at 98K context. It also scales to 1M-token training across 32 Blackwell GPUs without a custom sparse kernel.

Perplexity published serving results for post-trained Qwen3 235B on NVIDIA GB200 NVL72 and argues Blackwell materially outperforms Hopper for large MoE inference. The deltas show up in NVLS all-reduce latency, MoE prefill combine time, and high-speed decode throughput.

Anthropic said a SpaceX compute deal will add 300+ MW and 220,000+ NVIDIA GPUs, and it doubled Claude Code 5-hour limits across paid plans. It also raised Opus API ceilings; users should still watch the unchanged weekly caps.

Zyphra released ZAYA1-8B, an Apache-2.0 reasoning MoE with compressed-convolutional attention and bounded-context Markovian RSA test-time compute. The model targets math and coding workloads while keeping the active parameter count below 1B.

OpenAI and partners released Multipath Reliable Connection, an RDMA transport that spreads training traffic across multiple network paths and is already deployed on the company's largest clusters. The protocol targets congestion and failure recovery in giant GPU trainings, and teams building similar clusters should track the Open Compute Project release.

Zyphra published folded Tensor and Sequence Parallelism, claiming 173M tok/s versus 86M for matched TP+SP on 1,024 MI300X GPUs. The design keeps more replicas inside a node, reducing per-GPU memory pressure and cross-node communication.

Zyphra launched serverless inference on AMD MI355X for DeepSeek V3.2, Kimi K2.6, and GLM 5.1, aimed at long-horizon agent workloads. The service leans on high-HBM nodes to keep more long-context sessions resident and reduce queueing.

Moondream shipped Photon 1.2.0, expanding its inference engine to Apple Silicon, Windows CUDA, Blackwell, and Jetson Thor, then outlined how custom Metal kernels and fused ops made local vision practical without MLX. That broadens deployment options for edge and on-device vision workloads while keeping server-class latency on B200 systems.

Alibaba Qwen introduced FlashQLA, a TileLang-based linear-attention kernel stack that reports 2–3x faster forward passes and 2x faster backward passes. The release gives edge and long-context deployments a new optimization lever below the model layer itself.

SGLang and Miles published a technical breakdown of their DeepSeek V4 day-zero stack, including ShadowRadix caching, Flash Compressor, FP4 expert-weight handling, and measured B200/H200 throughput. That gives deployers concrete serving and training-path numbers for V4 beyond generic launch-day compatibility claims.

DeepSeek published Tile Kernels, an open-source TileLang repo covering Engram, mHC, MoE routing, and FP4/FP8 kernels, with claims that some are already used in internal training and inference. That matters because it exposes reusable low-level performance work behind DeepSeek’s stack instead of keeping the kernels fully private.

Google DeepMind introduced Decoupled DiLoCo, a distributed-training method that trained a 12B Gemma model across four US regions and mixed TPU6e/v5p hardware while tolerating failures. It matters because it targets the networking and uptime bottlenecks that make frontier training geographically rigid and operationally fragile.