SemiAnalysis says DeepInfer supports only offline, single-turn inference

• DeepInfer cannot yet run InferenceX's serving tests: SemiAnalysis says the released code supports only offline, single-turn inference.
• vLLM gets within 6% of DeepInfer's published decode performance under matched model and context settings, according to vLLM's comparison.
• DeepSeek's $38-per-accelerator-hour disclosure is revenue from a cache-heavy production workload: ZhihuFrontier's breakdown reports a 96% input cache hit rate.

The API companion translates three request formats but leaves HTTP transport to its caller. DeepInfer's configuration notes expose a 140-token bounded-replay suffix and deterministic sampling that requires setting DEEPINFER_ENABLE_MTP_VARLEN=0.

SemiAnalysis says filling in DeepInfer's missing serving infrastructure would require too many assumptions for an InferenceX submission. DeepSeek's README explicitly excludes three components needed for a complete service:

• A KV cache system.
• Disaggregation of vision encoding, prefill and decode, although most required engine functionality is already implemented.
• Load balancing across data-parallel replicas for prefill and decode.

The Apache-2.0 release supports V4.1-Flash on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, as described in the release overview.

The companion deepseek-recipe supplies Rust libraries and Python bindings for Chat Completions, Responses and Messages conversion. Model inference and tool execution also remain the caller's responsibility.

DeepSeek reports 155 trillion tokens/day from one production cluster in the deployment breakdown. Its operational table separates cached input, newly processed input and generated output:

DeepSeek's production measurements, including cached and uncached throughput, latency and revenue

Behind those cache hits sits a formidable storage dependency:

• Roughly 100 PiB of SSD capacity in a separate 3FS cluster stores persistent KV prefixes.
• Any prefill worker can retrieve any cached prefix, decoupling cache access from worker placement.
• Retention reaches three days for agent traffic and seven days for chatbot traffic, according to the accompanying report.
• Asynchronous cache loading overlaps host-to-device transfers with layer execution.

vLLM reports decode throughput within 6% of DeepInfer's published curve across batch sizes 8 to 256. The comparison uses:

• Model and context: DeepSeek-V4.1-Flash at 131K tokens.
• Speculation: three DSpark draft tokens and a fixed 3.25 accepted tokens per step.
• vLLM configuration: 32 GB200 GPUs, wide expert parallelism and expert-parallel load balancing.
• DeepInfer reference: the published 64-GPU EP64/DP64 curve, with 384+64 experts versus vLLM's 384+32.

The engines share DeepSeek kernel libraries, which vLLM credits for their similar performance. DeepInfer's acknowledgments also state that it was originally built primarily on vLLM.

The InferenceX dashboard covers long-context, multi-turn serving workloads. SemiAnalysis links configurations and logs for the vLLM serving results alongside its profit estimator.

vLLM warns that its AgentX revenue estimate uses a different workload from DeepSeek's production figure. The numbers circulating around the comparison also use different accounting assumptions:

• DeepSeek production: $38 in revenue per accelerator-hour at off-peak API prices, as reported in the deployment breakdown.
• vLLM on AgentX: $38.96 in revenue per GPU-hour at 196 P90 tokens/s/user. Its 954K tokens/s/GPU total includes cached input, uncached input and output.
• Profit-estimator snapshot: $33.96 modeled profit after $5/GPU-hour compute expense, an 87.2% margin, using 100% utilization and a three-year rental commitment.
• SemiAnalysis's separate estimate: $50/GPU-hour in profit on GB300 at 125 TPS and 60% utilization, using OpenRouter prices.

A separate AgentX test reached 829 P90 tokens/s for one user on four B300 GPUs with vLLM's specialized decode path. The comparison reached 656 tokens/s on GB300 with the standard path, so both hardware and execution path changed; both runs used five DSpark draft tokens.