Back previous/next report: k j

NVIDIA Vera Rubin NVL72 (vLLM + Dynamo)

Submitted by NVIDIAon 2026-11-10 Published on 2026-11-10

Result statement

NVIDIA ran deepseek-v4_1-flash on NVIDIA Vera Rubin NVL72 (vLLM + Dynamo) and achieved a provisional 659,767.6 tokens/second System TPS at 2,054.2ms TTFT P90 in the Standardized division of the Available category on November 10, 2026. Pending MLCommons Association verification, results are subject to change pending peer review. .

[1] MLPerf Endpoints v1.0 deepseek-v4_1-flash, published November 10, 2026; retrieved from endpoints.mlcommons.org on November 13, 2026. Result —, run 10: concurrency = 18,432 queries, TTFT P90 = 2,054.2ms, System TPS = 659,767.6 tokens/sec, interactivity = 35.8 tokens/sec per user and total provisioned system power = 75.67 kW. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Submission Info

NVIDIA

535bb487-f9c0-4427-8777-a24717f8cd28

Submission Status REVIEW_PENDING

2026-11-10

Version v1.0

Division Standardized

Deployment Client on Premises

Model

Model Name deepseek-v4_1-flash

Division Standardized

Model Precision mxfp4

Model Link —

Calibration Link —

Model Notes —

Datasets

Dataset Name mlperf_agentic_inference_dataset

Dataset Type performance

Average Input Tokens —

Dataset Link —

System TPS vs. Tokens/s per User

System TPS vs. Concurrency

Time to First Token P90 vs. Concurrency

Tokens/s per User vs. Concurrency

SUT Summary

NVIDIA Vera Rubin NVL72 (vLLM + Dynamo)

Available

System Category — · —

24 accelerators

Max Supported Concurrency 18,432

Node Type ID 0

Node Count 6

Software

Framework vLLM (zihaok/vllm d63b8107b7081598c098bb34f1acc995f7ff36f5) + Dynamo 1.6.0.dev20261016

vLLM (zihaok/vllm d63b8107b7081598c098bb34f1acc995f7ff36f5) + Dynamo 1.6.0.dev20261016

vLLM behind Dynamo vLLM sidecars and a Dynamo frontend

620

Link to Container https://github.com/zihaok/vllm/tree/d63b8107b7081598c098bb34f1acc995f7ff36f5

Operating System Ubuntu 26.04.1 LTS, Linux 7.0.0-2015-nvidia-bos-64k

Filesystem Lustre (shared models, datasets, compilation caches and results)

Other Software See docs/software_disclosure.md

Software Notes Public serving source: https://github.com/zihaok/vllm/tree/d63b8107b7081598c098bb34f1acc995f7ff36f5; SM107 MegaMoE plugin f51c0953e85b3e562d23df7053f79fcc6569d425.

Hardware

Processor

NVIDIA Vera CPU

176

VCPUs Per Processor 352

Accelerator

—

Host / Storage

1.5 TB (MemTotal 1,499,461,952 kB)

Memory Configuration LPDDR5X SOCAMM; two NUMA nodes, one per Vera CPU (MemTotal 745,599,040 kB and 753,862,912 kB)

Storage Capacity 32.6 TB (4 x 7.68 TB + 1 x 1.92 TB NVMe)

Storage Type NVMe SSD: 4 x Micron MTFDMBT7T6THM 7.68 TB and 1 x Micron MTFDLBT1T9THG 1.92 TB; /raid is a 6.0 TB ext4 LVM volume used as per-allocation scratch

Networking

10: 8 ConnectX-9 (InfiniBand, 800 Gb/s XDR each) and 2 BlueField-4 integrated network controllers (2 Ethernet ports each, 400 Gb/s)

NVLink + InfiniBand

Network Topology —

Other

Other Hardware —

Cooling Liquid-cooled

Hardware Notes —

Run Data

Field Name	Run 1	Run 2	Run 3	Run 4	Run 5	Run 6	Run 7	Run 8	Run 9	Run 10
Run Date	11/10/2026	11/10/2026	11/10/2026	11/10/2026	11/10/2026	11/10/2026	11/10/2026	11/10/2026	11/10/2026	11/10/2026
Concurrency	16	32	128	512	1,024	1,408	2,560	3,072	6,144	18,432
System Tokens/Second	4,392.2	8,639.9	32,483.7	117,192.5	204,207.7	272,711.7	362,484.3	404,045.2	562,667.8	659,767.6
Tokens/Second per User	274.5	270.0	253.8	228.9	199.4	193.7	141.6	131.5	91.6	35.8
TTFT P90 (ms)	120.9	129.4	160.5	364.5	505.3	427.0	1511.9	1582.2	857.1	2054.2
Utilization	0.7%	1.3%	4.9%	17.8%	31.0%	41.3%	54.9%	61.2%	85.3%	100.0%
Configuration Summary	deepseek-v4.1-flash-nvflow · online · c16	deepseek-v4.1-flash-nvflow · online · c32	deepseek-v4.1-flash-nvflow · online · c128	deepseek-v4.1-flash-nvflow · online · c512	deepseek-v4.1-flash-nvflow · online · c1024	deepseek-v4.1-flash-nvflow · online · c1408	deepseek-v4.1-flash-nvflow · online · c2560	deepseek-v4.1-flash-nvflow · online · c3072	deepseek-v4.1-flash-nvflow · online · c6144	deepseek-v4.1-flash-nvflow · online · c18432
Config Type	online	online	online	online	online	online	online	online	online	online
Config Region	—	—	—	—	—	—	—	—	—	—
Target QPS	—	—	—	—	—	—	—	—	—	—
Result State	complete	complete	complete	complete	complete	complete	complete	complete	complete	complete
Measured Accuracy Score	—	—	—	—	—	—	—	—	—	—
Benchmark Version	unknown	unknown	unknown	unknown	unknown	unknown	unknown	unknown	unknown	unknown
Result Version	0.1.0	0.1.0	0.1.0	0.1.0	0.1.0	0.1.0	0.1.0	0.1.0	0.1.0	0.1.0
Archive URI	gs://mlperf-endpoints-blob/runs/471c611d-482b-4781-ad9b-66b7681c3350/archive.tar.gz	gs://mlperf-endpoints-blob/runs/300e02f0-9b79-43c1-afd7-f6d8ac23b1d9/archive.tar.gz	gs://mlperf-endpoints-blob/runs/4cc50eae-7312-4866-9154-910847dc438f/archive.tar.gz	gs://mlperf-endpoints-blob/runs/1a675216-2fba-4ef2-b67c-de61ace6f2f2/archive.tar.gz	gs://mlperf-endpoints-blob/runs/1292bca4-bb8a-45e9-953e-49e82fe0a553/archive.tar.gz	gs://mlperf-endpoints-blob/runs/1e6f7b57-97f5-49ef-b24a-38971e8f7677/archive.tar.gz	gs://mlperf-endpoints-blob/runs/84d71bb2-e587-49c7-ace9-318f73c87b89/archive.tar.gz	gs://mlperf-endpoints-blob/runs/5c270e3a-c21f-4ebd-9919-543ab1cdae3f/archive.tar.gz	gs://mlperf-endpoints-blob/runs/9cc8ce4a-24fd-4528-bead-3bf7b8a0d89c/archive.tar.gz	gs://mlperf-endpoints-blob/runs/33eae96d-e5c6-4b05-86d8-37a965e234d1/archive.tar.gz
QPS	4.7891	9.5062	35.3865	126.8932	220.5183	294.5940	387.4391	431.8287	602.5129	700.0510
Total Output Tokens	17,969,187	36,407,671	109,230,870	364,503,026	540,560,775	718,868,281	1,101,269,433	1,103,186,345	1,668,317,772	2,202,613,012
Average Output Tokens	917.12	908.87	917.97	923.55	926.03	925.72	935.59	935.66	933.87	942.46
Run Duration (s)	4,091.15	4,213.88	3,362.63	3,110.29	2,647.11	2,636.00	3,038.12	2,730.35	2,965.01	3,338.47
Total Queries	19,593	40,058	118,992	394,675	583,737	776,550	1,177,085	1,179,045	1,786,459	2,337,098
Total Queries Issued	19,593	40,058	118,992	394,675	583,737	776,550	1,177,085	1,179,045	1,786,459	2,337,098
Total Queries Failed	0	0	0	0	0	0	0	0	0	0
Time to First Token (TTFT) (ms)
Minimum	65.3	66.8	69.8	78.9	80.4	82.2	148.4	196.6	232.3	162.7
Average	107.0	111.8	136.0	253.7	362.9	308.2	1260.5	1374.4	686.6	1197.9
P50	94.2	98.7	115.2	222.0	309.2	266.7	1253.2	1333.0	591.1	727.7
P90	120.9	129.4	160.5	364.5	505.3	427.0	1511.9	1582.2	857.1	2054.2
P95	132.9	141.4	181.1	414.5	599.8	483.2	1588.2	1665.0	974.8	2776.3
P99	169.4	176.7	248.3	535.2	828.1	616.2	1774.3	1889.3	1340.1	13922.4
P999	1745.1	1469.2	7097.1	9221.5	14568.6	13085.0	13888.4	15004.3	15175.1	28353.6
Maximum	12860.8	14325.6	16610.0	13827.7	16551.7	15831.9	15795.1	19565.8	20096.1	32000.4
Time Per Output Token (TPOT) (ms)
Minimum	0.34	0.53	-0.00	0.71	-0.16	-0.13	-2.55	-0.19	-2.10	-0.16
Average	1.44	1.49	1.67	1.92	2.33	2.49	3.17	3.48	6.66	18.9
P50	1.32	1.38	1.55	1.77	2.15	2.30	2.93	3.22	6.15	17.4
P90	2.09	2.18	2.44	2.80	3.39	3.63	4.63	5.09	9.72	27.6
P95	2.27	2.37	2.64	3.02	3.67	3.92	5.01	5.50	10.5	29.8
P99	2.62	2.72	3.03	3.47	4.22	4.50	5.74	6.29	12.0	34.1
P999	3.09	3.15	3.55	3.99	4.87	5.17	6.62	7.27	13.8	39.0
Maximum	80.0	13.7	74.9	299.6	290.3	273.4	736.2	760.2	1972.0	2264.7
Query Latency (ms)
Minimum	107.7	97.1	109.6	124.8	166.2	147.4	293.7	383.7	423.5	449.9
Average	1722.0	1783.1	2023.9	2431.4	3012.9	3143.5	4915.9	5388.6	8354.4	23215.1
P50	346.1	363.8	413.9	602.2	788.4	746.8	1865.9	1998.8	1808.9	4485.1
P90	3925.4	4065.2	4643.9	5479.6	6788.0	7193.1	10227.7	11286.2	19243.8	54670.9
P95	11628.3	11750.8	13726.8	16054.9	19699.8	20982.3	28900.9	31815.5	59129.1	179488.6
P99	17624.5	18473.8	20621.1	23743.1	28849.1	30785.8	40052.2	43944.8	82228.5	231990.3
P999	18687.5	19432.6	21705.1	24991.8	30399.3	32391.9	42740.2	47114.5	86835.0	244587.1
Maximum	30769.6	29781.2	37673.4	38205.2	47100.1	48098.0	61197.0	65900.4	98468.7	271365.2