Back previous/next report: k j

NVIDIA NVIDIA Vera Rubin NVL72 (4-GPU SUT)

Submitted by NVIDIAon 2026-11-07 Published on 2026-11-09

Result statement

NVIDIA ran Qwen3.6 35B-A3B on NVIDIA Vera Rubin NVL72 (4-GPU SUT) and achieved a provisional 85,567.6 tokens/second System TPS at 223.6ms TTFT P90 in the Standardized division of the Available category on November 9, 2026. Pending MLCommons Association verification, results are subject to change pending peer review. .

[1] MLPerf Endpoints v1.0 Qwen3.6 35B-A3B, published November 9, 2026; retrieved from results.mlcommons.org on November 18, 2026. Result —, run 11: concurrency = 1,024 queries, TTFT P90 = 223.6ms, System TPS = 85,567.6 tokens/sec, interactivity = 83.6 tokens/sec per user and total provisioned system power = 12.61 kW. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Submission Info
NVIDIA
f195f05e-7cca-4b25-95ed-a4db747814b6
Submission Status REVIEW_PENDING
2026-11-07
2026-11-09
Version v1.0
Division Standardized
Deployment Client on Premises
Model
Model Name Qwen3.6 35B-A3B
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Datasets
Dataset Name agentic_coding
Dataset Type performance
Average Input Tokens —
Dataset Link —

System TPS vs. Tokens/s per User

020k40k60k80kSystem TPS100150200Tokens/s per User

System TPS vs. Concurrency

020k40k60k80kSystem TPS02004006008001,000# of queries

Time to First Token P90 vs. Concurrency

50100150200TTFT P90 (ms)02004006008001,000# of queries

Tokens/s per User vs. Concurrency

100150200Tokens/s per User02004006008001,000# of queries
SUT Summary
NVIDIA
NVIDIA Vera Rubin NVL72 (4-GPU SUT)
Available
System Category — · —
4 accelerators
1
1
Max Supported Concurrency 1,024
Node Type ID 0
Node Count 1
Software
Ubuntu
Framework vLLM 1af1c649355f behind Dynamo 1.6.0.dev20261021 vLLM sidecars
vLLM 1af1c649355f behind Dynamo 1.6.0.dev20261021 vLLM sidecars
vLLM (CentML/vllm 1af1c649355f747b7bd8aa6ed36446675f0ebc86)
620
Link to Container —
Operating System Ubuntu 26.04.1 LTS, Linux 7.0.0-2015-nvidia-bos-64k
Filesystem Shared Lustre for persistent files; per-allocation RAID for model and compilation scratch
Other Software Dynamo 1.6.0.dev20261021 (ai-dynamo, source bc2b62478adc); torch 2.15.0.dev20260908+cu134; transformers 5.22.1
Software Notes vLLM CentML/vllm mlperf-end-multiturn-v1.0-qwen-union 1af1c649355f747b7bd8aa6ed36446675f0ebc86 with Dynamo 1.6.0.dev20261021 vLLM sidecars.
Hardware

Processor

NVIDIA Vera CPU
2
88
VCPUs Per Processor 176

Accelerator

—
0
—
—
—
—

Host / Storage

1.5 TB (MemTotal 1,499,461,952 kB)
Memory Configuration LPDDR5X SOCAMM; two NUMA nodes, one per Vera CPU (MemTotal 745,599,040 kB and 753,862,912 kB)
Storage Capacity 32.6 TB (4 x 7.68 TB + 1 x 1.92 TB NVMe)
Storage Type NVMe SSD: 4 x Micron MTFDMBT7T6THM 7.68 TB and 1 x Micron MTFDLBT1T9THG 1.92 TB; /raid is a 6.0 TB ext4 LVM volume used as per-allocation scratch

Networking

10: 8 ConnectX-9 (InfiniBand, 800 Gb/s XDR each) and 2 BlueField-4 integrated network controllers (2 Ethernet ports each, 400 Gb/s)
NVLink + InfiniBand
Network Topology —

Other

Other Hardware —
Cooling Liquid-cooled
Hardware Notes At every point the benchmark client runs on a second node allocated with the server (its GPUs idle) and reaches the Dynamo frontend over the server node's 400 Gb/s BlueField-4 Ethernet port in one routed hop (TCP connect p50 0.389 ms). docs/software_disclosure.md lists the placement and the latency measurement.
Run Data
Field NameRun 1Run 2Run 3Run 4Run 5Run 6Run 7Run 8Run 9Run 10Run 11
Run Date11/07/202611/07/202611/07/202611/07/202611/07/202611/07/202611/07/202611/07/202611/07/202611/07/202611/07/2026
Concurrency3236641281441922563203845121,024
System Tokens/Second6,488.97,228.512,490.323,072.225,696.332,712.841,142.647,785.554,972.764,498.485,567.6
Tokens/Second per User202.8200.8195.2180.3178.4170.4160.7149.3143.2126.083.6
TTFT P90 (ms)39.239.941.444.552.957.865.975.282.3103.0223.6
Utilization7.6%8.4%14.6%27.0%30.0%38.2%48.1%55.8%64.2%75.4%100.0%
Configuration Summaryqwen-agentic-benchmark · online · c32qwen-agentic-benchmark · online · c36qwen-agentic-benchmark · online · c64qwen-agentic-benchmark · online · c128qwen-agentic-benchmark · online · c144qwen-agentic-benchmark · online · c192qwen-agentic-benchmark · online · c256qwen-agentic-benchmark · online · c320qwen-agentic-benchmark · online · c384qwen-agentic-benchmark · online · c512qwen-agentic-benchmark · online · c1024
Config Typeonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonline
Config Region———————————
Target QPS———————————
Result Statecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecomplete
Measured Accuracy Score———————————
Benchmark Versionunknownunknownunknownunknownunknownunknownunknownunknownunknownunknownunknown
Result Version0.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.0
Archive URIgs://mlperf-endpoints-blob/runs/b157ad82-e640-4cad-b9db-95b4c5a482b9/archive.tar.gzgs://mlperf-endpoints-blob/runs/e57eb243-dc36-4d01-9e1e-0bfb31a483b9/archive.tar.gzgs://mlperf-endpoints-blob/runs/bb2f10d8-6308-4804-bb63-3d026be614c9/archive.tar.gzgs://mlperf-endpoints-blob/runs/c3b57bce-8397-4349-8a5a-3c7f45dc8d61/archive.tar.gzgs://mlperf-endpoints-blob/runs/462cec70-7e53-483d-9a27-0e31a162d75a/archive.tar.gzgs://mlperf-endpoints-blob/runs/221b68e0-8279-4c90-a49a-858ed95c3a6e/archive.tar.gzgs://mlperf-endpoints-blob/runs/3c370097-50f7-46e8-98d8-6e0b5fb84bf6/archive.tar.gzgs://mlperf-endpoints-blob/runs/9e265696-b003-4bd5-b3fe-16b4203485aa/archive.tar.gzgs://mlperf-endpoints-blob/runs/4febf8ce-b5b2-41ec-973d-72821e6d4b7c/archive.tar.gzgs://mlperf-endpoints-blob/runs/4009a761-9d85-45ab-af31-82213a321393/archive.tar.gzgs://mlperf-endpoints-blob/runs/d0305039-ef4f-457a-bd36-d9d3feca00d6/archive.tar.gz
QPS17.281419.310033.270961.502568.441187.3910110.0323128.0382146.6946171.8280228.3868
Total Output Tokens30,490,10830,363,02745,370,14182,960,83982,828,76889,786,753104,512,031118,323,122141,225,407169,643,614228,674,546
Average Output Tokens375.48374.34375.41375.14375.45374.33373.91373.21374.74375.37374.66
Run Duration (s)4,698.814,200.463,632.423,595.713,223.372,744.702,540.242,476.132,569.012,630.202,672.44
Total Queries81,20281,111120,854221,145220,611239,862279,508317,039376,860451,942610,351
Total Queries Issued81,20281,111120,854221,145220,611239,862279,508317,039376,860451,942610,351
Total Queries Failed00000000000
Time to First Token (TTFT) (ms)
Minimum9.4910.011.113.114.717.418.019.820.421.730.6
Average27.628.229.432.438.342.547.854.159.073.8155.0
P5025.225.726.529.635.239.144.049.854.268.3145.7
P9039.239.941.444.552.957.865.975.282.3103.0223.6
P9545.746.948.952.361.567.677.687.896.5119.9257.1
P9969.171.877.884.294.6109.1125.1138.2152.6181.2364.4
P999141.1144.3159.4200.0220.1293.7346.3373.4430.9496.3852.1
Maximum360.6368.7360.5435.3497.2487.6594.7656.9677.8833.91863.6
Time Per Output Token (TPOT) (ms)
Minimum-0.00-0.00-0.00-0.00-0.00-0.00-0.00-0.00-0.00-0.000.51
Average0.720.750.901.301.341.561.922.332.593.517.09
P500.670.700.831.221.261.471.792.182.453.296.61
P900.930.971.161.671.722.012.442.983.354.509.33
P951.031.071.271.811.872.182.653.233.624.8610.2
P991.221.291.522.132.172.563.113.774.225.5812.0
P9991.681.942.183.153.103.864.545.305.807.3016.7
Maximum199.2387.7311.7505.5580.8518.9760.6761.11656.01289.53538.1
Query Latency (ms)
Minimum22.523.025.229.932.033.641.455.649.160.0121.9
Average290.5302.9353.3510.5530.9619.7746.3906.11018.81369.72776.9
P50104.5108.9124.4170.6181.2209.4249.5300.7334.0444.1904.0
P90705.6744.0871.21260.51298.11526.71849.02252.72521.53404.96900.4
P951153.61189.21405.12081.02133.92500.03031.13696.44170.95613.211355.5
P992480.42607.03102.04565.94771.35453.46599.98058.29075.512361.125070.9
P9995255.35425.26275.59326.69697.911284.213818.816298.119016.625028.651643.5
Maximum8303.77946.610385.314003.114602.316543.220672.225211.529980.437295.285306.5