Back previous/next report: k j

NVIDIA NVIDIA Vera Rubin NVL72 (TRT-LLM + Dynamo)

Submitted by NVIDIAon 2026-11-07 Published on 2026-11-09

Result statement

NVIDIA ran kimi-k3 on NVIDIA Vera Rubin NVL72 (TRT-LLM + Dynamo) and achieved a provisional 48,285.5 tokens/second System TPS at 9,510.5ms TTFT P90 in the Standardized division of the Available category on November 9, 2026. Pending MLCommons Association verification, results are subject to change pending peer review. .

[1] MLPerf Endpoints v1.0 kimi-k3, published November 9, 2026; retrieved from results.mlcommons.org on November 18, 2026. Result —, run 10: concurrency = 1,536 queries, TTFT P90 = 9,510.5ms, System TPS = 48,285.5 tokens/sec, interactivity = 31.4 tokens/sec per user and total provisioned system power = 100.89 kW. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Submission Info
NVIDIA
5a8af467-e182-4c57-9102-402cc0b5d16d
Submission Status REVIEW_PENDING
2026-11-07
2026-11-09
Version v1.0
Division Standardized
Deployment Client on Premises
Model
Model Name kimi-k3
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Datasets
Dataset Name agentic_combined
Dataset Type performance
Average Input Tokens —
Dataset Link —

System TPS vs. Tokens/s per User

010k20k30k40k50kSystem TPS406080100120Tokens/s per User

System TPS vs. Concurrency

010k20k30k40k50kSystem TPS05001,0001,500# of queries

Time to First Token P90 vs. Concurrency

02.0k4.0k6.0k8.0k10kTTFT P90 (ms)05001,0001,500# of queries

Tokens/s per User vs. Concurrency

406080100120Tokens/s per User05001,0001,500# of queries
SUT Summary
NVIDIA
NVIDIA Vera Rubin NVL72 (TRT-LLM + Dynamo)
Available
System Category — · —
32 accelerators
1
8
Max Supported Concurrency 1,536
Node Type ID 0
Node Count 8
Software
Ubuntu
Framework TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1
620
Link to Container kimi-k3-dynamo@sha256:2cd9c50158c5111884089499581d44093d0ec5193a9c2c13345c422732b90000
Operating System Ubuntu 26.04.1 LTS, Linux 7.0.0-2015-nvidia-bos-64k
Filesystem Lustre
Other Software See docs/software_disclosure.md
Software Notes A Dynamo frontend (Rust, conversation-affinity prefill routing) in front of TensorRT-LLM prefill and decode workers, with etcd and NATS for discovery. Speculative decoding uses a DSpark draft head.
Hardware

Processor

NVIDIA Vera CPU
2
88
VCPUs Per Processor 176

Accelerator

—
0
—
—
—
—

Host / Storage

1.5 TB per node (MemTotal 1,499,461,952 kB)
Memory Configuration LPDDR5X SOCAMM; two NUMA nodes per node, one per Vera CPU (MemTotal 745,599,040 kB and 753,862,912 kB)
Storage Capacity 32.6 TB per node (4 x 7.68 TB + 1 x 1.92 TB NVMe)
Storage Type NVMe SSD: 4 x Micron MTFDMBT7T6THM 7.68 TB and 1 x Micron MTFDLBT1T9THG 1.92 TB per node; /raid is a 6.0 TB ext4 LVM volume used as per-allocation scratch

Networking

10 per node: 8 ConnectX-9 (InfiniBand, 800 Gb/s XDR each) and 2 BlueField-4 integrated network controllers (2 Ethernet ports each, 400 Gb/s)
NVLink + InfiniBand
Network Topology —

Other

Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
Run Data
Field NameRun 1Run 2Run 3Run 4Run 5Run 6Run 7Run 8Run 9Run 10
Run Date11/07/202611/07/202611/07/202611/07/202611/07/202611/07/202611/07/202611/07/202611/07/202611/07/2026
Concurrency15243264961121925121,0241,536
System Tokens/Second1,841.92,652.23,280.75,513.77,262.97,910.110,848.825,689.241,414.248,285.5
Tokens/Second per User122.8110.5102.586.275.770.656.550.240.431.4
TTFT P90 (ms)493.8536.6576.4751.1964.11176.23009.13253.54673.99510.5
Utilization3.8%5.5%6.8%11.4%15.0%16.4%22.5%53.2%85.8%100.0%
Configuration Summarykimi-k3-dynamo-final · online · c15kimi-k3-dynamo-final · online · c24kimi-k3-dynamo-final · online · c32kimi-k3-dynamo-final · online · c64kimi-k3-dynamo-final · online · c96kimi-k3-dynamo-final · online · c112kimi-k3-dynamo-final · online · c192kimi-k3-dynamo-final · online · c512kimi-k3-dynamo-final · online · c1024kimi-k3-dynamo-final · online · c1536
Config Typeonlineonlineonlineonlineonlineonlineonlineonlineonlineonline
Config Region——————————
Target QPS——————————
Result Statecompletecompletecompletecompletecompletecompletecompletecompletecompletecomplete
Measured Accuracy Score——————————
Benchmark Versionunknownunknownunknownunknownunknownunknownunknownunknownunknownunknown
Result Version0.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.0
Archive URIgs://mlperf-endpoints-blob/runs/a15977f7-7807-491e-a06c-8f57124ec5b5/archive.tar.gzgs://mlperf-endpoints-blob/runs/6da3a059-1d34-41ae-ae8f-471e993ade36/archive.tar.gzgs://mlperf-endpoints-blob/runs/759d2776-eb3e-4adf-945f-0789753a2baa/archive.tar.gzgs://mlperf-endpoints-blob/runs/6ffab7cd-1aef-44f9-b0d2-20dd52658601/archive.tar.gzgs://mlperf-endpoints-blob/runs/653ad021-9686-4bed-8001-da72605e5612/archive.tar.gzgs://mlperf-endpoints-blob/runs/f4653f8d-7b1e-427f-a763-8e660f0175ae/archive.tar.gzgs://mlperf-endpoints-blob/runs/bd2a8346-0502-42ff-bcb9-9a26a58bc501/archive.tar.gzgs://mlperf-endpoints-blob/runs/f8279a75-6855-41a9-84e8-b43ddd062c5d/archive.tar.gzgs://mlperf-endpoints-blob/runs/0e23ae90-9b1d-4966-8d14-ed163f4d3702/archive.tar.gzgs://mlperf-endpoints-blob/runs/f026fc19-eeff-4abb-b5aa-a059dc6d2fcd/archive.tar.gz
QPS3.86015.56016.953611.626915.519316.813023.480656.011389.8196103.8714
Total Output Tokens9,247,3678,948,6088,698,99517,878,92926,484,49426,275,46833,126,99771,817,923106,231,033130,712,934
Average Output Tokens477.16477471.8474.22467.99470.47462.04458.65461.08464.86
Run Duration (s)5,020.623,374.042,651.563,242.663,646.563,321.783,053.502,795.652,565.082,707.09
Total Queries19,38018,76018,43837,70256,59255,84971,698156,588230,395281,189
Total Queries Issued19,38018,76018,43837,70256,59255,84971,698156,588230,395281,189
Total Queries Failed0000000000
Time to First Token (TTFT) (ms)
Minimum179.8186.0190.9195.7204.6201.81389.11415.21486.51500.1
Average372.1414.7461.3565.5670.4767.72378.82544.03253.65587.4
P50331.3359.3384.4464.7533.3595.42265.22357.42977.05124.4
P90493.8536.6576.4751.1964.11176.23009.13253.54673.99510.5
P95579.5655.3721.6977.51245.61497.03239.43703.65504.010819.4
P991162.21427.21588.82100.72451.22710.94399.25423.98588.114101.7
P9995244.78848.612696.818886.621847.720254.912848.614380.119619.521275.0
Maximum14244.418113.123291.023541.234702.585937.224759.244613.369381.999441.3
Time Per Output Token (TPOT) (ms)
Minimum0.000.310.623.021.650.581.320.010.010.02
Average4.265.045.657.539.189.929.4011.214.216.0
P504.194.945.547.498.999.869.3311.214.315.9
P905.286.286.988.7910.611.510.712.816.718.6
P955.676.737.449.2511.212.011.213.417.519.5
P996.627.748.6410.612.713.413.015.219.622.1
P9998.139.3710.613.016.016.921.728.725.231.1
Maximum27.8350.8918.9620.63489.61589.1907.92055.52407.93175.0
Query Latency (ms)
Minimum300.8313.6332.5409.7447.4459.41664.31734.31847.81879.4
Average2239.52603.32882.03769.34464.44898.16269.67144.69148.012260.9
P50887.61003.51106.91449.91750.71955.73598.63995.05196.88533.2
P904908.65755.26341.38400.09840.810755.111815.713750.917579.022097.1
P958596.09960.611207.814377.217082.718924.618792.522078.428360.134378.6
P9922997.626423.929157.738161.145509.448402.348228.955748.272889.186023.5
P99936456.842407.648173.660832.573203.176589.576518.090849.8118097.3133624.0
Maximum43889.654929.053543.272793.089380.5106503.7118191.5107787.7209115.0254390.8