Back previous/next report: k j

NVIDIA NVIDIA Vera Rubin NVL72 (TRT-LLM + Dynamo)

Submitted by NVIDIA Corporationon 2026-10-16 Published on 2026-11-04

Result statement

NVIDIA Corporation ran kimi-k3 on NVIDIA Vera Rubin NVL72 (TRT-LLM + Dynamo) and achieved a verified 25,495.4 tokens/second System TPS at 25,719.8ms TTFT P90 in the Standardized division of the Available category on November 4, 2026 .

[1] MLPerf Endpoints v1.0 kimi-k3, published November 4, 2026; retrieved from results.mlcommons.org on November 18, 2026. Result —, run 13: concurrency = 1,536 queries, TTFT P90 = 25,719.8ms, System TPS = 25,495.4 tokens/sec, interactivity = 16.6 tokens/sec per user and total provisioned system power = 151.33 kW. Result verified by MLCommons Association. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Submission Info
NVIDIA Corporation
6f73baf8-4231-489b-9293-1ef0c0c97344
Submission Status PUBLISHED
2026-10-16
2026-11-04
Version v1.0
Division Standardized
Deployment Client on Premises
Model
Model Name kimi-k3
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Datasets
Dataset Name agentic_combined
Dataset Type performance
Average Input Tokens —
Dataset Link —

System TPS vs. Tokens/s per User

05.0k10k15k20k25kSystem TPS20406080100Tokens/s per User

System TPS vs. Concurrency

05.0k10k15k20k25kSystem TPS05001,0001,500# of queries

Time to First Token P90 vs. Concurrency

05.0k10k15k20k25kTTFT P90 (ms)05001,0001,500# of queries

Tokens/s per User vs. Concurrency

20406080100Tokens/s per User05001,0001,500# of queries
SUT Summary
NVIDIA
NVIDIA Vera Rubin NVL72 (TRT-LLM + Dynamo)
Available
System Category — · —
48 accelerators
1
12
Max Supported Concurrency 1,536
Node Type ID 0
Node Count 12
Software
Ubuntu
Framework TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1
620
Link to Container kimi-k3-dynamo@sha256:796bbbb499a61221fc04930179763f8be3c9af412782843bc9ec763cf2e4c2c8
Operating System Ubuntu Linux
Filesystem Lustre
Other Software unspecified
Software Notes —
Hardware

Processor

NVIDIA Vera CPU
2
176
VCPUs Per Processor 352

Accelerator

—
0
—
—
—
—

Host / Storage

unspecified
Memory Configuration unspecified
Storage Capacity unspecified
Storage Type unspecified

Networking

unspecified
NVLink + InfiniBand
Network Topology —

Other

Other Hardware —
Cooling Liquid-cooled
Hardware Notes Client and Dynamo frontend share a host; HTTP uses loopback with no external network hops. Prefill/decode workers use NVLink and InfiniBand.
Run Data
Field NameRun 1Run 2Run 3Run 4Run 5Run 6Run 7Run 8Run 9Run 10Run 11Run 12Run 13
Run Date10/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/2026
Concurrency151840485664961921932565121,0241,536
System Tokens/Second1,564.31,718.93,589.53,607.54,482.64,412.65,569.69,940.09,382.911,259.716,172.423,260.825,495.4
Tokens/Second per User104.395.589.775.280.068.958.051.848.644.031.622.716.6
TTFT P90 (ms)553.0589.1734.3829.2715.81165.32062.71286.01280.72280.54897.515712.725719.8
Utilization6.1%6.7%14.1%14.1%17.6%17.3%21.8%39.0%36.8%44.2%63.4%91.2%100.0%
Configuration Summarykimi-k3-dynamo-final · online · c15kimi-k3-dynamo-final · online · c18kimi-k3-dynamo-final · online · c40kimi-k3-dynamo-final · online · c48kimi-k3-dynamo-final · online · c56kimi-k3-dynamo-final · online · c64kimi-k3-dynamo-final · online · c96kimi-k3-dynamo-final · online · c192kimi-k3-dynamo-final · online · c193kimi-k3-dynamo-final · online · c256kimi-k3-dynamo-final · online · c512kimi-k3-dynamo-final · online · c1024kimi-k3-dynamo-final · online · c1536
Config Typeonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonline
Config Region—————————————
Target QPS—————————————
Result Statecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecomplete
Measured Accuracy Score—————————————
Benchmark Versionunknownunknownd3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fceunknownunknownd3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fce
Result Version0.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.0
Archive URIgs://mlperf-endpoints-blob/runs/de652d3a-ba81-4aed-a4ab-4a346046ad1b/archive.tar.gzgs://mlperf-endpoints-blob/runs/caace767-32d5-4a1a-8dd3-52c8cdc993e0/archive.tar.gzgs://mlperf-endpoints-blob/runs/687e8dfe-6ff5-49f3-af7f-1bae1b650728/archive.tar.gzgs://mlperf-endpoints-blob/runs/080eee8a-141a-4445-b2a9-6e22688483f4/archive.tar.gzgs://mlperf-endpoints-blob/runs/0404b876-66c7-4fab-be7a-bb3f35ed46b4/archive.tar.gzgs://mlperf-endpoints-blob/runs/3aef0054-e4c1-491f-95df-1302be6a304d/archive.tar.gzgs://mlperf-endpoints-blob/runs/90f5a34d-2a51-4112-9956-a9f74c90c9d9/archive.tar.gzgs://mlperf-endpoints-blob/runs/6ffb47e2-4c87-426e-a0a0-5e7fb4904256/archive.tar.gzgs://mlperf-endpoints-blob/runs/5db272a2-fa36-4f4e-99e6-823dbc5a0af8/archive.tar.gzgs://mlperf-endpoints-blob/runs/d6b9e94f-113a-44f8-83b8-b674be66f0c1/archive.tar.gzgs://mlperf-endpoints-blob/runs/5efac990-6501-4481-b5f7-1160b8457b2f/archive.tar.gzgs://mlperf-endpoints-blob/runs/03442ff1-1b14-43d3-8bb1-bf0b41e40d41/archive.tar.gzgs://mlperf-endpoints-blob/runs/ba78df10-2cbd-4dcb-a047-303138830a76/archive.tar.gz
QPS3.30243.61567.67377.78309.53969.269711.968621.603020.559424.668635.283750.375255.3366
Total Output Tokens9,099,2789,101,77918,142,85817,774,41817,876,69817,940,27416,788,99233,128,61932,746,53640,533,03961,554,11666,448,72374,156,831
Average Output Tokens473.67475.41467.77463.52469.9476.02465.35460.12456.38456.44458.35461.75460.74
Run Duration (s)5,816.985,295.055,054.404,927.123,987.994,065.703,014.393,332.873,490.033,599.843,806.122,856.682,908.63
Total Queries19,21019,14538,78638,34838,04437,68836,07872,00071,75388,803134,294143,906160,954
Total Queries Issued19,21019,14538,78638,34838,04437,68836,07872,00071,75388,803134,294143,906160,954
Total Queries Failed0000000000000
Time to First Token (TTFT) (ms)
Minimum182.2185.4193.0224.2214.4224.5248.8226.4235.8230.41489.21531.01616.2
Average410.2435.9536.8589.6532.7747.81121.0839.6839.51285.63748.77497.713087.0
P50346.9372.5446.1473.5418.9567.3818.8597.8606.7970.83272.04801.210112.7
P90553.0589.1734.3829.2715.81165.32062.71286.01280.72280.54897.515712.725719.8
P95704.2723.0915.71059.2970.81557.62544.21727.11758.82850.75873.319736.530901.1
P991444.51608.71922.42283.82329.42781.24096.53725.44247.35151.89816.634673.462113.2
P9996748.18233.214737.714704.912317.719778.522921.124109.219345.020885.965253.271793.9115780.4
Maximum18997.415957.727589.927262.626996.429726.324673.464943.441909.158090.890948.7133825.6149361.3
Time Per Output Token (TPOT) (ms)
Minimum0.01-0.000.53-0.012.873.960.010.000.010.070.030.010.04
Average5.436.326.749.048.179.9311.514.315.416.619.622.524.5
P505.336.236.638.958.019.8711.214.115.116.419.422.224.1
P906.857.968.4610.510.211.613.916.618.619.323.226.429.2
P957.378.599.1011.111.012.214.917.620.020.424.528.031.3
P998.549.9610.612.712.913.817.320.023.323.327.832.937.4
P99910.612.213.515.716.117.921.324.629.130.734.258.273.1
Maximum97.5216.6903.21950.3817.1361.82726.42550.33961.52525.75098.27307.311321.6
Query Latency (ms)
Minimum321.1333.8364.5264.6401.3455.8524.7605.2563.6650.12036.42084.72497.5
Average2884.83311.53547.44467.54170.25135.76078.46911.47383.68271.812066.117037.123440.0
P501062.31185.21332.71643.11483.21964.32636.52498.92639.33376.36062.310382.416455.5
P906296.17309.17725.19734.29406.911289.012975.615406.916514.818086.123949.633776.644136.3
P9511142.112744.213556.116931.316104.119405.222472.927290.728513.630794.241241.752605.967137.4
P9931428.636155.137997.648019.445294.955170.761209.773612.977970.683403.0105897.2124658.3145271.1
P99953827.761478.467565.681928.980855.890165.4105568.1127530.1144499.7146198.8181734.8209270.6229762.5
Maximum70869.0109979.784574.2129553.5133877.8171992.1125083.7229480.2188326.9173902.8301287.6254284.0293699.3