Back previous/next report: k j

NVIDIA NVIDIA GB300 NVL72 (TRT-LLM + Dynamo)

Submitted by NVIDIA Corporationon 2026-10-15 Published on 2026-11-04

Result statement

NVIDIA Corporation ran kimi-k3 on NVIDIA GB300 NVL72 (TRT-LLM + Dynamo) and achieved a verified 13,905.8 tokens/second System TPS at 15,223.0ms TTFT P90 in the Standardized division of the Available category on November 4, 2026 .

[1] MLPerf Endpoints v1.0 kimi-k3, published November 4, 2026; retrieved from results.mlcommons.org on November 18, 2026. Result —, run 16: concurrency = 1,024 queries, TTFT P90 = 15,223.0ms, System TPS = 13,905.8 tokens/sec, interactivity = 13.6 tokens/sec per user and total provisioned system power = 90 kW. Result verified by MLCommons Association. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Submission Info
NVIDIA Corporation
b987824f-ff60-4444-b532-ac0d77e7229b
Submission Status PUBLISHED
2026-10-15
2026-11-04
Version v1.0
Division Standardized
Deployment Client on Premises
Model
Model Name kimi-k3
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Datasets
Dataset Name agentic_combined
Dataset Type performance
Average Input Tokens —
Dataset Link —

System TPS vs. Tokens/s per User

05.0k10k15kSystem TPS20406080Tokens/s per User

System TPS vs. Concurrency

05.0k10k15kSystem TPS02004006008001,000# of queries

Time to First Token P90 vs. Concurrency

05.0k10k15kTTFT P90 (ms)02004006008001,000# of queries

Tokens/s per User vs. Concurrency

20406080Tokens/s per User02004006008001,000# of queries
SUT Summary
NVIDIA
NVIDIA GB300 NVL72 (TRT-LLM + Dynamo)
Available
System Category — · —
48 accelerators
1
12
Max Supported Concurrency 1,024
Node Type ID 0
Node Count 12
Software
Ubuntu
Framework TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1
NVIDIA 580
Link to Container kimi-k3-dynamo@sha256:796bbbb499a61221fc04930179763f8be3c9af412782843bc9ec763cf2e4c2c8
Operating System Ubuntu Linux
Filesystem unspecified
Other Software unspecified
Software Notes —
Hardware

Processor

NVIDIA Grace (ARM Neoverse-V2)
2
144
VCPUs Per Processor 144

Accelerator

—
0
—
—
—
—

Host / Storage

unspecified
Memory Configuration unspecified
Storage Capacity unspecified
Storage Type unspecified

Networking

unspecified
NVLink + InfiniBand
Network Topology —

Other

Other Hardware —
Cooling Liquid-cooled
Hardware Notes Client and Dynamo frontend share a host; HTTP uses loopback with no external network hops. Prefill/decode workers use NVLink and InfiniBand.
Run Data
Field NameRun 1Run 2Run 3Run 4Run 5Run 6Run 7Run 8Run 9Run 10Run 11Run 12Run 13Run 14Run 15Run 16
Run Date10/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/202610/15/2026
Concurrency12131624324864801922883844485125766401,024
System Tokens/Second975.1945.21,068.61,413.81,720.22,210.42,650.43,030.35,897.17,514.78,938.39,567.710,614.311,179.712,091.213,905.8
Tokens/Second per User81.372.766.858.953.846.041.437.930.726.123.321.420.719.418.913.6
TTFT P90 (ms)1232.71240.11270.81511.51733.92305.13107.04295.66052.27442.48946.46071.36178.66482.97090.315223.0
Utilization7.0%6.8%7.7%10.2%12.4%15.9%19.1%21.8%42.4%54.0%64.3%68.8%76.3%80.4%87.0%100.0%
Configuration Summarykimi-k3-dynamo-final · online · c12kimi-k3-dynamo-final · online · c13kimi-k3-dynamo-final · online · c16kimi-k3-dynamo-final · online · c24kimi-k3-dynamo-final · online · c32kimi-k3-dynamo-final · online · c48kimi-k3-dynamo-final · online · c64kimi-k3-dynamo-final · online · c80kimi-k3-dynamo-final · online · c192kimi-k3-dynamo-final · online · c288kimi-k3-dynamo-final · online · c384kimi-k3-dynamo-final · online · c448kimi-k3-dynamo-final · online · c512kimi-k3-dynamo-final · online · c576kimi-k3-dynamo-final · online · c640kimi-k3-dynamo-final · online · c1024
Config Typeonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonlineonline
Config Region————————————————
Target QPS————————————————
Result Statecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecompletecomplete
Measured Accuracy Score————————————————
Benchmark Versiond3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fced3c57f2eba127c819e5a8fd83101d9e5f4e15fce
Result Version0.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.0
Archive URIgs://mlperf-endpoints-blob/runs/97067acb-7055-4639-8fa1-8389418d0a01/archive.tar.gzgs://mlperf-endpoints-blob/runs/dad8c41c-7749-4ca8-81ba-7370af09e3e2/archive.tar.gzgs://mlperf-endpoints-blob/runs/da0fec31-c6d1-4c4a-9043-d2ff2ab8aeef/archive.tar.gzgs://mlperf-endpoints-blob/runs/ae31d132-cc6e-4007-8ea4-c231f3974149/archive.tar.gzgs://mlperf-endpoints-blob/runs/b6445006-3b94-4927-95d3-618be3e1434c/archive.tar.gzgs://mlperf-endpoints-blob/runs/86a69f66-f609-4c6a-aa6a-2b28201ee43b/archive.tar.gzgs://mlperf-endpoints-blob/runs/25e2c4db-f229-48fb-93a7-9bb062e43b9b/archive.tar.gzgs://mlperf-endpoints-blob/runs/0516c949-e2b6-4cdb-997c-58a318d1ce7d/archive.tar.gzgs://mlperf-endpoints-blob/runs/afd033e8-67e8-44fd-8df7-a9db0cd99ac1/archive.tar.gzgs://mlperf-endpoints-blob/runs/3447de59-a445-4f66-9fae-07b02222a0e0/archive.tar.gzgs://mlperf-endpoints-blob/runs/78efcf44-8dd4-44d6-8d18-8ff2d6d6518c/archive.tar.gzgs://mlperf-endpoints-blob/runs/42e01778-f203-42d4-be26-cab52682d9a5/archive.tar.gzgs://mlperf-endpoints-blob/runs/632bb8b5-17e8-411b-a414-0b8c64c91644/archive.tar.gzgs://mlperf-endpoints-blob/runs/09ac0f12-c2e5-4017-bc41-3161fea25522/archive.tar.gzgs://mlperf-endpoints-blob/runs/d6b46956-8f9d-4130-b1bb-ee0c29602723/archive.tar.gzgs://mlperf-endpoints-blob/runs/1294146a-85b3-4ee8-88b4-7b21efcae7bc/archive.tar.gz
QPS2.08322.00212.25983.03173.65134.78195.67036.523912.719016.225419.191120.946923.173524.576726.447931.1159
Total Output Tokens9,152,0679,180,7869,127,7828,587,8768,514,62717,656,23126,976,82426,435,78142,098,02968,273,03055,946,49253,208,76561,449,11758,989,83757,952,85941,386,334
Average Output Tokens468.09472.09472.87466.33471.12462.24467.42464.49463.65463.15465.76456.76458.04454.89457.17446.91
Run Duration (s)9,385.509,713.418,542.076,074.444,949.727,987.8710,178.278,723.927,138.769,085.286,259.215,561.315,789.275,276.504,792.982,976.20
Total Queries19,55219,44719,30318,41618,07338,19757,71456,91490,798147,412120,121116,492134,158129,679126,76492,607
Total Queries Issued19,55219,44719,30318,41618,07338,19757,71456,91490,798147,412120,121116,492134,158129,679126,76492,607
Total Queries Failed0000000000000000
Time to First Token (TTFT) (ms)
Minimum324.9319.3336.9362.5378.5392.9412.8420.32232.52256.22251.12177.32210.12150.22270.42255.1
Average787.6789.0800.6925.31055.11321.41727.82426.34373.55091.95947.24806.54813.15122.95323.39194.1
P50632.6638.3656.4734.7794.5958.41242.81948.74021.14545.85027.34349.44405.64529.84681.56068.6
P901232.71240.11270.81511.51733.92305.13107.04295.66052.27442.48946.46071.36178.66482.97090.315223.0
P951573.81593.21605.71893.42226.12940.43847.45101.97231.18866.310935.06993.77091.87556.68444.719786.9
P993032.83017.92806.03299.23851.24447.75762.97130.110390.712836.116619.513054.012602.814631.816080.483315.1
P99911688.19661.510313.216898.324710.332698.840221.954818.437339.355063.880216.179179.064116.995101.885394.3148660.9
Maximum31356.031518.027492.446008.337400.851474.680083.583549.065864.1136005.8104486.8131005.4130059.3135230.9147934.0212096.4
Time Per Output Token (TPOT) (ms)
Minimum0.623.534.473.220.000.020.002.030.010.012.0311.810.211.211.910.9
Average7.369.0310.412.013.316.218.118.520.324.726.733.233.836.936.642.6
P507.048.8910.111.913.116.017.718.420.024.526.732.833.536.536.442.3
P909.7511.713.614.816.619.522.022.324.329.331.839.641.444.444.852.0
P9510.612.614.615.717.620.823.423.526.031.233.642.144.047.247.655.7
P9912.414.816.717.920.023.926.626.731.037.138.050.250.656.256.368.0
P99915.418.720.722.524.229.432.833.147.054.047.765.161.473.273.291.3
Maximum238.51621.4879.0745.2286.32636.23870.42563.15189.65154.35840.813514.34932.811139.64232.83536.6
Query Latency (ms)
Minimum513.5508.9567.3563.8650.0830.2755.8855.82700.02859.63044.42919.02970.02943.33187.23342.7
Average4127.94851.85400.76166.76912.18230.99507.310384.513096.115686.517484.418473.819173.520267.820559.226576.1
P501653.31867.62036.32341.72635.33349.34073.65011.77020.18303.49383.78790.18989.49516.59888.114051.9
P908751.310246.411907.513258.215519.218364.120106.321754.125201.930527.634124.837380.139275.941652.441372.855676.9
P9515051.018247.020964.523751.126791.231371.135394.637483.141750.450858.157241.064182.766131.870935.471192.792864.2
P9943469.051527.157446.364812.974392.079076.997848.097136.9108271.2130225.1148111.2164952.2174951.5183939.6186217.1226701.2
P99977668.187727.997620.4110196.2121363.0142887.0163923.9167012.2188275.6224794.6246582.3292314.4307566.7323702.8325232.1380076.9
Maximum95157.9107075.3136759.4135414.2153192.5192985.1202524.4207601.9260846.3324527.8305991.5352318.9383457.6640491.1420650.1594156.3