Back previous/next report: k j

Google GB300-NVL72_GB300-288GB_aarch64x32_TRTLLM-Dynamo

Submitted by Google LLCon 2026-10-16 Published on 2026-11-04

Result statement

Google LLC ran kimi-k3 on GB300-NVL72_GB300-288GB_aarch64x32_TRTLLM-Dynamo and achieved a verified 13,560.5 tokens/second System TPS at 15,995.5ms TTFT P90 in the Standardized division of the Available category on November 4, 2026 .

[1] MLPerf Endpoints v1.0 kimi-k3, published November 4, 2026; retrieved from results.mlcommons.org on November 18, 2026. Result —, run 10: concurrency = 1,024 queries, TTFT P90 = 15,995.5ms, System TPS = 13,560.5 tokens/sec, interactivity = 13.2 tokens/sec per user and total provisioned system power = 67.48 kW (MLC Estimated Power). Result verified by MLCommons Association. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Submission Info
Google LLC
bdce9746-c2fa-4967-a927-a7c5304135bf
Submission Status PUBLISHED
2026-10-16
2026-11-04
Version v1.0
Division Standardized
Deployment Client on Premises
Model
Model Name kimi-k3
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Datasets
Dataset Name agentic_combined
Dataset Type performance
Average Input Tokens —
Dataset Link —

System TPS vs. Tokens/s per User

05.0k10kSystem TPS204060Tokens/s per User

System TPS vs. Concurrency

05.0k10kSystem TPS02004006008001,000# of queries

Time to First Token P90 vs. Concurrency

05.0k10k15kTTFT P90 (ms)02004006008001,000# of queries

Tokens/s per User vs. Concurrency

204060Tokens/s per User02004006008001,000# of queries
SUT Summary
NVIDIA
GB300-NVL72_GB300-288GB_aarch64x32_TRTLLM-Dynamo
Available
System Category Datacenter · —
8 nodes x 4 NVIDIA GB300 (32 GPUs) of one NVL72 domain
1
8
Max Supported Concurrency 1,024
Node Type ID 0
Node Count 8
Software
Framework TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1
NVIDIA 580.173.02 (GKE 'default' COS driver; CUDA 13.0 driver API)
Link to Container kimi-k3-dynamo@sha256:985260a01a2dfb66bdd0c68f0bd482e78f69c1b101120c7341782d101e2602a4 (built from public sources via src/trtllm-dynamo-gke/container/build.sh: https://github.com/NVIDIA/TensorRT-LLM.git feat/k3_26.08 @ 45c5ae41 + https://github.com/ai-dynamo/dynamo.git @ a9792dbe + PyPI ai-dynamo==1.6.0.dev20260918)
Operating System Container-Optimized OS, kernel 6.12.94+, GKE 1.35.8-gke.1036000
Filesystem ext4 on node-local NVMe SSD (/mnt/stateful_partition/kube-ephemeral-ssd)
Other Software MLCommons endpoints client ghcr.io/mlcommons/endpoints@sha256:ba85b060b9c1de0d3b44d52f6fce8369e81f88ac31c460807cdc1e61441249bf (tag 89904fb); containerd 2.2.7
Software Notes —
Hardware

Processor

NVIDIA Grace (Arm Neoverse V2)
0
—
VCPUs Per Processor 144

Accelerator

—
0
—
—
—
—

Host / Storage

~943 GiB per node (kubectl capacity 988634624Ki)
Memory Configuration 960 GB LPDDR5X (2x NVIDIA Grace CPU integrated memory)
Storage Capacity ~12.5 TB local SSD per node (kubectl ephemeral-storage)
Storage Type —

Networking

4x NVIDIA ConnectX-7 + 1x Google Cloud Titanium NIC (a4x-maxgpu-4g-metal)
Google Cloud Titanium / RoCE + NVLink 5 (NVL72)
Network Topology —

Other

Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
Run Data
Field NameRun 1Run 2Run 3Run 4Run 5Run 6Run 7Run 8Run 9Run 10
Run Date10/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/16/2026
Concurrency131624324864804486401,024
System Tokens/Second912.21,033.11,366.91,650.92,161.42,605.02,976.19,351.811,807.113,560.5
Tokens/Second per User70.264.657.051.645.040.737.220.918.413.2
TTFT P90 (ms)1287.61350.41600.11808.32362.83202.64496.45943.87473.215995.5
Utilization6.7%7.6%10.1%12.2%15.9%19.2%21.9%69.0%87.1%100.0%
Configuration Summarykimi-k3-c13 · onlinekimi-k3-c16 · onlinekimi-k3-c24 · onlinekimi-k3-c32 · onlinekimi-k3-c48 · onlinekimi-k3-c64 · onlinekimi-k3-c80 · onlinekimi-k3-c448 · onlinekimi-k3-c640 · onlinekimi-k3-c1024 · online
Config Typeonlineonlineonlineonlineonlineonlineonlineonlineonlineonline
Config Region——————————
Target QPS——————————
Result Statecompletecompletecompletecompletecompletecompletecompletecompletecompletecomplete
Measured Accuracy Score——————————
Benchmark Versionunknownunknownunknownunknownunknownunknownunknownunknownunknownunknown
Result Version0.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.00.1.0
Archive URIgs://mlperf-endpoints-blob/runs/69f46b7b-6e58-407a-a6cf-8c4c2ead3037/archive.tar.gzgs://mlperf-endpoints-blob/runs/eee8bd4b-03bf-4913-a6b7-4f1bb67d4653/archive.tar.gzgs://mlperf-endpoints-blob/runs/ab39afc7-91b6-4c72-9ca4-f2265d8bef57/archive.tar.gzgs://mlperf-endpoints-blob/runs/f6952fc3-110c-4458-ab17-c2694fffe2db/archive.tar.gzgs://mlperf-endpoints-blob/runs/6ebd80e5-06d4-4465-915f-cad198c07413/archive.tar.gzgs://mlperf-endpoints-blob/runs/e4cc5006-64dc-44e3-a7c0-986665d0eb0d/archive.tar.gzgs://mlperf-endpoints-blob/runs/1c8f2015-e809-4e26-a569-a420131e80c0/archive.tar.gzgs://mlperf-endpoints-blob/runs/a78d2b72-e925-406a-bbe8-3247c48ef48a/archive.tar.gzgs://mlperf-endpoints-blob/runs/69573621-2b86-4aa4-907d-3aa60a28417f/archive.tar.gzgs://mlperf-endpoints-blob/runs/fa129779-dd28-4bea-afb2-08965911fb59/archive.tar.gz
QPS1.91912.19752.94273.51174.61395.59156.374120.592525.952630.4515
Total Output Tokens9,270,5099,016,2838,571,5078,539,00417,821,26626,839,44726,503,27652,574,58285,079,63841,546,986
Average Output Tokens475.34470.14464.5470.11468.48465.89466.91454.15454.95445.32
Run Duration (s)10,162.648,727.036,270.675,172.418,245.1110,302.918,905.315,621.857,205.823,063.82
Total Queries19,50319,17818,45318,16438,04257,60956,763115,768187,01093,298
Total Queries Issued19,50319,17818,45318,16438,04257,60956,763115,768187,01093,298
Total Queries Failed0000000000
Time to First Token (TTFT) (ms)
Minimum360.7378.6391.7411.0426.9442.5453.92191.12228.82313.1
Average843.5878.11007.01137.01391.51790.32553.14767.75460.69618.9
P50690.7733.1809.3869.91029.21307.32047.14336.64813.46090.6
P901287.61350.41600.11808.32362.83202.64496.45943.87473.215995.5
P951636.61710.82025.22351.12997.83962.25316.56847.38988.021099.0
P993062.32922.53557.64103.64668.35834.57312.113090.316893.190708.1
P99911856.412372.019451.627804.131731.340589.657010.279821.875574.4175393.6
Maximum43207.740401.231545.143921.052128.168019.884695.8122065.5140412.3222547.6
Time Per Output Token (TPOT) (ms)
Minimum1.852.534.054.944.610.054.6410.111.511.5
Average9.3910.912.413.916.518.419.034.240.143.6
P509.3610.412.413.616.418.118.733.839.743.3
P9012.214.015.217.120.122.522.840.848.653.1
P9513.115.016.218.321.423.924.043.451.757.0
P9915.317.218.420.724.527.227.351.861.769.2
P99919.421.222.226.030.333.334.267.081.492.1
Maximum121.01962.0328.32745.2972.01541.74096.17712.017249.17585.3
Query Latency (ms)
Minimum530.8644.3682.1720.4849.3849.7944.92932.43131.33298.2
Average5133.15627.16402.17234.28607.59715.410709.418792.021953.627338.1
P501959.62143.42495.62777.73437.04197.05209.28847.810492.714459.8
P9011049.312028.213960.115604.718533.920858.722077.038094.244457.256024.0
P9519719.821097.324398.027453.732429.236288.738263.765707.775237.396429.5
P9953752.661104.767581.577773.687956.894266.599077.5166727.2200233.7223361.4
P99993276.799592.2114520.8127635.5152434.7166732.8171846.4296736.6354862.2389814.4
Maximum110499.8117973.8130854.1154993.2198692.7223672.5250971.3572379.8765624.2477499.6