Back previous/next report: k j

Microsoft Azure ND GB300-v6

Submitted by Microsofton 2026-10-17 Published on 2026-11-18

Result statement

Microsoft ran kimi-k3 on Azure ND GB300-v6 and achieved a verified 14,050.1 tokens/second System TPS at 15,412.0ms TTFT P90 in the Standardized division of the Available category on November 18, 2026 .

[1] MLPerf Endpoints v1.0 kimi-k3, published November 18, 2026; retrieved from results.mlcommons.org on November 22, 2026. Result —, run 7: concurrency = 1,024 queries, TTFT P90 = 15,412.0ms, System TPS = 14,050.1 tokens/sec, interactivity = 13.7 tokens/sec per user and total provisioned system power = 101.23 kW (MLC Estimated Power). Result verified by MLCommons Association. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Submission Info
Microsoft
91176046-ce12-4045-ba1b-563966f84321
Submission Status PUBLISHED
2026-10-17
2026-11-18
Version v1.0
Division Standardized
Deployment Client on Premises
Model
Model Name kimi-k3
Division Standardized
Model Precision —
Model Link —
Calibration Link —
Model Notes —
Datasets
Dataset Name agentic_combined
Dataset Type performance
Average Input Tokens —
Dataset Link —

System TPS vs. Tokens/s per User

05.0k10k15kSystem TPS20406080Tokens/s per User

System TPS vs. Concurrency

05.0k10k15kSystem TPS02004006008001,000# of queries

Time to First Token P90 vs. Concurrency

05.0k10k15kTTFT P90 (ms)02004006008001,000# of queries

Tokens/s per User vs. Concurrency

20406080Tokens/s per User02004006008001,000# of queries
SUT Summary
NVIDIA
Azure ND GB300-v6
Available
System Category — · —
48 accelerators
1
12
Max Supported Concurrency 1,024
Node Type ID 0
Node Count 12
Software
Ubuntu
Framework TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1 + Dynamo 1.6.0.dev20260918
TensorRT-LLM 1.3.0rc25.post1
NVIDIA 580
Link to Container registry.gitlab.com/nvidia/mlperf-inference-partner/nv-mlpinf-partner/kimi-k3-dynamo@sha256:985260a01a2dfb66bdd0c68f0bd482e78f69c1b101120c7341782d101e2602a4
Operating System Ubuntu Linux
Filesystem unspecified
Other Software unspecified
Software Notes —
Hardware

Processor

NVIDIA Grace (ARM Neoverse-V2)
2
72
VCPUs Per Processor 144

Accelerator

—
0
—
—
—
—

Host / Storage

unspecified
Memory Configuration unspecified
Storage Capacity unspecified
Storage Type unspecified

Networking

unspecified
NVLink + InfiniBand
Network Topology —

Other

Other Hardware —
Cooling Liquid-cooled
Hardware Notes —
Run Data
Field NameRun 1Run 2Run 3Run 4Run 5Run 6Run 7
Run Date10/16/202610/16/202610/16/202610/16/202610/16/202610/16/202610/17/2026
Concurrency121632801925121,024
System Tokens/Second970.11,050.31,684.53,008.75,956.610,627.214,050.1
Tokens/Second per User80.865.652.637.631.020.813.7
TTFT P90 (ms)1289.71421.21881.84517.86128.86379.515412.0
Utilization6.9%7.5%12.0%21.4%42.4%75.6%100.0%
Configuration Summarykimi-k3-c12 · onlinekimi-k3-c16 · onlinekimi-k3-c32 · onlinekimi-k3-c80 · onlinekimi-k3-c192 · onlinekimi-k3-c512 · onlinekimi-k3-c1024 · online
Config Typeonlineonlineonlineonlineonlineonlineonline
Config Region———————
Target QPS———————
Result Statecompletecompletecompletecompletecompletecompletecomplete
Measured Accuracy Score———————
Benchmark Versionunknownunknownunknownunknownunknownunknownunknown
Result Version0.1.00.1.00.1.00.1.00.1.00.1.00.1.0
Archive URIgs://mlperf-endpoints-blob/runs/1cc8e4b5-00ad-4ae3-9175-a5e00a12f50f/archive.tar.gzgs://mlperf-endpoints-blob/runs/693915a1-ff79-4f2a-8907-7792e158021c/archive.tar.gzgs://mlperf-endpoints-blob/runs/cd48e664-4e46-4eac-9530-30725575b864/archive.tar.gzgs://mlperf-endpoints-blob/runs/6150517c-f955-45d5-bd27-11e13fb8394f/archive.tar.gzgs://mlperf-endpoints-blob/runs/6bbaba5f-f784-483e-9058-c3bea5dae689/archive.tar.gzgs://mlperf-endpoints-blob/runs/5adc9f1f-c628-42eb-9bd3-2f6596e906e1/archive.tar.gzgs://mlperf-endpoints-blob/runs/5311f0d7-313f-4cf5-b1bb-b40221ddb7cb/archive.tar.gz
QPS2.05052.23903.59836.370112.871023.234931.7085
Total Output Tokens9,302,4659,044,6218,482,92126,783,53142,217,79061,149,13841,157,022
Average Output Tokens473.12469.1468.15472.32462.79457.38443.1
Run Duration (s)9,588.758,611.425,035.788,902.107,087.565,754.022,929.31
Total Queries19,66219,28118,12056,70791,224133,69492,884
Total Queries Issued19,66219,28118,12056,70791,224133,69492,884
Total Queries Failed0000000
Time to First Token (TTFT) (ms)
Minimum370.8385.6420.8489.72258.22216.22271.6
Average866.5944.01208.62615.44393.14977.79426.2
P50720.3787.5900.72085.24025.94504.06063.4
P901289.71421.21881.84517.86128.86379.515412.0
P951635.21800.72494.55273.37344.27468.620056.9
P992987.83203.44686.07372.910552.013880.388876.0
P99915877.013639.530453.165811.538176.669592.5167566.8
Maximum35782.926440.549822.798101.185969.9138577.5215583.6
Time Per Output Token (TPOT) (ms)
Minimum2.874.411.480.020.0211.511.2
Average7.3510.213.618.720.033.941.3
P507.069.9913.118.419.833.241.1
P909.7413.516.522.323.741.250.4
P9510.514.517.623.625.143.854.0
P9912.416.519.826.728.550.365.5
P99915.620.224.432.835.861.389.7
Maximum211.9391.34023.44298.010891.512378.65567.6
Query Latency (ms)
Minimum560.1650.5757.3989.42796.83019.53227.8
Average4224.45491.67032.910715.512938.719249.726149.7
P501730.42171.12780.65201.86982.59117.413886.4
P908755.011852.815394.522213.025115.839075.454326.0
P9515494.320964.026829.838440.541359.566545.992355.4
P9945304.556985.468906.0102080.4106581.3175590.3211578.2
P99973312.193299.2121208.9170513.3182530.1310734.5365352.7
Maximum89339.3113740.3160399.4222038.7287420.8508610.1453614.0