Results
Menu

Endpoints › Agentic Model Results

Agentic: Throughput vs. E2E Average Interactivity

Remember to review the Messaging Guidelines when using this material.

0100,000200,000300,000400,000500,000600,000System Throughput (Tokens/s) — higher is better0100200300400500E2E Average Interactivity(Tokens/s per User) — constraint:minimum or equalMLPerf Endpoints v1.0,deepseek-v4_1-flash, Standardizeddivision: Krai HPE Cray XD670 (8xH200-SXM-141GB, vLLM) (Available,published November 10, 2026, 9.73 kWprovisioned power (MLC EstimatedPower)); Krai HPE Cray XD670 (8xH200-SXM-141GB, vLLM) (Available,published November 10, 2026, 9.73 kWprovisioned power (MLC EstimatedPower)); NVIDIA Vera Rubin NVL72 (vLLM +Dynamo) (Available, published November10, 2026, 75.67 kW provisioned power);Google LLCGB300-NVL72_GB300-288GB_aarch64x4_vLLM-Dynamo(Available, published November 4, 2026,8.43 kW provisioned power (MLC EstimatedPower)); NVIDIA Corporation NVIDIA GB300NVL72 (4-GPU SUT) (Available, publishedNovember 4, 2026, 7.5 kW provisionedpower); Krai Dell PowerEdge XE9680 (8xMI300X-OAM-192GB, vLLM + KraiMoE+sparseMLA kernels) (Available,published November 3, 2026, 9.9 kWprovisioned power); Krai Dell PowerEdgeXE9680 (8x MI300X-OAM-192GB, stock vLLM)(Available, published October 27, 2026,9.9 kW provisioned power); Krai DellPowerEdge XE9680 (8x MI300X-OAM-192GB,vLLM + Krai MoE kernels) (Available,published October 27, 2026, 9.9 kWprovisioned power); NVIDIA Vera RubinNVL72 (4-GPU SUT) (Available, publishedOctober 19, 2026, 12.61 kW provisionedpower); Krai HPE Cray XD670 (8xH200-SXM-141GB, vLLM) (Available,published October 13, 2026, 9.73 kWprovisioned power (MLC EstimatedPower)). Retrieved fromendpoints.mlcommons.org on November 13,2026. 8 of 10 results (square markers)are provisional. Pending MLCommonsAssociation verification, results aresubject to change pending peer review.Other results verified by MLCommonsAssociation. The MLPerf name and logoare registered and unregisteredtrademarks of MLCommons Association inthe United States and other countries.All rights reserved. Unauthorized usestrictly prohibited. Seewww.mlcommons.org for more information.

Results shown

10 of 10 results
Series color VersionStatus Total Provisioned System PowerMLPerf Result ID
KraiHPE Cray XD670 (8x H200-SXM-141GB, vLLM)deepseek-v4_1-flashStandardized2026-11-10v1.0 Available9.73 kW (MLC Estimated Power)1.0.1.deepseek-v4_1-flash.agentic_combined.1p
KraiHPE Cray XD670 (8x H200-SXM-141GB, vLLM)deepseek-v4_1-flashStandardized2026-11-10v1.0 Available9.73 kW (MLC Estimated Power)1.0.1.deepseek-v4_1-flash.agentic_combined.2p
NVIDIANVIDIA Vera Rubin NVL72 (vLLM + Dynamo)deepseek-v4_1-flashStandardized2026-11-10v1.0 Available75.67 kW1.0.1.deepseek-v4_1-flash.mlperf_agentic_inference_dataset.1p
Google LLCGB300-NVL72_GB300-288GB_aarch64x4_vLLM-Dynamodeepseek-v4_1-flashStandardized2026-11-04v1.0 Available8.43 kW (MLC Estimated Power)1.0.0.deepseek-v4_1-flash.mlperf_agentic_inference_dataset.2
NVIDIA CorporationNVIDIA GB300 NVL72 (4-GPU SUT)deepseek-v4_1-flashStandardized2026-11-04v1.0 Available7.5 kW1.0.0.deepseek-v4_1-flash.mlperf_agentic_inference_dataset.1
KraiDell PowerEdge XE9680 (8x MI300X-OAM-192GB, vLLM + Krai MoE+sparseMLA kernels)deepseek-v4_1-flashStandardized2026-11-03v1.0 Available9.9 kW1.0.0.deepseek-v4_1-flash.agentic_combined.2p
KraiDell PowerEdge XE9680 (8x MI300X-OAM-192GB, stock vLLM)deepseek-v4_1-flashStandardized2026-10-27v1.0 Available9.9 kW1.0.0.deepseek-v4_1-flash.agentic_combined.4p
KraiDell PowerEdge XE9680 (8x MI300X-OAM-192GB, vLLM + Krai MoE kernels)deepseek-v4_1-flashStandardized2026-10-27v1.0 Available9.9 kW1.0.0.deepseek-v4_1-flash.agentic_combined.3p
NVIDIANVIDIA Vera Rubin NVL72 (4-GPU SUT)deepseek-v4_1-flashStandardized2026-10-19v1.0 Available12.61 kW1.0.0.deepseek-v4_1-flash.mlperf_agentic_inference_dataset.3p
KraiHPE Cray XD670 (8x H200-SXM-141GB, vLLM)deepseek-v4_1-flashStandardized2026-10-13v1.0 Available9.73 kW (MLC Estimated Power)1.0.0.deepseek-v4_1-flash.agentic_combined.1p

MLPerf Endpoints v1.0, deepseek-v4_1-flash, Standardized division: Krai HPE Cray XD670 (8x H200-SXM-141GB, vLLM) (Available, published November 10, 2026, 9.73 kW provisioned power (MLC Estimated Power)); Krai HPE Cray XD670 (8x H200-SXM-141GB, vLLM) (Available, published November 10, 2026, 9.73 kW provisioned power (MLC Estimated Power)); NVIDIA Vera Rubin NVL72 (vLLM + Dynamo) (Available, published November 10, 2026, 75.67 kW provisioned power); Google LLC GB300-NVL72_GB300-288GB_aarch64x4_vLLM-Dynamo (Available, published November 4, 2026, 8.43 kW provisioned power (MLC Estimated Power)); NVIDIA Corporation NVIDIA GB300 NVL72 (4-GPU SUT) (Available, published November 4, 2026, 7.5 kW provisioned power); Krai Dell PowerEdge XE9680 (8x MI300X-OAM-192GB, vLLM + Krai MoE+sparseMLA kernels) (Available, published November 3, 2026, 9.9 kW provisioned power); Krai Dell PowerEdge XE9680 (8x MI300X-OAM-192GB, stock vLLM) (Available, published October 27, 2026, 9.9 kW provisioned power); Krai Dell PowerEdge XE9680 (8x MI300X-OAM-192GB, vLLM + Krai MoE kernels) (Available, published October 27, 2026, 9.9 kW provisioned power); NVIDIA Vera Rubin NVL72 (4-GPU SUT) (Available, published October 19, 2026, 12.61 kW provisioned power); Krai HPE Cray XD670 (8x H200-SXM-141GB, vLLM) (Available, published October 13, 2026, 9.73 kW provisioned power (MLC Estimated Power)). Retrieved from endpoints.mlcommons.org on November 13, 2026. 8 of 10 results (square markers) are provisional. Pending MLCommons Association verification, results are subject to change pending peer review. Other results verified by MLCommons Association. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Selected result

Select a point on the chart to see additional detail.