Inference Datacenter v3.1


1	ID	Submitter	System	Nodes	Processor	p#	Accelerator	a#	Software		Result																				Details	Code	Notes
2										Task	Image classification		Object detection		Medical imaging		Speech-to-text		Natural Language Processing				Recommendation				Large Language Model
3										Data	ImageNet		OpenImages (800x800)		KiTS19		LibriSpeech		SQuAD v1.1				Criteo 4TB				CNN-DailyMail News
4										Model	ResNet		Retinanet		3D-UNet		RNN-T		BERT				dlrm-v2-99		dlrm-v2-99.9		gptj-99		gptj-99.9
5										Accuracy	99.00		99.00		99.00	99.90	99.00		99.00		99.90		99.00		99.90		99.00		99.90
6										Scenario	Server	Offline	Server	Offline	Offline	Offline	Server	Offline	Server	Offline	Server	Offline	Server	Offline	Server	Offline	Server	Offline	Server	Offline
7										Units	Queries/s	Samples/s	Queries/s	Samples/s	Samples/s	Samples/s	Queries/s	Samples/s	Queries/s	Samples/s	Queries/s	Samples/s	Queries/s	Samples/s	Queries/s	Samples/s	Queries/s	Samples/s	Queries/s	Samples/s

8	Available
9	3.1-0001	ASUSTeK	ESC4000A-E12 (8x L4, TensorRT)	1	AMD EPYC 9654 96-Core Processor	1	NVIDIA L4	8	TensorRT 9.0.0, CUDA 12.2		105,512.00	107,899.00	1,601.45	1,822.58	8.77	8.77	30,512.70	32,393.90	7,304.64	7,559.83	5,005.01	5,186.72									details	code
10	3.1-0002	ASUSTeK	ESC8000A-E12 (8xH100-PCIe-80GB, TensorRT	1	AMD EPYC 9654 96-Core Processor	2	NVIDIA H100-PCIe-80GB	8	TensorRT 9.0.0, CUDA 12.2		400,094.00	451,941.00	8,402.57	9,186.07	37.73	37.73	120,017.00	144,446.00	36,812.70	46,010.90	32,012.90	38,875.00	175,017.00	201,716.00	175,017.00	199,526.00					details	code
11	3.1-0003	Azure	ND_H100_v5 (8x H100-SXM-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480C	2	NVIDIA H100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2																		84.22	105.76	84.22	105.76	details	code
12	3.1-0004	CTuning	Google Cloud Platform (g2.standard.4)	1	Intel(R) Xeon(R) CPU @ 2.20GHz	1	NVIDIA L4	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6		11,703.80	12,557.80	144.90	169.04	1.05	1.05	3,654.74	3,818.19	863.86	893.47	369.71	406.71									details	code	Powered by MLCommons CM automation language and CK playground.
13	3.1-0005	CTuning	AWS cloud instance g4dn.xlarge	1	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	1	NVIDIA T4	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6		4,604.05	6,049.73	59.99	84.13	0.46	0.46			289.68	435.81											details	code	Powered by MLCommons CM automation language and CK playground.
14	3.1-0058	Dell	Dell PowerEdge R750xa (4x A100-PCIe-80GB, TensorRT)	1	Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz	2	NVIDIA A100-PCIe-80GB	4	TensorRT 9.0.0, CUDA 12.2		147,251.00	158,285.00							11,703.80	12,652.90	5,854.85	6,533.00	50,017.40	63,639.60	50,017.40	63,639.60					details	code
15	3.1-0059	Dell	Dell PowerEdge Server R760 (1x Intel Xeon Platinum 8480+)	1	Intel(R) Xeon(R) Platinum 8480+	2			PyTorch		16,006.70	20,562.60	199.74	282.63		1.67			1,054.34	1,386.38							0.59	2.07			details	code
16	3.1-0060	Dell	Dell PowerEdge Server R760 (1x Intel Xeon Platinum 8480+)	1	Intel(R) Xeon(R) Platinum 8480+	2			PyTorch								4,204.80	5,784.79													details	code
17	3.1-0061	Dell	Dell PowerEdge R750xa (4x A100-PCIe-80GB, TensorRT)	1	Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz	2	NVIDIA A100-PCIe-80GB	4	TensorRT 9.0.0, CUDA 12.2				2,703.40	2,970.43	14.22	14.22	49,616.60	54,070.00									13.81	14.70			details	code
18	3.1-0062	Dell	Dell PowerEdge R750xa (4x H100-PCIe-80GB, TensorRT)	1	Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz	2	NVIDIA H100-PCIe-80GB	4	TensorRT 9.0.0, CUDA 12.2		196,555.00	191,776.00	4,280.81	2,035.91	17.81	17.82	64,620.20	68,571.90	15,321.80	21,392.50	14,966.90	17,895.50	92,240.40	92,569.80	92,240.40	92,569.80					details	code	NVIDIA H100-PCIe-80GB (TDP: 310W)
19	3.1-0063	Dell	Dell PowerEdge R760xa (2x H100-PCIe-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480+	2	NVIDIA H100-PCIe-80GB	2	TensorRT 9.0.0, CUDA 12.2		106,002.00	115,122.00	2,248.32	2,308.93	9.40	9.40	34,054.40	36,405.50	9,146.73	11,834.20	8,252.69	9,824.04									details	code
20	3.1-0064	Dell	Dell PowerEdge R760xa (4x H100-PCIe-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480+	2	NVIDIA H100-PCIe-80GB	4	TensorRT 9.0.0, CUDA 12.2		206,018.00	200,132.00	4,354.53	2,557.39	18.66	18.66	68,021.40	71,644.90	18,069.60	22,940.60	16,306.80	19,467.10	98,916.10	99,964.80	98,916.10	99,964.80					details	code
21	3.1-0065	Dell	Dell PowerEdge R760xa (4x L40, TensorRT)	1	Intel(R) Xeon(R) Platinum 8460Y+	2	NVIDIA L40	4	TensorRT 9.0.0, CUDA 12.2		130,021.00	115,291.00	1,901.51	2,069.79	13.21	13.20			8,619.13	8,311.43	3,505.20	3,853.08					13.81	18.50	13.81	18.50	details	code
22	3.1-0066	Dell	Dell PowerEdge XE8640 (4x NVIDIA H100-SXM-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8468	2	NVIDIA H100-SXM-80GB	4	TensorRT 9.0.0, CUDA 12.2		311,998.00	353,940.00	6,756.77	6,888.28	25.42	25.42	96,223.30	95,863.90	29,092.20	36,155.70	25,506.80	31,595.10	135,702.00	174,733.00	135,702.00	174,733.00	41.00	51.80	41.00	51.80	details	code
23	3.1-0067	Dell	Dell PowerEdge XE9640 (4x H100-SXM-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480+	2	NVIDIA H100-SXM-80GB	4	TensorRT 9.0.0, CUDA 12.2		305,053.00	352,037.00	6,656.63	6,869.47			90,021.90	93,774.30	28,011.80	35,402.30	24,807.90	31,369.40					41.00	51.99	41.00	51.99	details	code
24	3.1-0068	Dell	Dell PowerEdge XE9680 (8x A100-SXM-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480+	2	NVIDIA A100-SXM-80GB CTS	8	TensorRT 9.0.0, CUDA 12.2														145,021.00	149,673.00	145,021.00	149,673.00	33.17	42.32			details	code	NVIDIA A100-SXM4-80GB (TDP: 500W)
25	3.1-0069	Dell	Dell PowerEdge XE9680 (8x H100-SXM-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8470	2	NVIDIA H100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		620,874.00	706,858.00	12,484.80	14,055.80	51.10	51.10	178,016.00	187,469.00	57,331.00	70,307.40	51,217.90	62,520.90	326,049.00	344,370.00	326,049.00	344,370.00	81.28	101.83	81.28	101.83	details	code
26	3.1-0076	Fujitsu	PRIMERGY_CDI_V1 (4x A100-PCIe-80GB, TensorRT)	1	Intel(R) Xeon(R) Gold 6430 CPU @ 2.10GHz	2	NVIDIA A100-PCIE-80GB	4	TensorRT 9.0.0, CUDA 12.2		130,599.00	155,688.00	2,916.47	2,972.54	13.92	13.92	48,216.90	52,506.10	11,848.40	12,496.10	5,787.45	6,416.10									details	code	GPUs are installed in an external PCIe box.
27	3.1-0077	GigaComputing	GIGABYTE G593-SD0	1	Intel(R) Xeon(R) Platinum 8480+	2	NVIDIA H100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		584,197.00	706,531.00	12,884.60	14,170.40	51.42	51.50	180,017.00	161,453.00	57,222.00	71,212.80	49,617.50	62,556.40	323,049.00	340,121.00	323,049.00	340,121.00	82.26	103.45	82.26	103.45	details	code
28	3.1-0078	H3C	H3C UniServer R5300 G6 (8x L40, TensorRT)	1	Intel(R) Xeon(R) Platinum 8458P	2	NVIDIA L40	8	TensorRT 9.0.0, CUDA 12.2		282,029.00	259,974.00	4,904.60	5,268.21	26.91	26.91	81,015.50	84,025.20	18,567.70	17,448.40											details	code
29	3.1-0079	H3C	H3C UniServer R5350 G6 (8x L40, TensorRT)	1	AMD EPYC 9754 128-Core Processor	2	NVIDIA L40	8	TensorRT 9.0.0, CUDA 12.2		280,029.00	246,554.00	4,874.86	5,272.14	26.78	26.78	80,815.20	86,358.50	18,507.70	15,360.80											details	code
30	3.1-0081	HPE	1-node-2S-SPR-PyTorch-INT8	1	Intel(R) Xeon(R) Platinum 8480+	2			PyTorch		16,306.80	20,631.70	199.74	276.25		1.59			1,104.65	1,469.64							0.59	2.14			details	code	HPE ProLiant DL380a Gen11. N/A
31	3.1-0082	HPE	1-node-2S-SPR-PyTorch-MIX	1	Intel(R) Xeon(R) Platinum 8480+	2			PyTorch								4,104.84	5,968.03													details	code	HPE ProLiant DL380a Gen11. N/A
32	3.1-0083	HPE	HPE ProLiant DL320 Gen11 (4x L4-PCIe-24GB, TensorRT)	1	Intel(R) Xeon(R) Gold 5412U	1	NVIDIA L4-PCIe-24GB	4	TensorRT 9.0.0, CUDA 12.0		47,618.20	50,116.00	799.34	880.28	4.27																details	code
33	3.1-0084	HPE	HPE ProLiant DL380a Gen11 (4x H100-PCIe-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480+	2	NVIDIA H100-PCIe-80GB	4	TensorRT 9.0.0, CUDA 12.2		188,015.00	225,970.00	4,004.73	4,516.89	18.85	18.85			14,007.20	23,441.10	14,007.20	20,329.90									details	code
34	3.1-0085	HPE	HPE ProLiant XL675d Gen10 Plus (8x A100-SXM-80GB, TensorRT)	1	AMD EPYC 7763 64-Core Processor	2	NVIDIA A100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		305,033.00	340,422.00	5,603.34	6,543.46	30.48	30.48			25,406.20	28,464.20	12,824.10	14,688.20									details	code
35	3.1-0086	HPE	HPE ProLiant DL385 Gen10 Plus v2 (8x QAIC100 Standard)	1	AMD EPYC 7543 32-Core Processor	2	QUALCOMM Cloud AI 100 PCIe/HHHL Standard	8	QUALCOMM Cloud AI SDK v1.9.1		156,018.00	159,270.00	2,229.07	2,277.69					5,479.20	5,917.47	2,728.70	2,956.58									details	code	With 75W Accelerator TDP constraints. Powered by the KRAI X and KILT technologies
36	3.1-0088	IEI	NF5468M6 (8x A40, TensorRT)	1	Intel(R) Xeon(R) Platinum 8380	2	NVIDIA A40	8	TensorRT 9.0.0, CUDA 12.2		145,820.00	156,472.00	2,453.00	2,554.78	14.88	15.61	37,912.40	54,707.50	13,155.40	14,022.00	6,004.71	6,863.70									details	code
37	3.1-0090	Intel	1-node-2S-SPR-PyTorch-INT4+INT8	1	Intel(R) Xeon(R) Platinum 8480+	2			PyTorch																		0.95	1.90			details	code	QuantaGrid D54Q-2U. N/A
38	3.1-0091	Intel	1-node-2S-SPR-PyTorch-INT8	1	Intel(R) Xeon(R) Platinum 8480+	2			PyTorch		16,505.90	20,565.50	214.58	284.75	1.72	1.72			1,089.77	1,357.33			4,704.13	5,367.77			0.59	2.05			details	code	QuantaGrid D54Q-2U. N/A
39	3.1-0092	Intel	1-node-2S-SPR-PyTorch-MIX	1	Intel(R) Xeon(R) Platinum 8480+	2			PyTorch								4,204.80	5,782.18													details	code	QuantaGrid D54Q-2U
40	3.1-0093	Intel	1-node-2S-SPRHBM-PyTorch-BF16	1	Intel (R) Xeon (R) CPU Max 9480	2			PyTorch																		0.30	1.03	0.30	1.03	details	code	SC09WPRF0134SR. N/A
41	3.1-0094	Intel-HabanaLabs	HLS-Gaudi2-PT	1	Intel(R) Xeon(R) Platinum 8380	2	Habana Gaudi2	8	PyTorch 2.0.1a0																		78.58	84.08	78.58	84.08	details	code
42	3.1-0095	Krai	Dell Precision 7920 Tower (2x NVIDIA RTX A5000 GPU)	1	Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz	1	NVIDIA RTX A5000 GPU	2	KRAI Inference Library Technology (KILT) with TensorRT support				420.65	442.68					1,851.15	2,493.04	900.56	1,183.87									details	code	Powered by the KRAI X and KILT technologies
43	3.1-0102	Lenovo	Lenovo ThinkSystem SR675 V3 (8x H100-PCIe-80GB, TensorRT)	1	AMD EPYC 9554 64-Core Processor	2	NVIDIA H100-PCIe-80GB	8	TensorRT 9.0.0, CUDA 12.2		376,074.00	450,530.00	8,801.85	9,176.37	37.43	37.43	129,622.00	135,639.00	35,213.10	46,720.20											details	code
44	3.1-0103	Lenovo	Lenovo ThinkSystem SR665v1 (5x QAIC100 Pro)	1	AMD EPYC 75F3 32-Core Processor	2	QUALCOMM Cloud AI 100 PCIe/HHHL Pro	5	QUALCOMM Cloud AI SDK v1.9.1		115,019.00	116,773.00	1,386.48	1,459.08					3,404.83	3,833.70	1,666.36	1,894.73									details	code	With 75W Accelerator TDP constraints. Powered by the KRAI X and KILT technologies
45	3.1-0106	NVIDIA	ASROCKRACK 1U1G-MILAN (1x L4, TensorRT)	1	AMD EPYC 7313P 16-Core Processor	2	NVIDIA L4	1	TensorRT 9.0.0, CUDA 12.2		12,204.40	12,881.70	199.74	225.92	1.07	1.07	3,754.56	3,899.48	898.95	1,028.95	539.24	631.46	3,305.38	3,672.79	3,305.38	3,672.79	0.89	1.30	0.89	1.30	details	code
46	3.1-0107	NVIDIA	NVIDIA DGX H100 (1x H100-SXM-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480C	2	NVIDIA H100-SXM-80GB	1	TensorRT 9.0.0, CUDA 12.2		73,019.20	88,526.00	1,621.29	1,728.03	6.45	6.45	21,510.70	23,306.60	7,003.98	9,102.26	6,104.67	7,877.73	41,516.80	42,856.40	41,516.80	42,856.40	10.15	13.07	10.15	13.07	details	code
47	3.1-0108	NVIDIA	NVIDIA DGX H100 (8x H100-SXM-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480C	2	NVIDIA H100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		584,197.00	704,412.00	12,884.60	14,091.40	51.61	51.61	144,022.00	151,663.00	56,022.10	70,169.70	49,617.50	62,136.70	315,044.00	329,529.00	315,044.00	329,529.00	82.26	106.32	82.26	106.32	details	code
48	3.1-0109	NVIDIA	NVIDIA DGX H100 (8x H100-SXM-80GB, MaxQ, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480C	2	NVIDIA H100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		400,094.00	474,849.00	8,801.85	10,113.70	38.29	38.29	112,015.00	125,479.00	42,416.10	54,050.30	39,214.70	51,006.90	244,023.00	273,527.00	244,023.00	273,527.00	48.98	64.51	48.98	64.51	details	code
49	3.1-0110	NVIDIA	NVIDIA GH200-GraceHopper-Superchip (1x GH200-96GB_aarch64, TensorRT)	1	NVIDIA Grace CPU	1	NVIDIA GH200-GraceHopper-Superchip	1	TensorRT 9.0.0, CUDA 12.2		77,018.20	93,198.30	1,731.49	1,849.39	6.76	6.76	24,008.00	25,974.70	7,704.01	10,163.40	7,003.98	8,645.74	48,516.90	49,001.80	48,516.90	49,001.80	10.96	13.34	10.96	13.34	details	code	NVIDIA MGX Reference Platform
50	3.1-0111	NVIDIA	Gigabyte G482-Z54 (1x H100-PCIe-80GB, TensorRT)	1	AMD EPYC 7742 64-Core Processor	2	NVIDIA H100-PCIe-80GB	1	TensorRT 9.0.0, CUDA 12.2		47,017.50	54,851.40	1,049.36	1,111.11	4.55	4.55	15,006.80	17,106.70	4,564.11	5,711.00	4,004.73	4,961.92	24,507.70	25,153.40	24,507.70	25,153.40					details	code
51	3.1-0112	NVIDIA	Gigabyte G482-Z54 (8x H100-PCIe-80GB, TensorRT)	1	AMD EPYC 7742 64-Core Processor	2	NVIDIA H100-PCIe-80GB	8	TensorRT 9.0.0, CUDA 12.2		368,074.00	442,960.00	8,402.57	9,176.43	37.02	37.02	100,015.00	115,297.00	35,373.40	45,698.50	32,012.90	39,412.80	170,013.00	192,829.00	170,013.00	192,829.00					details	code
52	3.1-0113	NVIDIA	Gigabyte G482-Z54 (8x H100-PCIe-80GB, MaxQ, TensorRT)	1	AMD EPYC 7742 64-Core Processor	2	NVIDIA H100-PCIe-80GB	8	TensorRT 9.0.0, CUDA 12.2		240,024.00	348,572.00	6,303.18	6,719.02	27.36	27.36	88,023.00	98,684.60	33,011.50	39,646.90	28,513.00	34,374.10	132,024.00	162,281.00	132,024.00	162,281.00	40.01	50.57	40.01	50.57	details	code
53	3.1-0118	Nutanix	NX_3155G_G8_A100_PCIe_80GBx2	1	Intel(R) Xeon(R) Gold 6354 CPU @ 3.00GHz	2	NVIDIA A100-PCIe-80GB	2	TensorRT 8.6.0, CUDA 12.0		64,520.40	74,652.60	1,250.31	1,297.28	6.94	6.94	24,008.00	26,250.10	5,603.34	6,241.74	2,803.89	3,275.44									details	code
54	3.1-0119	Oracle	BM.GPU.A10.4	1	Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz	2	NVIDIA A10-PCI-24GB	4	TensorRT 9.0.0, CUDA 12.2				855.00	953.53	5.15		9,202.52	16,989.30													details	code
55	3.1-0120	Oracle	BM.GPU.A100-v2.8	1	AMD EPYC 7J13 64-Core Processor	2	NVIDIA A100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		290,028.00	325,567.00	5,603.34	6,512.98	30.31	30.33	104,012.00	107,408.00	25,406.20	28,028.60	12,824.10	14,534.40	80,018.10	138,331.00	80,018.10	138,179.00	16.92	27.13	17.04	25.29	details	code
56	3.1-0121	Oracle	BM.GPU.H100.8	1	Intel(R) Xeon(R) Platinum 8480+	2	NVIDIA H100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		584,197.00	703,548.00	12,884.60	14,047.20	51.45	51.48			56,022.10	70,689.90	49,617.50	62,285.50	300,033.00	339,265.00	300,033.00	339,050.00	79.90	106.69			details	code
57	3.1-0122	Qualcomm	GIGABYTE G292-Z43 (16x QAIC100 Pro)	1	AMD EPYC 7713 64-Core Processor	2	QUALCOMM Cloud AI 100 PCIe/HHHL Pro	16	QUALCOMM Cloud AI SDK v1.9.1		370,071.00	398,010.00	4,578.88	4,671.42					12,003.70	12,536.80	5,934.41	6,315.87									details	code	With 75W Accelerator TDP constraints. Powered by the KRAI X and KILT technologies
58	3.1-0123	Qualcomm	GIGABYTE G292-Z43 (16x QAIC100 Pro, EE)	1	AMD EPYC 7742 64-Core Processor	2	QUALCOMM Cloud AI 100 PCIe/HHHL Pro	16	QUALCOMM Cloud AI SDK v1.9.1		328,050.00	337,737.00	3,804.75	3,949.99					9,777.31	10,068.20	5,354.05	5,584.38									details	code	With 75W Accelerator TDP constraints. Powered by the KRAI X and KILT technologies
59	3.1-0124	Qualcomm	GIGABYTE R282-Z93 (8x QAIC100 Pro, EE)	1	AMD EPYC 7282 16-Core Processor	2	QUALCOMM Cloud AI 100 PCIe/HHHL Pro	8	QUALCOMM Cloud AI SDK v1.9.1		148,018.00	169,969.00	1,901.51	1,975.93					4,804.25	5,031.42	2,778.99	2,915.72									details	code	With 75W Accelerator TDP constraints. 3x QAIC100 on riser CRS2033; 3x QAIC100 on riser CRS2033; 2x QAIC100 on riser CRS2026. Powered by the KRAI X and KILT technologies
60	3.1-0128	Quanta_Cloud_Technology	1-node-2S-SPR-PyTorch-INT8	1	Intel(R) Xeon(R) Platinum 8480+ 56-Core Processor	2			PyTorch		16,505.90	20,282.00	211.65	288.02			4,104.84	5,643.03	1,079.75	1,354.06			4,504.80	4,767.43			0.59	2.07			details	code	QuantaGrid D54Q-2U
61	3.1-0129	Quanta_Cloud_Technology	D54Q_2U (2x H100-PCIe-80GB, TensorRT)	1	Intel(R) Xeon(R) Gold 6430 32-Core Processor	2	NVIDIA H100-PCIe-80GB	2	TensorRT 9.0.0, CUDA 12.2		94,018.40	112,561.00	2,001.26	2,265.11	9.34	9.34	30,013.00	36,083.80	9,002.36	11,251.90	8,003.55	9,491.09									details	code
62	3.1-0130	Quanta_Cloud_Technology	D54Q_2U (4x L4-PCIe-24GB, TensorRT)	1	Intel(R) Xeon(R) Gold 6430 32-Core Processor	2	NVIDIA L4-PCIe-24GB	4	TensorRT 9.0.0, CUDA 12.2		48,818.30	51,005.20	799.34	889.04	4.32	4.32	15,006.80	15,709.20	3,604.96	3,733.81	2,162.57	2,554.79					3.44	5.21			details	code
63	3.1-0132	Supermicro	AS-8125GS-TNHR (8x H100-SXM-80GB, TensorRT)	1	AMD EPYC 9554	2	NVIDIA H100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		590,083.00	707,537.00	12,996.70	14,136.20	51.85	51.85	172,014.00	176,599.00	56,893.70	70,619.70	50,617.40	62,456.10	322,447.00	342,065.00	325,049.00	341,806.00	84.50	105.53	84.50	105.91	details	code
64	3.1-0133	Supermicro	SYS-421GU-TNXR (4x H100-SXM-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8470Q	2	NVIDIA H100-SXM-80GB	4	TensorRT 9.0.0, CUDA 12.2																		40.01	37.93	40.01	37.93	details	code
65	3.1-0134	Supermicro	SYS-521GE-TNRT (8xH100-PCIe-80GB)	1	Intel(R) Xeon(R) Platinum 8462Y+	2	NVIDIA H100-PCIe-80GB	8	TensorRT 9.0.0, CUDA 12.2		368,074.00	446,436.00	8,402.57	9,170.89	37.64	37.64	100,015.00	131,664.00	35,012.80	46,250.70	30,512.70	40,132.90	170,013.00	198,707.00	170,013.00	198,707.00					details	code
66	3.1-0135	Supermicro	SYS-821GE-TNHR (8x H100-SXM-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8468	2	NVIDIA H100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		593,579.00	705,305.00	13,020.70	14,195.80	51.99	51.98			57,101.70	70,682.60	50,969.10	62,479.80	327,051.00	340,928.00	324,250.00	340,658.00	85.57	107.33	85.43	107.06	details	code
67	3.1-0136	TTA	KR580S1	1	Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz	2	NVIDIA T4	2	TensorRT 8.6.0, CUDA 12.0		8,202.98	10,300.70																			details	code
68	3.1-0137	TTA	KR580S1	1	Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz	2	NVIDIA T4	2	tensorrt 8.6.1,		8,701.87	10,639.80	130.09	160.77	0.87				539.24	742.60	249.41	359.26									details	code	Powered by MLCommons CM automation language and CK playground
69	3.1-0138	xFusion	xFusion FusionServer G5500V7(10x NVIDIA A30, TensorRT)	1	Intel(R) Xeon(R) Gold 6458Q @ 3.1 GHz	2	NVIDIA A30	10	TensorRT 9.0.0, CUDA 12.2		184,417.00	194,705.00	3,864.53	3,897.79	17.92	17.92	58,821.10	72,693.30	15,407.70	17,263.50	6,723.79	8,687.66	53,519.80	68,727.90	53,519.80	68,727.90					details	code
70	3.1-0139	xFusion	xFusion FusionServer G5500V7(8x NVIDIA A30, TensorRT)	1	Intel(R) Xeon(R) Gold 6458Q @ 3.1 GHz	2	NVIDIA A30	8	TensorRT 9.0.0, CUDA 12.2		147,668.00	155,998.00	3,100.41	3,047.61	14.32	14.32	46,517.00	58,061.90	12,304.90	13,609.60	5,504.20	6,954.59	42,516.50	55,253.20	42,516.50	55,253.20	8.93	9.83	8.93	9.82	details	code
71	3.1-0140	xFusion	xFusion FusionServer G5500V7(10x NVIDIA L40, TensorRT)	1	Intel(R) Xeon(R) Gold 6458Q @ 3.1 GHz	2	NVIDIA L40	10	TensorRT 9.0.0, CUDA 12.2		295,531.00	258,506.00	5,264.69	4,991.75	28.65	28.87	94,219.30	97,499.00	20,508.90	17,996.30	8,902.43	8,527.61	66,022.20	82,179.50	66,022.20	82,179.50					details	code
72	3.1-0141	xFusion	xFusion FusionServer G5500V7(8x NVIDIA L40, TensorRT)	1	Intel(R) Xeon(R) Gold 6458Q @ 3.1 GHz	2	NVIDIA L40	8	TensorRT 9.0.0, CUDA 12.2		234,743.00	206,778.00	4,204.80	3,981.97	23.07	23.07	75,540.40	83,390.70	16,726.80	14,439.70	9,701.77	8,797.10	64,021.20	67,717.70	64,021.20	67,717.70	30.15	40.19	30.15	40.19	details	code
73	3.1-0142	xFusion	xFusion FusionServer 2288H V7(6x NVIDIA L4, TensorRT)	1	Intel(R) Xeon(R) Platinum 8458P CPU @ 2.7 GHz	2	NVIDIA L4	6	TensorRT 9.0.0, CUDA 12.2		74,520.70	76,663.90	1,280.38	1,227.30	6.82	6.82	14,807.60	24,375.60	5,404.18	5,748.72	3,754.56	3,942.95	8,003.55	8,914.83	8,003.55	8,903.54	6.95	8.83	6.95	8.85	details	code
74	Preview
75	3.1-0143	Google	tpu-v5e-4	1	AMD EPYC 7B13	1	TPU v5e	4	SAX																		7.13	9.81			details	code
76	3.1-0144	Quanta_Cloud_Technology	D54U-3U (4x H100-PCIe-80GB, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480+	2	NVIDIA H100-PCIe-80GB	4	TensorRT 9.0.0, CUDA 12.2		188,015.00	221,741.00	4,004.73	4,503.27	18.31	18.31	60,023.10	70,529.20	18,006.50	22,801.60	16,006.70	19,285.10									details	code


1	ID	Submitter	System	Nodes	Processor	p#	Accelerator	a#	Software	Results																																									Details	Code	Notes
2										Task	Image classification				Object detection				Medical imaging				Speech-to-text				Natural Language Processing								Recommendation								Large Language Model
3										Data	ImageNet				OpenImages (800x800)				KiTS19				LibriSpeech				SQuAD v1.1								Criteo 4TB								CNN-DailyMail News
4										Model	ResNet				Retinanet				3D-UNet				RNN-T				BERT								dlrm-v2-99								gptj-99
5										Accuracy (%FP32 ref)	99.00				99.00				99.00		99.90		99.00				99.00				99.90				99.00				99.90				gptj-99				gptj-99.9
6										Scenario	Server		Offline		Server		Offline		Offline		Offline		Server		Offline		Server		Offline		Server		Offline		Server		Offline		Server		Offline		Server		Offline		Server		Offline
7										Units	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	samples/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)
8	Available
9	3.1-0109	NVIDIA	NVIDIA DGX H100 (8x H100-SXM-80GB, MaxQ, TensorRT)	1	Intel(R) Xeon(R) Platinum 8480C	2	NVIDIA H100-SXM-80GB	8	TensorRT 9.0.0, CUDA 12.2		400,094.00	4,135.72	474,849.00	4,063.98	8,801.85	4,545.92	10,113.70	4,554.48	38.29	4,166.64	38.29	4,166.64	112,015.00	4,400.42	125,479.00	4,199.93	42,416.10	5,223.90	54,050.30	5,038.23	39,214.70	5,528.30	51,006.90	5,594.39	244,023.00	5,794.91	273,527.00	5,629.87	244,023.00	5,794.91	273,527.00	5,629.87	48.98	3,830.87	64.51	3,805.44	48.98	3,830.87	64.51	3,805.44	details	code
10	3.1-0113	NVIDIA	Gigabyte G482-Z54 (8x H100-PCIe-80GB, MaxQ, TensorRT)	1	AMD EPYC 7742 64-Core Processor	2	NVIDIA H100-PCIe-80GB	8	TensorRT 9.0.0, CUDA 12.2		240,024.00	2,272.53	348,572.00	2,268.11	6,303.18	2,347.19	6,719.02	2,254.60	27.36	2,144.75	27.36	2,144.75	88,023.00	2,248.54	98,684.60	2,235.99	33,011.50	3,348.38	39,646.90	3,047.66	28,513.00	3,116.67	34,374.10	3,310.20	132,024.00	3,049.98	162,281.00	2,955.39	132,024.00	3,049.98	162,281.00	2,955.39	40.01	2,187.31	50.57	2,195.34	40.01	2,187.31	50.57	2,195.34	details	code
11	3.1-0123	Qualcomm	GIGABYTE G292-Z43 (16x QAIC100 Pro, EE)	1	AMD EPYC 7742 64-Core Processor	2	QUALCOMM Cloud AI 100 PCIe/HHHL Pro	16	QUALCOMM Cloud AI SDK v1.9.1		328,050.00	1,417.29	337,737.00	1,425.32	3,804.75	980.13	3,949.99	982.86									9,777.31	1,091.04	10,068.20	1,098.44	5,354.05	1,163.16	5,584.38	1,191.47																	details	code	With 75W Accelerator TDP constraints. Powered by the KRAI X and KILT technologies
12	3.1-0124	Qualcomm	GIGABYTE R282-Z93 (8x QAIC100 Pro, EE)	1	AMD EPYC 7282 16-Core Processor	2	QUALCOMM Cloud AI 100 PCIe/HHHL Pro	8	QUALCOMM Cloud AI SDK v1.9.1		148,018.00	631.21	169,969.00	686.11	1,901.51	455.41	1,975.93	469.22									4,804.25	529.57	5,031.42	534.62	2,778.99	599.40	2,915.72	617.11																	details	code	With 75W Accelerator TDP constraints. 3x QAIC100 on riser CRS2033; 3x QAIC100 on riser CRS2033; 2x QAIC100 on riser CRS2026. Powered by the KRAI X and KILT technologies


1	ID	Submitter	System	Nodes	Processor	p#	Accelerator	a#	Software	UsedModel	Accuracy	Result																Details	Code	Notes
2												Task	Image classification		Object detection		Medical imaging		Speech-to-text		Natural Language Processing				Recommendation		Large Language Model
3												Data	ImageNet		OpenImages (800x800)		KiTS19		LibriSpeech		SQuAD v1.1				Criteo 4TB		CNN-DailyMail News
4												Model	ResNet		Retinanet		3D-UNet		RNN-T		BERT				dlrm-v2-99		gptj-99
5												Accuracy	99.00		99.00		99.00	99.90	99.00		99.00		99.90		99.00		99.00
6												Scenario	Server	Offline	Server	Offline	Offline	Offline	Server	Offline	Server	Offline	Server	Offline	Server	Offline	Offline
7												Units	Queries/s	Samples/s	Queries/s	Samples/s	Samples/s	Samples/s	Queries/s	Samples/s	Queries/s	Samples/s	Queries/s	Samples/s	Queries/s	Samples/s	Samples/s
8	Available
9	3.1-0148	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	3d-unet-99	0.86						4.18											details	code	Powered by MLCommons CM automation language and CK playground.
10	3.1-0149	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	3d-unet-99.9	0.86							4.18										details	code	Powered by MLCommons CM automation language and CK playground.
11	3.1-0150	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	bert-99	90.36											4144.88						details	code	Powered by MLCommons CM automation language and CK playground.
12	3.1-0150	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1		bert-99	90.51										3954.44							details	code
13	3.1-0151	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	bert-99.9	90.87												1521.60					details	code	Powered by MLCommons CM automation language and CK playground.
14	3.1-0151	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1		bert-99.9	90.88													1682.05				details	code
15	3.1-0152	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	resnet50	76.16		37013.80	45772.10														details	code	Powered by MLCommons CM automation language and CK playground.
16	3.1-0153	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	retinanet	37.39					615.99												details	code	Powered by MLCommons CM automation language and CK playground.
17	3.1-0153	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1		retinanet	37.41				589.09													details	code
18	3.1-0154	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	rnnt	92.55								14157.00	15377.20								details	code	Powered by MLCommons CM automation language and CK playground.
19	3.1-4184	Dell	Dell PowerEdge Server R760 (Intel Xeon Platinum 8480+)	1	Intel(R) Xeon(R) Platinum 8480+	2			PyTorch	dlrm-v2-99	80.23														4404.50	5016.63		details	code
20	3.1-4185	Intel	1-node-2S-SPR-PyTorch-INT8	1	Intel(R) Xeon(R) Platinum 8480+	2			PyTorch	bert-99.9_MiniLM	90.80													6543.62				details	code	QuantaGrid D54Q-2U. N/A
21	3.1-4187	Moffett	H3C R5300 G5 (1x SparseOne S30, PCIe/FHFL, Moffett-SDK)	1	Intel(R) Xeon(R) Gold 6348	2	MOFFETT S30-PCIe/FHFL-60GB	1	Moffett SDK	gptj-99	42.96																23.28	details	code
22	3.1-4188	Moffett	H3C R5300 G5 (4x SparseOne S30, PCIe/FHFL, Moffett-SDK)	1	Intel(R) Xeon(R) Gold 6348	2	MOFFETT S30-PCIe/FHFL-60GB	4	Moffett SDK	gptj-99	42.96																91.57	details	code
23	3.1-4189	Moffett	Inspur NF5468M6 (8x SparseOne S30, PCIe/FHFL, Moffett-SDK)	1	Intel(R) Xeon(R) Platinum 8380 CPU @ 2.30GHz	2	MOFFETT S30-PCIe/FHFL-60GB	8	Moffett SDK	gptj-99	42.96																170.59	details	code
24	3.1-4190	NeuralMagic	aws.c6g_2xlarge	1	ARM Neoverse-N1	1			deepsparse v1.6.0.20230801	mobilebert-14layer_pruned50-none-vnni-bert-99	90.43											32.98						details	code	Powered by MLCommons Collective Mind framework (CK2).
25	3.1-4191	NeuralMagic	aws.c6g_2xlarge	1	ARM Neoverse-N1	1			deepsparse v1.6.0.20230801	mobilebert-14layer_pruned50_quant-none-vnni-bert-99	90.35											72.32						details	code	Powered by MLCommons Collective Mind framework (CK2).
26	3.1-4192	NeuralMagic	aws.c6g_2xlarge	1	ARM Neoverse-N1	1			deepsparse v1.6.0.20230801	mobilebert-base_quant-none-bert-99	90.78											39.83						details	code	Powered by MLCommons Collective Mind framework (CK2).
27	3.1-4193	NeuralMagic	aws.c6g_2xlarge	1	ARM Neoverse-N1	1			deepsparse v1.6.0.20230801	mobilebert-none-base-none-bert-99	90.89											18.86						details	code	Powered by MLCommons Collective Mind framework (CK2).
28	3.1-4194	NeuralMagic	aws.c6g_2xlarge	1	ARM Neoverse-N1	1			deepsparse v1.6.0.20230801	obert-large-pruned97-quant-none-bert-99	90.09											9.56						details	code	Powered by MLCommons Collective Mind framework (CK2).
29	3.1-4195	NeuralMagic	aws.c7g_2xlarge	1	ARM Neoverse-V1	1			deepsparse v1.6.0.20230801	bert-large-pruned80_quant-none-vnni-bert-99	90.23											10.54						details	code	Powered by MLCommons Collective Mind framework (CK2).
30	3.1-4196	NeuralMagic	aws.c7g_2xlarge	1	ARM Neoverse-V1	1			deepsparse v1.6.0.20230801	mobilebert-14layer_pruned50-none-vnni-bert-99	90.43											49.28						details	code	Powered by MLCommons Collective Mind framework (CK2).
31	3.1-4197	NeuralMagic	aws.c7g_2xlarge	1	ARM Neoverse-V1	1			deepsparse v1.6.0.20230801	mobilebert-14layer_pruned50_quant-none-vnni-bert-99	90.35											107.21						details	code	Powered by MLCommons Collective Mind framework (CK2).
32	3.1-4198	NeuralMagic	aws.c7g_2xlarge	1	ARM Neoverse-V1	1			deepsparse v1.6.0.20230801	mobilebert-base_quant-none-bert-99	90.78											56.11						details	code	Powered by MLCommons Collective Mind framework (CK2).
33	3.1-4199	NeuralMagic	aws.c7g_2xlarge	1	ARM Neoverse-V1	1			deepsparse v1.6.0.20230801	mobilebert-none-base-none-bert-99	90.89											21.50						details	code	Powered by MLCommons Collective Mind framework (CK2).
34	3.1-4200	NeuralMagic	aws.c7g_2xlarge	1	ARM Neoverse-V1	1			deepsparse v1.6.0.20230801	obert-large-pruned97-quant-none-bert-99	90.09											15.91						details	code	Powered by MLCommons Collective Mind framework (CK2).
35	3.1-4201	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	bert-base-pruned90-none-bert-99	88.42											29.85						details	code	Powered by MLCommons Collective Mind framework (CK2).
36	3.1-4202	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	bert-base-pruned95_obs_quant-none-bert-99	87.89											77.47						details	code	Powered by MLCommons Collective Mind framework (CK2).
37	3.1-4203	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	bert-base_cased-pruned90-none-bert-99	4.53											28.84						details	code	Powered by MLCommons Collective Mind framework (CK2).
38	3.1-4204	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	bert-large-base-none-bert-99	89.65											3.08						details	code	Powered by MLCommons Collective Mind framework (CK2).
39	3.1-4205	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	bert-large-pruned80_quant-none-vnni-bert-99	90.27											20.13						details	code	Powered by MLCommons Collective Mind framework (CK2).
40	3.1-4206	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	mobilebert-14layer_pruned50-none-vnni-bert-99	90.43											77.57						details	code	Powered by MLCommons Collective Mind framework (CK2).
41	3.1-4207	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	mobilebert-14layer_pruned50_quant-none-vnni-bert-99	90.40											158.57						details	code	Powered by MLCommons Collective Mind framework (CK2).
42	3.1-4208	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	mobilebert-base_quant-none-bert-99	90.79											88.67						details	code	Powered by MLCommons Collective Mind framework (CK2).
43	3.1-4209	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	mobilebert-none-base-none-bert-99	90.89											38.77						details	code	Powered by MLCommons Collective Mind framework (CK2).
44	3.1-4210	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	obert-base-pruned90-none-bert-99	88.31											29.66						details	code	Powered by MLCommons Collective Mind framework (CK2).
45	3.1-4211	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	obert-large-base-none-bert-99	89.65											3.08						details	code	Powered by MLCommons Collective Mind framework (CK2).
46	3.1-4212	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	obert-large-pruned95-none-vnni-bert-99	90.18											13.08						details	code	Powered by MLCommons Collective Mind framework (CK2).
47	3.1-4213	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	obert-large-pruned95_quant-none-vnni-bert-99	90.03											30.74						details	code	Powered by MLCommons Collective Mind framework (CK2).
48	3.1-4214	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	obert-large-pruned97-none-bert-99	90.14											14.97						details	code	Powered by MLCommons Collective Mind framework (CK2).
49	3.1-4215	NeuralMagic	gcp.c3_standard_8	1	Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz	1			deepsparse v1.6.0.20230801	obert-large-pruned97-quant-none-bert-99	90.18											27.09						details	code	Powered by MLCommons Collective Mind framework (CK2).
50	3.1-4240	Supermicro	1-node-4S-SPR-PyTorch-INT8	1	Intel(R) Xeon(R) Platinum 8480+	4			PyTorch	gptj-99	42.92																2.81	details	code
51	RDI
52	3.1-4242	NVIDIA	NVIDIA L4 (1x L4, TensorRT)	1	AMD EPYC 7313P 16-Core Processor	2	NVIDIA L4	1	TensorRT 9.0.0, CUDA 12.2	bert-99	90.17										4,264.85							details	code
53	3.1-4242	NVIDIA	NVIDIA L4 (1x L4, TensorRT)	1	AMD EPYC 7313P 16-Core Processor	2	NVIDIA L4	1	TensorRT 9.0.0, CUDA 12.2	bert-99	90.18											4,609.04						details	code
54
55
56
57
58
59
60
61
62


1	ID	Submitter	System	Nodes	Processor	p#	Accelerator	a#	Software	UsedModel	Accuracy		Result																								Details	Code	Notes
2												Task	Image classification				Object detection				Medical imaging				Speech-to-text				Natural Language Processing
3												Data	ImageNet				OpenImages (800x800)				KiTS19				LibriSpeech				SQuAD v1.1
4												Model	ResNet				Retinanet				3D-UNet				RNN-T				BERT
5												Accuracy (%FP32 ref)	99.00				99.00				99.00		99.90		99.00				99.00				99.90
6												Scenario	Server		Offline		Server		Offline		Offline		Offline		Server		Offline		Server		Offline		Server		Offline
7												Units	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	samples/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)	queries/s	System Power (W)	samples/s	System Power (W)
8	Available
9	3.1-0148	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	3d-unet-99	0.86										4.18	601.89															details	code	Powered by MLCommons CM automation language and CK playground.
10	3.1-0149	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	3d-unet-99.9	0.86												4.18	601.89													details	code	Powered by MLCommons CM automation language and CK playground.
11	3.1-0150	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	bert-99	90.36																				4144.88	625.26					details	code	Powered by MLCommons CM automation language and CK playground.
12	3.1-0150	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1		bert-99	90.51																		3954.44	621.15							details	code
13	3.1-0151	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	bert-99.9	90.87																						1521.60	620.65			details	code	Powered by MLCommons CM automation language and CK playground.
14	3.1-0151	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1		bert-99.9	90.88																								1682.05	612.72	details	code
15	3.1-0152	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	resnet50	76.16		37013.80	581.93	45772.10	617.54																					details	code	Powered by MLCommons CM automation language and CK playground.
16	3.1-0153	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	retinanet	37.39								615.99	582.65																	details	code	Powered by MLCommons CM automation language and CK playground.
17	3.1-0153	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1		retinanet	37.41						589.09	566.96																			details	code
18	3.1-0154	CTuning	PCSPECIALIST AMD AM5 PC with Nvidia RTX 4090	1	AMD Ryzen 9 7950X 16-Core Processor	1	NVIDIA GeForce RTX 4090 (Ada Lovelace)	1	Nvidia inference implementation with CM API, TensorRT v8.6.1.6	rnnt	92.55														14157.00	621.62	15377.20	615.30									details	code	Powered by MLCommons CM automation language and CK playground.