Stardog Benchmarks - Google Drive


1	Clients	1	4.00000	8	64
2	QMPH	41,237.20	57,416.37	101,740.45	145,048.83
3
4	1 CLIENT
5	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
6	q1	8.032	0.00394		Seed	9834533
7	q2	19.757	0.00342		Warmups	2000
8	q3	5.356	0.00553		Runs	500
9	q4	7.806	0.00639		Drill	ON
10	q5	4.996	0.16652		Query Mix Size	25
11	q7	11.597	0.00551		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
12	q8	4.921	0.00362
13	q9	0.000	0.00183
14	q10	1.870	0.00400
15	q11	10.000	0.00163
16	q12	0.000	0.00357
17
18	QMpH	41,237.20	query mixes per hour
19	CQET	0.084	seconds average runtime of query mix
20
21	4 CLIENTS
22	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
23	q1	8.750	0.00675		Seed	8188326
24	q2	18.083	0.00429		Warmups	8
25	q3	3.250	0.00600		Runs	4
26	q4	9.500	0.01150		Drill	ON
27	q5	5.000	0.25500		Query Mix Size	25
28	q7	8.688	0.00675		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
29	q8	4.750	0.00513
30	q9	0.000	0.00275
31	q10	1.625	0.00500
32	q11	10.000	0.00325
33	q12	0.000	0.00500
34
35	QMpH	57,416.37	query mixes per hour
36	CQET	0.209	seconds average runtime of query mix
37
38	8 CLIENTS
39	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
40	q1	7.500	0.00475		Seed	9175932
41	q2	19.125	0.00544		Warmups	16
42	q3	4.000	0.00650		Runs	8
43	q4	7.500	0.00775		Drill	ON
44	q5	5.000	0.20425		Query Mix Size	25
45	q7	12.250	0.00813		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
46	q8	4.188	0.00450
47	q9	0.000	0.01194
48	q10	1.750	0.00513
49	q11	10.000	0.00275
50	q12	0.000	0.01875
51
52	QMpH	101,740.45	query mixes per hour
53	CQET	0.194	seconds average runtime of query mix
54
55	64 CLIENTS
56	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
57	q1	7.375	0.10566		Seed	4187411
58	q2	19.690	0.10144		Warmups	128
59	q3	5.266	0.05956		Runs	64
60	q4	7.531	0.09661		Drill	ON
61	q5	5.000	0.72231		Query Mix Size	25
62	q7	11.293	0.09790		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
63	q8	4.961	0.04456
64	q9	0.000	0.09959
65	q10	1.648	0.13488
66	q11	10.000	0.09180
67	q12	0.000	0.16175
68
69	QMpH	145,048.83	query mixes per hour
70	CQET	1.239	seconds average runtime of query mix


1	Clients	1	4	8	64
2	QMPH	5,439.47	6,341.26	11,866.85	18,118.07
3
4	1 CLIENT
5	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
6	q1	9.432	0.00853		Seed	9834533
7	q2	19.994	0.00379		Warmups	2000
8	q3	7.386	0.01192		Runs	500
9	q4	9.662	0.01676		Drill	ON
10	q5	5.000	1.54240		Query Mix Size	25
11	q7	12.193	0.01798		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
12	q8	4.954	0.00946
13	q9	0.000	0.00215
14	q10	1.922	0.01306
15	q11	10.000	0.00195
16	q12	0.000	0.00389
17
18	QMpH	5,439.47	query mixes per hour
19	CQET	0.655	seconds average runtime of query mix
20
21	4 CLIENTS
22	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
23	q1	10.000	0.01350		Seed	8188326
24	q2	20.125	0.00492		Warmups	8
25	q3	3.000	0.01000		Runs	4
26	q4	10.000	0.01525		Drill	ON
27	q5	5.000	1.91363		Query Mix Size	25
28	q7	16.813	0.03375		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
29	q8	6.000	0.01288
30	q9	0.000	0.00269
31	q10	2.125	0.01675
32	q11	10.000	0.00250
33	q12	0.000	0.00425
34
35	QMpH	6,341.26	query mixes per hour
36	CQET	1.369	seconds average runtime of query mix
37
38	8 CLIENTS
39	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
40	q1	9.250	0.01338		Seed	9175932
41	q2	18.792	0.05331		Warmups	16
42	q3	7.000	0.01325		Runs	8
43	q4	9.500	0.01938		Drill	ON
44	q5	5.000	2.63988		Query Mix Size	25
45	q7	9.750	0.05044		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
46	q8	5.500	0.09775
47	q9	0.000	0.00303
48	q10	2.000	0.01944
49	q11	10.000	0.00313
50	q12	0.000	0.00463
51
52	QMpH	11,866.85	query mixes per hour
53	CQET	2.034	seconds average runtime of query mix
54
55	64 CLIENTS
56	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
57	q1	9.531	1.12803		Seed	4187411
58	q2	19.763	0.76948		Warmups	128
59	q3	7.047	0.79533		Runs	64
60	q4	9.688	0.41338		Drill	ON
61	q5	5.000	5.44763		Query Mix Size	25
62	q7	11.020	0.83338		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
63	q8	4.961	1.08064
64	q9	0.000	0.83600
65	q10	2.164	0.52151
66	q11	10.000	0.44827
67	q12	0.000	0.87933
68
69	QMpH	18,118.07	query mixes per hour
70	CQET	9.686	seconds average runtime of query mix


1	Clients	1	4	8	64
2	QMPH	239.00	828.26	1,103.86	1,765.49
3
4	1 CLIENT
5	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
6	q1	10.000	0.18504		Seed	9834533
7	q2	17.892	0.03718		Warmups	2000
8	q3	9.030	0.34582		Runs	500
9	q4	10.000	0.30748		Drill	ON
10	q5	5.000	11.45206		Query Mix Size	25
11	q7	11.547	0.86723		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
12	q8	5.020	0.46963
13	q9	0.000	0.00578
14	q10	1.935	0.62723
15	q11	10.000	0.00586
16	q12	0.000	0.00871
17
18	QMpH	239.00	query mixes per hour
19	CQET	14.833	seconds average runtime of query mix
20
21	4 CLIENTS
22	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
23	q1	10.000	0.17650		Seed	8188326
24	q2	17.625	0.04142		Warmups	8
25	q3	9.250	0.23250		Runs	4
26	q4	10.000	0.46700		Drill	ON
27	q5	5.000	15.08988		Query Mix Size	25
28	q7	9.875	0.93419		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
29	q8	7.750	0.72650
30	q9	0.000	0.00588
31	q10	2.375	0.75538
32	q11	10.000	0.00600
33	q12	0.000	0.00900
34
35	QMpH	828.26	query mixes per hour
36	CQET	12.681	seconds average runtime of query mix
37
38	8 CLIENTS
39	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
40	q1	10.000	0.29675		Seed	9175932
41	q2	18.083	0.04327		Warmups	16
42	q3	10.000	2.34113		Runs	8
43	q4	10.000	0.34750		Drill	ON
44	q5	5.000	17.43744		Query Mix Size	25
45	q7	11.563	1.62441		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
46	q8	4.438	0.45294
47	q9	0.000	0.00600
48	q10	2.313	0.89813
49	q11	10.000	0.00575
50	q12	0.000	0.09800
51
52	QMpH	1,103.86	query mixes per hour
53	CQET	15.816	seconds average runtime of query mix
54
55	64 CLIENTS
56	Stardog	Avg Result Count	Avg Exec Time (s)		Test Parameters
57	q1	10.000	6.80808		Seed	4187411
58	q2	18.083	9.34922		Warmups	128
59	q3	8.875	7.26044		Runs	64
60	q4	10.000	8.36434		Drill	ON
61	q5	5.000	61.53376		Query Mix Size	25
62	q7	11.172	9.15402		Query Mix	1 2 2 3 2 2 4 2 2 5 7 7 5 7 7 8 9 9 8 9 9 10 10 11 12
63	q8	5.008	8.72555
64	q9	0.000	8.17165
65	q10	2.047	8.98420
66	q11	10.000	8.70198
67	q12	0.000	10.87286
68
69	QMpH	1,765.49	query mixes per hour
70	CQET	108.631	seconds average runtime of query mix


1	SP2B 250k	Avg Result Count	Avg Exec Time (ms)	Test Parameters
2	q1	1	2.0	Warmups	10
3	q2	6,197	71.2	Runs	10
4	q3a	15,853	28.0	Query Mix Size	16
5	q3b	127	6.4	Query Mix	q1 q2 q3a q3b q3c q4 q5b q6 q7 q8 q9 q10 q11 q12a q12b q12c
6	q3c	0	6.0
7	q4	542,801	1457.4
8	q5b	6,904	63.0
9	q6	12,093	97.8
10	q7	62	103.8
11	q8	332	104.0
12	q9	4	67.0
13	q10	452	2.8
14	q11	10	49.6
15	q12a	1	13.8
16	q12b	1	101.6
17	q12c	0	2.2
18
19	Triples	250,128	triples
20	QMpH	1313.45	query mixes per hour
21	CQET	2.74	seconds average runtime of query mix
22
23	SP2B 5M	Avg Result Count	Avg Exec Time (ms)	Test Parameters
24	q1	1	2.8	Warmups	10
25	q2	248,738	2335.4	Runs	10
26	q3a	192,373	353.4	Query Mix Size	16
27	q3b	1,317	8.2	Query Mix	q1 q2 q3a q3b q3c q4 q5b q6 q7 q8 q9 q10 q11 q12a q12b q12c
28	q3c	0	10.0
29	q4	18,362,955	107750.8
30	q5b	210,662	2081.0
31	q6	417,625	3242.2
32	q7	1,200	2104.2
33	q8	493	299.0
34	q9	4	2399.4
35	q10	656	3.0
36	q11	10	1558.0
37	q12a	1	412.0
38	q12b	1	283.2
39	q12c	0	3.4
40
41	Triples	5,000,632	triples
42	QMpH	29.9	query mixes per hour
43	CQET	120.4	seconds average runtime of query mix


1	LUBM100	Avg Result Count	Avg Exec Time (ms)	Test Parameters
2	q1	4	10.2	Reasoning	SL
3	q2	264	1404.6	Warmups	10
4	q3	6	3.6	Runs	10
5	q4	34	109.0	Query Mix Size	14
6	q5	719	10.4	Query Mix	q1 q2 q3 q4 q5 q6 q7 q8 q9 q10 q11 q12 q13 q14
7	q6	1,048,532	1728.4
8	q7	67	10.0
9	q8	7,790	576.8
10	q9	27,247	856.4
11	q10	4	8.2
12	q11	224	13.6
13	q12	15	55.0
14	q13	472	10.2
15	q14	795,970	860.2
16
17	Triples	13,405,722	triples
18	QMpH	489.53	query mixes per hour
19	CQET	7.35	seconds avg runtime of query mix
20
21	LUBM1000	Avg Result Count	Avg Exec Time (ms)	Test Parameters
22	q1	4	13.6	Reasoning	SL
23	q2	2,528	10,424.80	Warmups	10
24	q3	6	5.80	Runs	10
25	q4	34	136.6	Query Mix Size	14
26	q5	719	14.8	Query Mix	q1 q2 q3 q4 q5 q6 q7 q8 q9 q10 q11 q12 q13 q14
27	q6	10,447,381	23,885.00
28	q7	67	13.80
29	q8	7,790	1,282.40
30	q9	272,982	8,691.80
31	q10	4	10.60
32	q11	224	8.2
33	q12	15	84.60
34	q13	4,760	19.20
35	q14	7,924,765	13,215.80
36
37	Triples	133,574,293	triples
38	QMpH	59.41	query mixes per hour
39	CQET	60.6	seconds average runtime of query mix


1	Dataset	Size	Number of Files	Format	Indexing Strategy	Total Time (h:mm:ss)	Speed (K tps)
2	bsbm100m	100.1M	10	Turtle	QUADS	0:03:39	454.8
3	bsbm1b	999.7M	100	Turtle	TRIPLES	0:32:38	510.5
4	bsbm10b	9967.6M	100	Turtle	TRIPLES	8:43:19	317.4
5	lubm(10k)	1243.7M	10	RDF/XML	QUADS	0:49:08	421.8
6	lubm(60k)	8290.3M	60	RDF/XML	TRIPLES	5:59:23	384.5
7	sp2b	100.0M	1	Turtle	QUADS	0:11:32	144.5


1	Term	Description
2	Query Mix	List of queries executed at each iteration. Note that, in some benchmarks the same query is repeated multiple times in a query mix. In some benchmarks, most notably BSBM, queries are parameterized so each run of the query uses different constants.
3	Queries Mixes per Hour (QMpH)	Number of query mixes with different parameters that are executed per hour
4	Composite Query Execution Time (cQET)	Average time for executing the query mix multiple times (with different parameters if the benchmark queries are parametrized)


1	Software Specs
2	Stardog 4.0
3	Oracle JDK 1.8.0_45-b14
4	Ubuntu 14.04.2
5
6	Hardware Specs
7	4 eight-core AMD Opteron 6328 3.2GHz 24MB Cache
8	256GB 1600Mhz DDR3 ECC RAM
9	2 x 4TB SATA 6.0Gb/s 7200RPM HD
10	4 x 900GB SAS 2.0 6.0Gb/s 10000RPM HD
11	LSI MegaRAID 9271-8i SAS 6Gb/s controller with 1GB cache