Loadgentest 2

	A	B	C	D	E	F	G	H	I	J	K	L	M
1				Load generator stats				Target system stats				Notes
2	Tool	Params/config	RPS	Avg CPU	Max CPU	Avg Mem	Max Mem	Avg CPU	Max CPU	Avg Mem	Max Mem
3
4	wrk 4.1.0-4	-d 30s -t4 -c48 --latency 'http://192.168.0.121:8080'	52200	334%	358%	4350K	4368K	524%	582%	11428K	11432K
5			53700	326%	360%	4756K	5288K	546%	606%	11432K	11432K
6			54100	330%	352%	4740K	5216K	541%	595%	11432K	11432K
7
8	apachebench 2.3	-c 32 -t 30 -k -n 750000 'http://192.168.0.121:8080/'	16300	97%	100%	15475K	36240K	109%	231%	10220K	11484K
9			15000	98%	100%	14395K	34200K	98%	181%	10116K	11484K
10			15000	97%	100%	14402K	34184K	95%	190%	9704K	10064K
11
12	locust 0.13.5	--no-web -c 60 -r 60 --run-time 30s -f ./locust.py --master --expect-slaves=4	2900	320%	404%	170722K	198056K	23%	25%	9862K	11492K	Distributed execution and FastHttpLocust
13			2900	328%	401%	171937K	198368K	23%	26%	9617K	11500K
14			2900	347%	404%	178600K	197980K	24%	28%	9210K	10080K
15
16	locust 0.13.5	--no-web -c 60 -r 60 --run-time 30s -f ./locust.py	857	98%	102%	39930K	40916K	8%	10%	9573K	11504K	Single-threaded execution
17			857	97%	103%	39482K	40800K	8%	10%	9215K	10088K
18			861	97%	102%	39707K	40840K	8%	10%	10696K	11508K
19
20	k6 0.26.0-dev	run -u 30 -d 30s --compatibility-mode=base k6.js	11300	366%	383%	64339K	94640K	77%	99%	9334K	10080K
21			11300	369%	383%	66628K	94896K	77%	87%	9172K	11500K
22			11300	359%	387%	66614K	94724K	76%	84%	9705K	11500K
23
24	k6 0.26.0-dev	run -u 30 -d 30s k6-2.js	9900	335%	391%	217657K	268428K	68%	78%	9174K	10080K	Init is pretty slow
25			9900	334%	393%	216542K	266648K	68%	86%	9279K	11504K
26			9800	333%	391%	217289K	265816K	67%	78%	9492K	11504K
27
28	Gatling 3.3.1	vus=30, duration=30	4400	291%	400%	330082K	428328K	31%	84%	9558K	11512K	Slow startup and exit
29			4700	290%	400%	362519K	463568K	32%	77%	9072K	11512K
30			4400	294%	400%	360004K	473364K	30%	76%	9129K	11512K
31
32	Jmeter 5.2.1	-n -t jmeter.xml -j jmeter.log (30 VU, 10k reqs/vu)	6500	373%	400%	525051K	787044K	48%	79%	8938K	10092K	OpenJDK 11.0.5
33			6900	372%	400%	532803K	775248K	49%	73%	8726K	10092K
34			7000	377%	400%	491748K	766768K	50%	86%	9104K	11512K
35
36		Tried Oracle Java 13.0.1 and upping memory to 2g/512m - same performance
37
38	Tsung 1.7.0	-n -f ./tsung.xml start	8100	319%	385%	107169K	146312K	57%	69%	9395K	11512K	Tsung 1.7.0 and 1.6.0 seems to perform about the same in the lab setup
39			9000	357%	388%	118328K	208504K	61%	74%	9129K	10092K
40			9000	356%	390%	183382K	317176K	58%	73%	8857K	10092K
41
42	Siege 4.0.4	-c 6-t 30S -l siege.log -b --no-follow --no-parser	2200	178%	314%	7908K	7908K	41%	54%			Freezes at end of test 25% of the time, at <10 VU levels. 30s tests got aborted 3 times in a row due to excessive socket failure. Failures disappeared when turning off keep-alive, but that cut RPS throughput by 50%	Siege 4.0.4 seems about half as fast as 4.0.3
43			2400	193%	309%	7960K	7960K	43%	53%	11524K	11524K
44			2300	192%	316%	8096K	8096K	42%	55%	11527K	11528K
45
46	Artillery 1.6.0-29	arrival: 10/s, 12 reqs/scenario	118	76%	113%	90989K	100780K	2%	3%	8056K	11604K	CPU warnings whenever CPU usage averages over 80-90%. This config tries to eliminate warnings.
47			119	76%	114%	90588K	100540K	2%	3%	7129K	10164K
48			119	78%	97%	92089K	99988K	2%	3%	7292K	10172K
49
50		10/s, 2000 reqs/scenario	304	101%	120%	93091	98300	4%	5%	7870	10220	CPU warnings!
51		10/s, 10000 reqs/scenario	316	101%	119%	99934	123800	4%	5%	7747	11644	CPU warnings!
52		10/s, 30000 reqs/scenario	321	100%	138%	105980	144712	5%	5%	7764	11644	CPU warnings!
53
54
55	Hey	-c 10 -z 30s -disable-redirects	17900	366%	378%	95267K	152404K	117%	133%	11197K	11620K
56			18100	358%	384%	96430K	151224K	117%	129%	10512K	11620K
57			17900	358%	380%	96729K	152312K	123%	136%	11182K	11620K
58
59	Vegeta 12.7.0-dev	-duration=30s -max-workers=20 -rate=0	13600	352%	361%	25796	25856	89%	99%	11010	11644	latest commit to master - https://github.com/tsenart/vegeta/commit/85244b4acb411f39075b7f47533ec04ac438201b
60		-duration=30s -max-workers=20 -rate=0	13600	348%	364%	26347	26780	93%	112%	10272	11644
61		-duration=30s -max-workers=20 -rate=0	13600	347%	363%	26281	26792
62
63	curl-basher 0.0	concurrency=8 iterations=500	147	13%	18%	48668	77508	5%	5%	11676	11676	CPU usage is low, but we can't get higher RPS numbers by increasing concurrency, so some resource is running out. Perhaps it is CPU time elsewhere, like in the kernel. Perhaps it gets used up for memory allocation etc when starting so many curl instances all the time
64			147	13%	19%	49233	69508	5%	5%	11496	11676
65			147	13%	16%	46936	67744	5%	5%	11557	11676
66
67	Drill 0.5.0	-q -s -b drill.yml (8 threads, 500 iterations)	176	378%	400%	12952	13512	6%	6%	11533	11676
68			175	376%	397%	12876	13160	5%	6%	11313	11676
69			176	374%	399%	12780	13380	6%	6%	11533	11676
70
71
72
73			https://jsfiddle.net/hL57wkrj/3/
74
75
76
77			https://jsfiddle.net/d5tm2yua/
78
79
80
81
82
83
84
85
86		https://jsfiddle.net/d5tm2yua/5/
87
88
89
90
91
92
93
94
95
96
97
98
99
100