ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
Thanks for popping in!
ANYONE CAN EDIT THIS SECTION
2
Trying to figure out what is the current bottleneck in copying files (etc)
3
~/commoncrawl/flashdata is a ramdisk!
4
See the tabs at the bottom for various tests.
5
6
Primarily testing /dev/sda, which is a hardware backed raid array
7
Raid 0 array of 8 Corsair SSD's:https://www.amazon.ca/Crucial-MX500-500GB-NAND-Internal/dp/B0784SLQM6/ref=sr_1_1?keywords=corsair+ssd&qid=1580870487&s=electronics&sr=1-1
8
9
Corsair rated write:510MB/s
10
Drives:8
11
Real World Tax(%)-20%
12
Real World Tax(MB/s)-816
13
14
"Marketing" Rating:4080MB/s
15
Estimated Actual3264MB/s
16
17
18
19
RSYNC Testing:
20
Writing to Raid Array from RamDisk
21
root@server:~/commoncrawl# cpr flashdata/largedeleteme.tar.gz ./
22
sending incremental file list
23
largedeleteme.tar.gz
24
7,275,427,840100% 313.24MB/s
25
26
27
28
Reading from Raid Array, writing to Ramdisk:
29
root@server:~/commoncrawl# cpr largedeleteme.tar.gz flashdata
30
sending incremental file list
31
largedeleteme.tar.gz
32
7,275,427,840100% 346.40MB/s
33
34
35
Don't get me wrong, I recognize that 300MB+ per second is not the slowest thing ever, I just want to understand what the limiting factor is!
36
37
38
39
40
41
NEW - m.2 Mdadm raid 0 X 3
42
43
44
root@server:~/commoncrawl# cpr m2data/largedeleteme.tar.gz ./flashdata/
45
sending incremental file list
46
largedeleteme.tar.gz
47
7,275,427,840 100% 335.17MB/s 0:00:20 (xfr#1, to-chk=0/1)
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100