A | B | C | D | E | F | G | H | I | J | K | L | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Description | |||||||||||
2 | The intent of this experiment is to give us a baseline of where we stand wrt to the performance of ChromeApps using the chrome.socket API. In this document, we focus on the UDP API, specifically on the ability to handle heavy incoming traffic and measuring packet loss. | |||||||||||
3 | ||||||||||||
4 | tldr; | |||||||||||
5 | The socket v2 API is 10x faster than v1, but still 4x slower than node.js. | |||||||||||
6 | ||||||||||||
7 | Experiment details | |||||||||||
8 | Given a Win32 c++ apps sending UDP packets to "localhost" as fast as possible (naively, nothing fancy), measure the amount of packets various single instance clients can receive at the application layer. Note that packets are fairly small (10 bytes). First, we run only one instance of the "sender" app, then we run 3 instances in parralel to ensure we can overflow to receiver. There are 4 types of "receiver" apps: 1) A Win32 C++ app is used as the baseline to measure "theoritical" (naive implementation, nothing too fancy) best performance on the "native" platform. 2) A node.js app is used as the baseline of what should be possible to achieve with a V8 based asynchronous JavaScript platform. 3) A chome app based on socket.udp v1 API. 4) A Chrome app based on an experimetal socket.udp v2 API. The main difference from v1 is the use of Chrome App Events instead of explicit read+callback operation to receive udp packets. | |||||||||||
9 | ||||||||||||
10 | Summary of results | |||||||||||
11 | 1) node.js performance is within a factor (~50% on average) of c++ performance 2) socket v1 is two order of magnitude (100x!) slower than c++, probably unacceptable for "real-world" server applications. 3) socket v2 is still one order of magnitude slower than c++ but one order of magnitude faster than v1 and much close(r) to node.js performance (25% of node performance). 4) Packet size (10 bytes vs 5,000 bytes) doesn't change the picture much. Bigger packets seem to slow down the "sender" process more than the "receiver", thus decreasing the amount of packet loss of the receivers. | |||||||||||
12 | ||||||||||||
13 | Packet size = 15 bytes | |||||||||||
14 | ||||||||||||
15 | C++ sender | C++ receiver | node.js receiver | udp socket v2 | udp socket v1 | |||||||
16 | 1 sender instance | |||||||||||
17 | # send reqs | 1,000,000 | ||||||||||
18 | Time (s) | 4.2 | ||||||||||
19 | ||||||||||||
20 | # recv reqs | 1,000,000 | 550,000 | 117,909 | 10,982 | |||||||
21 | % loss | 0% | 45% | 88% | 99% | |||||||
22 | # recv / sec | 238,095 | 130,952 | 28,074 | 2,615 | |||||||
23 | Relative perf vs c++ | 100% | 55% | 12% | 1% | |||||||
24 | Relative perf vs node | 182% | 100% | 21% | 2% | |||||||
25 | ||||||||||||
26 | 3 sender instances running in parralel | |||||||||||
27 | # send reqs | 3,000,000 | ||||||||||
28 | Time (s) | 5.58 | ||||||||||
29 | ||||||||||||
30 | # recv reqs | 2,658,426 | 669,694 | 167,502 | 19,031 | |||||||
31 | % loss | 11% | 78% | 94% | 99% | |||||||
32 | # recv / sec | 476,420 | 120,017 | 30,018 | 3,411 | |||||||
33 | Relative perf vs c++ | 100% | 25% | 6% | 1% | |||||||
34 | Relative perf vs node | 397% | 100% | 25% | 3% | |||||||
35 | ||||||||||||
36 | ||||||||||||
37 | Packet size = 5000 bytes | |||||||||||
38 | ||||||||||||
39 | C++ sender | C++ receiver | node.js receiver | udp socket v2 | udp socket v1 | |||||||
40 | 1 sender instance | |||||||||||
41 | # send reqs | 1,000,000 | ||||||||||
42 | Time (s) | 8.99 | ||||||||||
43 | ||||||||||||
44 | # recv reqs | 998,223 | 681,306 | 157,071 | 14,951 | |||||||
45 | % loss | 0% | 32% | 84% | 99% | |||||||
46 | # recv / sec | 111,037 | 75,785 | 17,472 | 1,663 | |||||||
47 | Relative perf vs c++ | 100% | 68% | 16% | 1% | |||||||
48 | Relative perf vs node | 147% | 100% | 23% | 2% | |||||||
49 | ||||||||||||
50 | 3 sender instances running in parralel | |||||||||||
51 | # send reqs | 3,000,000 | ||||||||||
52 | Time (s) | 24.35 | ||||||||||
53 | ||||||||||||
54 | # recv reqs | 2,977,687 | 1,406,257 | 353,721 | 43,054 | |||||||
55 | % loss | 1% | 53% | 88% | 99% | |||||||
56 | # recv / sec | 122,287 | 57,752 | 14,527 | 1,768 | |||||||
57 | Relative perf vs c++ | 100% | 47% | 12% | 1% | |||||||
58 | Relative perf vs node | 212% | 100% | 25% | 3% |