ABCDEFGHIJKL
1
Description
2
The intent of this experiment is to give us a baseline of where we stand wrt to the performance of ChromeApps using the chrome.socket API. In this document, we focus on the UDP API, specifically on the ability to handle heavy incoming traffic and measuring packet loss.
3
4
tldr;
5
The socket v2 API is 10x faster than v1, but still 4x slower than node.js.
6
7
Experiment details
8
Given a Win32 c++ apps sending UDP packets to "localhost" as fast as possible (naively, nothing fancy), measure the amount of packets various single instance clients can receive at the application layer. Note that packets are fairly small (10 bytes). First, we run only one instance of the "sender" app, then we run 3 instances in parralel to ensure we can overflow to receiver.

There are 4 types of "receiver" apps:

1) A Win32 C++ app is used as the baseline to measure "theoritical" (naive implementation, nothing too fancy) best performance on the "native" platform.

2) A node.js app is used as the baseline of what should be possible to achieve with a V8 based asynchronous JavaScript platform.

3) A chome app based on socket.udp v1 API.

4) A Chrome app based on an experimetal socket.udp v2 API. The main difference from v1 is the use of Chrome App Events instead of explicit read+callback operation to receive udp packets.
9
10
Summary of results
11
1) node.js performance is within a factor (~50% on average) of c++ performance
2) socket v1 is two order of magnitude (100x!) slower than c++, probably unacceptable for "real-world" server applications.
3) socket v2 is still one order of magnitude slower than c++ but one order of magnitude faster than v1 and much close(r) to node.js performance (25% of node performance).
4) Packet size (10 bytes vs 5,000 bytes) doesn't change the picture much. Bigger packets seem to slow down the "sender" process more than the "receiver", thus decreasing the amount of packet loss of the receivers.
12
13
Packet size = 15 bytes
14
15
C++ senderC++ receivernode.js receiverudp socket v2udp socket v1
16
1 sender instance
17
# send reqs1,000,000
18
Time (s)4.2
19
20
# recv reqs1,000,000550,000117,90910,982
21
% loss0%45%88%99%
22
# recv / sec238,095130,95228,0742,615
23
Relative perf vs c++100%55%12%1%
24
Relative perf vs node182%100%21%2%
25
26
3 sender instances running in parralel
27
# send reqs3,000,000
28
Time (s)5.58
29
30
# recv reqs2,658,426669,694167,50219,031
31
% loss11%78%94%99%
32
# recv / sec476,420120,01730,0183,411
33
Relative perf vs c++100%25%6%1%
34
Relative perf vs node397%100%25%3%
35
36
37
Packet size = 5000 bytes
38
39
C++ senderC++ receivernode.js receiverudp socket v2udp socket v1
40
1 sender instance
41
# send reqs1,000,000
42
Time (s)8.99
43
44
# recv reqs998,223681,306157,07114,951
45
% loss0%32%84%99%
46
# recv / sec111,03775,78517,4721,663
47
Relative perf vs c++100%68%16%1%
48
Relative perf vs node147%100%23%2%
49
50
3 sender instances running in parralel
51
# send reqs3,000,000
52
Time (s)24.35
53
54
# recv reqs2,977,6871,406,257353,72143,054
55
% loss1%53%88%99%
56
# recv / sec122,28757,75214,5271,768
57
Relative perf vs c++100%47%12%1%
58
Relative perf vs node212%100%25%3%