High Performance Browser Networking

what every web developer should know about networking and browser performance

Ilya Grigorik - @igrigorik

igrigorik@google.com

Our agenda for the day...

@igrigorik

• Connection speed
• Latency and bandwidth
• Bandwidth doesn’t matter (much)
• Transports: TCP, TLS
• Connection setup + congestion control
• Protocol optimizations
• Wireless networking
• Implications of shared channel
• Impact of battery life
• Performance best practices
• Optimizing application delivery
• The good and the bad of HTTP 1.x
• Brief introducing to HTTP 2.0

​

Web performance this, …, that ...

Networks will just get faster and all of our problems will go away. Right?

… sadly, nope.

• ISPs and carriers love to advertise bandwidth as “speed”.
• ISP sells the “last mile” connection; ISP can’t guarantee end-to-end throughput.�
• When was the last time you saw an ISP advertise latency? Right, never...

@igrigorik

Speed is a feature (Chapter 1)

Speed of light is not fast enough!

• 42 ms RTT for a single packet between NYC and San Francisco
• 56 ms RTT for a single packet between NYC and London

@igrigorik

Speed is a feature (Chapter 1)

• Oh, and these are great-circle distances. The actual RTT’s are even higher!
• And then there is the “last-mile latency” problem...

Last mile is slow!

Fiber-to-the-home services provided 18 ms round-trip latency on average, while cable-based services averaged 26 ms, and DSL-based services averaged 43 ms. This compares to 2011 figures of 17 ms for fiber, 28 ms for cable and 44 ms for DSL.

@igrigorik

Measuring Broadband America - July 2012 - FCC

Last mile == Transatlantic hop? Wat!? Yeah...

Now you know why carriers don’t advertise these numbers…

• If you look hard enough, you can find them in their FAQ’s.
• Tip… $> traceroute destination.com

@igrigorik

Speed is a feature (Chapter 1)

(26 + 28) * 2

​

108 ms RTT!

Worldwide: ~100 ms

US: ~50~60 ms

​

Average RTT to Google in 2012 was…

and sadly it’s not improving much

4G will save us all! Says marketing guy...

"Users of the Sprint 4G network can expect to experience average speeds of 3 Mbps to 6 Mbps download and up to 1.5 Mbps upload with an average latency of 150 ms. On the Sprint 3G network, users can expect to experience average speeds of 600 Kbps - 1.4 Mbps download and 350 Kbps - 500 Kbps upload with an average latency of 400 ms."

@igrigorik

Mobile Networks (Chapter 7)

(everytime I see a “4G speed” advertisement)

Wireless “last mile” is, painful...

@igrigorik

Mobile Networks (Chapter 7)

OK. Latency is a problem.

but bandwidth is important too, how we’re doing there?

Brazil: ~2.5Mbps, Australia: ~5Mbps, US, UK: ~8Mbps

State of the Internet - Akamai - 2007-2013

Good news, mobile BW is improving rapidly!

That’s what the ads are all about. �Yes, last-mile latency is also much better (~50 ms), but that’s still worse than most wired connections!

​

​

@igrigorik

Mobile Networks (Chapter 7)

Bandwidth + Latency ≈ Performance

but what’s the actual relationship between these factors?

Components of an HTTP request

@igrigorik

Primer on Web Performance (Chapter 10)

• DNS lookup to resolve the hostname to IP address
• New TCP connection requires a handshake roundtrip to the server
• TLS handshake (not shown) with up to two extra server roundtrips�
• HTTP request requires minimum of a one roundtrip to the server
• Plus server processing time

Our pages consist of dozens of assets

• 52 requests
• 4+ seconds

Huh?

@igrigorik

Primer on Web Performance (Chapter 10)

… (snip 30 requests) ...

Latency vs. Bandwidth impact on Page Load Time

“To speed up the Internet at large, we should look for more ways to bring down RTT. What if we could reduce cross-atlantic RTTs from 150 ms to 100 ms? This would have a larger effect on the speed of the internet than increasing a user’s bandwidth from 3.9 Mbps to 10 Mbps or even 1 Gbps.” - Mike Belshe

@igrigorik

Single digit % perf improvement after�5 Mbps

Primer on Web Performance (Chapter 10)

Linear improvement in page load time!

• Video streaming is bandwidth limited.
• Web browsing is latency limited.

Why? To answer that, let’s dig a bit deeper...

Optimizing transport performance

aka, optimizing TCP and TLS...

SYN, SYN-ACK, ACK

@igrigorik

Building blocks of TCP (Chapter 2)

• Every TCP connection begins with a handshake (SYN > SYN-ACK > ACK)
• No data can be sent until the handshake is complete (one RTT)
• e.g. 56 ms in example above

TCP Slow-Start (congestion control)

@igrigorik

Building blocks of TCP (Chapter 2)

• Server is only allowed to send 4 segments (cwnd = 4)
• Once segments are acked, we double the window to 8 packets. Rinse, repeat...

Let’s do science to it…

@igrigorik

• RTT: roundtrip time between client and server
• cwnd: initial congestion window size
• N: number of segments we want to transfer
• segment: ~1460 bytes.

​

​

How long would it take to transfer 64 KB?

(on a new TCP connection)

​

Building blocks of TCP (Chapter 2)

Let’s fetch a 20 KB HTML file...

@igrigorik

• RTT: 56 ms
• Bandwidth: 5 Mbps
• cwnd: 4

• 56 ms for TCP handshake
• 40 ms for request processing
• 4x segments
• 8x segments
• 3x segments

​

236 milliseconds!

Building blocks of TCP (Chapter 2)

Let’s fetch same file over existing TCP connection.

@igrigorik

• RTT: 56 ms
• Bandwidth: 5 Mbps
• cwnd: 11

• no TCP handshake...
• 40 ms for request processing
• 11 segments

​

96 milliseconds - 275% improvement!

Building blocks of TCP (Chapter 2)

Re-use existing TCP connections!

Also, notice that we’re (still) nowhere close to our 5 Mbps “bandwidth ceiling”.

Most of the HTTP data flows consist of small, bursty data transfers, whereas TCP is optimized for long-lived connections and bulk data transfers. Network roundtrip time is the limiting factor in TCP throughput and performance in most cases. Consequently, latency is the performance bottleneck for HTTP and most web applications delivered over it.

“Connection view” tells the story...

@igrigorik

Speed, performance and human perception

• 30 connections
• DNS lookups
• TCP handshakes
• …�
• Blue: download time

​

​

​

We’re not BW limited, we’re literally idling, waiting on the network to deliver resources.

Mystery solved...

• Upgrade to Linux 3.2+
• Increases cwnd to 10!
• Proportional rate reduction
• Disable “slow-start after idle”
• Ensure TCP window scaling is enabled

​

• Re-use established TCP connections
• HTTP keepalive (default on HTTP 1.1)
• Position servers closer to the user
• Lower RTTs! Use a CDN.
• Compress transferred data

Optimizing for TCP (Chapter 2)

Now let’s make it secure...

TLS is an unoptimized frontier for most sites!

SYN, SYN-ACK, ACK, … ClientHello, ServerHello, ...

@igrigorik

Transport Layer Security (Chapter 4)

• First, TCP handshake RTT!
• Then…
• ClientHello
• ServerHello
• ChangeCipher (client)
• ChangeCipher (server)

​

​

• 3 RTTs to establish tunnel!

You might want to consider re-using TLS connections! ��Just saying...

Session resumption removes one RTT!

@igrigorik

• Session Identifier (old)
• Client sends session ID.
• Session Ticket (new)
• “Client cookie”

​

​

Verify that your server (at least) supports session ticket resumption.��

Transport Layer Security (Chapter 4)

“Early termination” is a big win for TLS

@igrigorik

• Nearby edge server terminates TCP
• TLS negotiation is with the edge
• Applicable for static and dynamic content

​

• Edge then routes to origin...
• Via a “warm” connection

​

​

Early termination helps non-encrypted traffic also, but it is especially useful for accelerating TLS connections!

Transport Layer Security (Chapter 4)

* Talk to your CDN provider, or, you can roll your own.. Spin up some “cloud servers” and you’re in business.

• Start by optimizing TCP!
• IW10, proportional rate reduction, ...
• Upgrade to latest OpenSSL libraries
• Big performance difference (try, $> openssl speed rsa)
• Enable TLS resumption
• Session Tickets, Session Identifiers
• Optimize TLS record size
• Reduce to single network segment
• And a lot more...
• Disable TLS compression (attack vector)
• Configure SNI support (please!)
• Configure OSCP stapling
• Use HSTS to avoid costly HTTPS redirects

TLS Performance Checklist (Chapter 4)

Measuring network performance

Gathering data from real users, on real networks, with real devices...

Navigation Timing (W3C)

@igrigorik

Synthetic and Real-User Performance Measurement (Chapter 10)

Navigation Timing (W3C)

@igrigorik

Synthetic and Real-User Performance Measurement (Chapter 10)

@igrigorik

Available in...

​

• IE 9+
• Firefox 7+
• Chrome 6+
• Android 4.0+

Real User Measurement (RUM) with Google Analytics

@igrigorik

<script>� _gaq.push(['_setAccount','UA-XXXX-X']);� _gaq.push(['_setSiteSpeedSampleRate', 100]); // #protip� _gaq.push(['_trackPageview']);� </script>

Google Analytics > Content > Site Speed

​

• Automagically collects this data for you - defaults to 1% sampling rate
• Maximum sample is 10k visits/day
• You can set custom sampling rate

​

You have all the power of Google Analytics! Segments, conversion metrics, ...

setSiteSpeedSampleRate docs

Full power of Google Analytics to segment, filter, compare, ...

@igrigorik

Head into the Technical reports to see the histograms and distributions!

Averages are misleading...

@igrigorik

Synthetic and Real-User Performance Measurement (Chapter 10)

Case study: igvita.com server response times

@igrigorik

Content > Site Speed > Page Timings > Performance

​

Bimodal response time distribution?

Theory: user cache vs. database cache vs. full recompute

​

Synthetic and Real-User Performance Measurement (Chapter 10)

2G, 3G, 4G...

have fundamentally different architecture at the radio layer

Design criteria #1: "Stable" performance + scalability

• Control over network performance and resource allocation
• Ability to manage 10~100's of active devices within single cell
• Coverage of much larger area

Design criteria #2: Battery life optimization

• Radio is the second most expensive component (after screen)
• Limited amount of available energy (as you are well aware)

Design criteria #1: "Stable" performance + scalability

Design criteria #2: Maximize battery life

Radio Resource Controller (RRC)

Radio Resource Controller

@igrigorik

• Phone: Hi, I want to transmit data, please?
• RRC: OK.
• Transmit in [x-y] timeslots
• Transmit with Z power
• Transmit with Q modulation�

... (some time later) ...

​

• RRC: Go into low power state.

​

RRC

All communication and power management is centralized and managed by the RRC.

Mobile Networks (Chapter 7)

Control and User-plane latencies

@igrigorik

RRC

I want to

send data!

1

2

1-X RTT's of negotiations

3

Application data

Control-plane

User-plane

• There is a one time cost for control-plane negotiation�
• User-plane latency is the one-way latency between packet availability in the device and packet at the base station

Same process happens for incoming data, just reverse steps 1 and 2

Anticipate RRC latency...

Plan for "first packet" delay

• 100's to 1000's of milliseconds of delay for first packet
• Adjust UX accordingly!�

Much of the "variability" is explained by RRC delays

• you can't predict it, but assume you will incur it

Watch those energy tails!

@igrigorik

Wasted energy

3G state machine�

• DCH = Active
• FACH = Low power
• IDLE = ...

Every data transfer, both big and small, will:

• cycle radio into high power
• timer driven transition back to idle state

Mobile Networks (Chapter 7)

Minimize periodic data transfers

• avoid polling, use adaptive strategy
• coalesce requests, defer requests
• leverage (smart) server push

�

Prefetch data

• turn off the radio, keep it idle

It's all about the battery...

A Call for More Energy-Efficient Apps

​

• 5 Wh battery capacity
• 5 Wh * 3600 J/Wh = 18000 J battery capacity

​

Hands on example....

• 10 Joules of consumed energy per "cycle"
• idle → high-power → low-power → idle

• 60 minutes * 10 Joules = 600 Joules of consumed energy per hour
• 3% of battery capacity per hour! (600 J / 18000J)

Measuring energy & radio...

@igrigorik

Ping, ping, ping, ... where'd my battery go?

Watch out for...

• "real-time" analytics
• "real-time" comments
• "real-time" <widget>...

Mobile Networks (Chapter 7)

* The radio is never fully "off"... It wakes up periodically to listen to broadcasts and notifications!

The radio can be off* while the transport is idle.

Carrier timeouts: 5-30 minutes.�

• eliminate unnecessary keep-alive beacons...
• are you sure it's not your own servers forcing timeouts? :-)

Not so good news everybody! ....

HSPA+ will be the dominant network type of the next decade!

​

​

• Latest HSPA+ releases are comparable to LTE in performance

​

• 3G networks will be with us for at least another decade

​

• LTE adoption in US and Canada is way ahead of the world-wide trends

Mobile Networks (Chapter 7)

• It's a multi-generation future: 2G, 3G, 4G
• users migrate between G's all the time... plan for it!�
• Bandwidth and latency is highly variable
• burst your data, and return to idle...

​

• Connectivity is intermittent, errors will happen!
• have an offline mode - i.e. use a local cache
• use a smart backoff algorithm, please!

​

Design for variable network performance & availability

var backoff = backoff.strategy({

randomisationFactor: 0,

initialDelay: 60,

maxDelay: 600

});

​

backoff.failAfter(10);

Hypertext Transfer Protocol

The good, the bad, … and the bright future ahead (aka, 2.0)

$> telnet igvita.com 80

Connected to 173.230.151.99

​

GET /archive

​

Hypertext delivery with HTTP 0.9! - eom.

(connection closed)

HTTP 0.9 is the ultimate MVP - one line, plain-text “protocol” to test drive the “www idea”.

$> telnet ietf.org 80

Connected to 74.125.xxx.xxx

​

GET /rfc/rfc1945.txt HTTP/1.0

User-Agent: CERN-LineMode/2.15 libwww/2.17b3

Accept: */*

​

HTTP/1.0 200 OK

Content-Type: text/plain

Content-Length: 137582

Last-Modified: Wed, 1 May 1996 12:45:26 GMT

Server: Apache 0.84

​

4 years of rapid iteration later… eom.

(connection closed)

HTTP 1.0 is an informational RFC - documents “common usage” of HTTP found in the wild.

$> telnet google.com 80

Connected to 74.125.xxx.xxx

​

GET /index.html HTTP/1.1

Host: website.org

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4)... (snip)

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Encoding: gzip,deflate,sdch

Accept-Language: en-US,en;q=0.8

Cookie: __qca=P0-800083390... (snip)

​

HTTP/1.1 200 OK

Connection: keep-alive

Transfer-Encoding: chunked

Server: nginx/1.0.11

Content-Type: text/html; charset=utf-8

Date: Wed, 25 Jul 2012 20:23:35 GMT

Expires: Wed, 25 Jul 2012 20:23:35 GMT

Cache-Control: max-age=0, no-cache

​

100

<!doctype html>

(snip)

HTTP 1.1 ships as RFC standard in 1999 - hyper{t̶e̶x̶t̶}media all the things!

@igrigorik

15 years later!

HTTPbis is getting dangerously close to closing all the outstanding 1.1 bugs. :)

In the meantime a few things have happened since...

Speed, performance and human perception

Geocities ftw!�(circa 1997-2000)

Then AJAX happened.

(circa 2002-2005)

For all its awesome, HTTP 1.X has a lot of perf problems...

@igrigorik

​

• Limited parallelism (~6)
• Client-side request queuing
• Browser queueing heuristics
• High protocol overhead
• ~800 bytes + cookies

​

• Competing TCP flows
• Spurious retransmissions
• Extra memory / FD resources
• Handshake overhead
• Slow-start overhead
• …

Dozens of competing TCP flows...

@igrigorik

Speed, performance and human perception

• Most stuck in Slow Start
• Poor congestion control
• Poor link utilization
• Setup latency + overhead
• Extra sockets
• Extra memory
• …

​

​

In short, there is no upside to this. But that’s what we have to work with...

E.g. Etsy over-sharding...

@igrigorik

Speed, performance and human perception

• img[0-3].etsystatic.com
• ~30 connections

​

Too much of a “good thing” can cause harm. In this case, parallel flows are competing and cause a significant amount of spurious retransmissions (duplicate bytes on the client).

​

Network Congestion and Web Browsing

Where there’s a will, there’s a way...

we’re an inventive bunch, so we came up with some “optimizations” (read, “hacks”)

• Concatenating files (JavaScript, CSS)
• Reduces number of downloads and latency overhead
• Less modular code and expensive cache invalidations (e.g. app.js)
• Slower execution (must wait for entire file to arrive)�

• Spriting images
• Reduces number of downloads and latency overhead
• Painful and annoying preprocessing and expensive cache invalidations
• Have to decode entire sprite bitmap - CPU time and memory�

• Resource inlining
• Eliminates the request for small resources
• Resource can’t be cached, inflates parent document
• 30% overhead on base64 encoding

• Domain sharding
• TCP Slow Start? Browser limits, Nah... 15+ parallel requests -- Yeehaw!!!
• Causes congestion and unnecessary latency and retransmissions�

… why not fix HTTP instead?

• Improve end-user perceived latency
• Address the "head of line blocking"
• Not require multiple connections
• Retain the semantics of HTTP/1.1

HTTP 2.0 work kicked off in Jan 2012.

"HTTP 2.0 is a protocol designed for low-latency transport of content over the World Wide Web"

​

HTTP 2.0 in one slide…

• One TCP connection

​

• Request = Stream
• Streams are multiplexed
• Streams are prioritized

​

• Binary framing layer
• Prioritization
• Flow control
• Server push

​

• Header compression

@igrigorik

“... we’re not replacing all of HTTP — the methods, status codes, and most of the headers you use today will be the same. Instead, we’re redefining how it gets used “on the wire” so it’s more efficient, and so that it is more gentle to the Internet itself ....”

- Mark Nottingham (HTTPbis chair)

Binary framing crash course in one slide...

• Length-prefixed frames
• All frames have same 8-byte header

​

• Type indicates … type of frame:
• DATA, HEADERS, PRIORITY, PUSH_PROMISE, …
• Each frame may have custom flags
• e.g. END_STREAM
• Each frame carries a 31-bit stream identifier
• After that, it’s frame specific payload

@igrigorik

frame = buffer.read(8)

if frame_i_care_about

do_something_smart

else

buffer.skip(frame.length)

end

Basic data flow in HTTP 2.0...

• Streams are multiplexed by splitting communication into frames
• e.g. HEADERS, DATA, etc.�
• Frames are interleaved
• Frames can be prioritized
• Frames can be flow controlled�

E.g. Please send critical.css with priority 1, please! kittens.jpg with priority 10.

@igrigorik

Connection + stream flow-control!

Stream flow-control enables fine-grained resource control between streams. E.g…�

• T(0): I am willing to receive 64 KB of kittens.jpg.
• T(0): I am willing to receive 500KB of critical.js
• …
• T(n): Ok, now send the remainder of kittens.jpg.

​

Client controls how and when the stream and connection window is incremented!

@igrigorik

Server push… is replacing inlining.

Inlining is server push. Except, HTTP 2.0 server push is cacheable! �

• One request, multiple responses.
• Push redirects, cache invalidations, … lots of new and exciting possibilities!

@igrigorik

HTTP 2.0 provides header compression!

• Both sides maintain “header tables”
• New requests “toggle” or “insert” new values into the table

​

@igrigorik

* as low as 8 bytes for an identical request.

​

• New header set is a “diff” of the previous set of headers
• Repeat request (polling) with exact same headers incurs no overhead

​

Byte cost of new stream: 8 bytes! *

tl;dr: framing, mux, prioritization, flow control

same HTTP protocol semantics, complete performance reboot!

Benefits

Opportunities

@igrigorik

• Unshard, un-concat, un-sprite, …
• Modular code, modular assets
• Improved cache performance
• Improved page load times

​

• Layer other browser transports
• XHR / SSE: work out of the gate
• (TODO) WebSockets over HTTP 2.0: free mux, flow control, prioritization

​

• We need smarter servers!
• Support for prioritization, flow control, and resource push!

​

• One RPC stack to rule them all
• Standardized, high-performance RPC protocol stack (internal, external)

​

​

• min(request overhead) = 8 bytes
• max(parallelism) = 100~1000 streams
• max(client queueing latency) = 0 ms

​

• Stream multiplexing
• Stream prioritization
• Stream flow control
• Server push

​

• Plus all the usual HTTP goodies!
• Browser caching, authentication, content negotiation, content-encoding, transfer encoding, encryption, …

�

​

* or today, via SPDY.

Available in your browser circa ~2014! *

HTTP 2.0 draft 6 implementations in Firefox, Chrome, IE11

has SPDY support (stepping stone). Plus client/server

implementations in C, JavaScript (node.js), Ruby, Perl, etc.

​

HTTP 2.0 is coming to a client / server near you in 2014.

github.com/igrigorik/http-2

• Start by optimizing TCP!
• IW10, proportional rate reduction, …
• Ensure you have keep-alive enabled!
• Use HSTS to avoid costly HTTPS redirects
• Optimize your TLS deployments
• Record and cert sizes, session resumption, ...
• Compress and eliminate
• Enable gzip (please!) and fetch what you need
• Cache
• Too many sites are missing cache headers to this day
• Expiry / max-age and revalidation (Etag, Last-Modified)
• Investigate SPDY / HTTP 2.0

Optimizing Application Delivery (Chapter 13)

Are we fast yet?

Slides @ bit.ly/hpbn-talk

​

​

Twitter igrigorik

Email igrigorik@google.com

P.S. Check out the book for (much) more!

Not fast enough? Ok, let’s talk

WiFi performance

requires its own (unique) set of optimizations...

Wireless LAN, aka Wi-Fi...

@igrigorik

• Wireless extension to existing LAN infrastructure
• Same protocol stack, new physical medium
• First commercial success ~1999 with 802.11b (11 Mbps)�
• Target: desktop, laptop

​

WiFi (Chapter 6)

@igrigorik

• Limited amount of shared spectrum
• Multiple parties who want to transmit and receive

​

​

How do we schedule communication?

​

WiFi (Chapter 6)

Carrier Sense Multiple Access + Collision Detection

@igrigorik

• Is anyone talking?
• No: begin transmission
• Yes: wait until they finish
• Collision: stop, sleep for rand() with backoff, retry

​

​

There is not much more to it...

WiFi (Chapter 6)

Wi-Fi channels 101

@igrigorik

Non-overlapping channels: 1, 6, 11

WiFi (Chapter 6)

Real-world Wi-Fi performance: 2.4 Ghz vs 5 Ghz (YMMV)

Real-world 1st hop latency

Sample data from my own home Wi-Fi network...

Adapt to variable bandwidth

• Adaptive bitrate streaming
• 5-10 second chunks of video content
• Adapt, do not predict!

�

Adapt to variable latency and jitter

​

• High variability for first hop for every packet
• Jitter in packet arrival times
• Variability != packet loss
• Link layer retransmissions and error correction...