1 of 125

Rendering performance

Inside Out

@g33konaut

2 of 125

Hi, I’m Martin

Head of Engineering @ Archilogic, from Zurich

​

@g33konaut

3 of 125

Hi, I’m Martin

Head of Engineering @ Archilogic, from Zurich

GDE, W3C, Mozillian, I <3 The Web

@g33konaut

4 of 125

Whhhhhyyyyyy?

Time

​

Emotion

@g33konaut

5 of 125

Whhhhhyyyyyy?

Time: Get out of the way of your users...

​

Emotion

@g33konaut

6 of 125

Whhhhhyyyyyy?

Time: Get out of the way of your users...

​

Emotion: Make people happy, not frustrated!

@g33konaut

7 of 125

Agenda

  • Part 1: Computer graphics 101

​

@g33konaut

8 of 125

Agenda

  • Part 1: Computer graphics 101
  • Part 2: Browser rendering

@g33konaut

9 of 125

Part 1

Computer Graphics 101

@g33konaut

10 of 125

Pipeline (simplified)

Layer

Layer

Shader

@g33konaut

11 of 125

Pipeline (simplified)

Wat?

Wat?

Shader

Shader

?!

@g33konaut

12 of 125

Let’s start with pixels

@g33konaut

13 of 125

Pixels...

@g33konaut

14 of 125

Pixels...

100% red

66% green

33% blue

@g33konaut

15 of 125

Pixels...

100% red

66% green

33% blue

255

170

85

FF

AA

55

@g33konaut

16 of 125

Pixels...

100% red

66% green

33% blue

255

170

85

FF

AA

55

A bunch of numbers

@g33konaut

17 of 125

Pixels...

100% red

66% green

33% blue

255

170

85

FF

AA

55

A bunch of bytes

@g33konaut

18 of 125

Pixels = Memory

const pixels = [ 255, 170, 85,

255, 255, 255,

255, 0, 0 ]

@g33konaut

19 of 125

Drawing = Writing

const pixels = [

0, 0, 0,

0, 0, 0,

0, 0, 0 ]

​

​

​

@g33konaut

20 of 125

Drawing = Writing

const pixels = [

255, 0, 0,

0, 0, 0,

0, 0, 0 ]

​

​

​

@g33konaut

21 of 125

Drawing = Writing

const pixels = [

255, 0, 0,

0, 255, 0,

0, 0, 0 ]

​

​

​

@g33konaut

22 of 125

Drawing = Writing

const pixels = [

255, 0, 0,

0, 255, 0,

0, 0, 255 ]

​

​

​

@g33konaut

23 of 125

Drawing = Writing

const pixels = [

255, 255, 0,

0, 255, 255,

255, 0, 255 ]

​

​

​

@g33konaut

24 of 125

Concurrency ftw!

const pixels = [

255, 0, 0,

0, 255, 0,

0, 0, 255 ]

The pixels are independent!

@g33konaut

25 of 125

This can get complex

255, 000, 000, 255, 255, 255, 255, 255, 255, 255, 000, 000, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255

255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 000, 000, 255, 255, 255, 255, 255, 255, 255, 000, 000, 255, 255, 255

@g33konaut

26 of 125

These are the writes...

255, 000, 000, 255, 255, 255, 255, 255, 255, 255, 000, 000, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255

255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 000, 000, 255, 255, 255, 255, 255, 255, 255, 000, 000, 255, 255, 255

@g33konaut

27 of 125

Sprite

Layer

Texture

@g33konaut

28 of 125

Sprites ftw!

Sprite 0

Sprite 1

@g33konaut

29 of 125

Sprites (layers) ftw!

Sprite 0

Sprite 1

Transparency =

A fourth number...

@g33konaut

30 of 125

Drawing Step 0:

Copy Sprite 0

onto the screen

@g33konaut

31 of 125

Drawing Step 1:

Copy Sprite 1

onto the screen

@g33konaut

32 of 125

Drawing Step 2:

Copy Sprite 0

onto the screen

(again)

@g33konaut

33 of 125

Drawing Step 3:

Copy Sprite 1

onto the screen

(at new position)

@g33konaut

34 of 125

Drawing: Repeat!

@g33konaut

35 of 125

Drawing: Repeat!

This is also known as compositing

@g33konaut

36 of 125

Compositing?

Input: - Screen memory before (destination)

​

@g33konaut

37 of 125

Compositing?

Input: - Screen memory before (destination)

- Sprite to be added (source)

​

​

@g33konaut

38 of 125

Compositing?

Input: - Screen memory before (destination)

- Sprite to be added (source)

​

Output: Screen memory after (destination)

@g33konaut

39 of 125

How to composite?

&

?

@g33konaut

40 of 125

How to composite?

  • output[x, y] = source[x,y] || dest[x, y]

||

@g33konaut

41 of 125

How to composite?

  • output[x, y] = source[x, y] || dest[x, y]
  • output[x, y] = source[x, y] * dest[x, y]

||

@g33konaut

42 of 125

How to composite?

  • output[x, y] = source[x, y] || dest[x, y]
  • output[x, y] = source[x, y] * dest[x, y]
  • A ton more equations available!

@g33konaut

43 of 125

How to composite?

  • output[x, y] = source[x, y] || dest[x, y]
  • output[x, y] = source[x, y] * dest[x, y]
  • A ton more equations available!
  • This is called blending

@g33konaut

44 of 125

CSS blend modes

@g33konaut

45 of 125

Example: “screen”

color(x,y) = 1 - (1 - cat(x,y)) * (1 - sky(x,y))

@g33konaut

46 of 125

More compositing!

  • Translate (move):

output[x, y] = source[x + a, y + b] + dest[x, y]

@g33konaut

47 of 125

More compositing!

  • Translate (move):

output[x, y] = source[x + a, y + b] + dest[x, y]

  • Scale:

output[x, y] = source[x*sx, y*sy] + dest[x, y]

@g33konaut

48 of 125

More compositing!

  • Translate (move):

output[x, y] = source[x + a, y + b] + dest[x, y]

  • Scale:

output[x, y] = source[x*sx, y*sy] + dest[x, y]

  • Rotate: Matrix multiplication...

@g33konaut

49 of 125

Rotation!

cos(a) -sin(a)

sin(a) cos(a)

​

(ox, oy) = (x, y) *

@g33konaut

50 of 125

Composite only

  • Rotate, translate, scale

@g33konaut

51 of 125

Composite only

  • Rotate, translate, scale
  • Blending

@g33konaut

52 of 125

Composite only

  • Rotate, translate, scale
  • Blending
  • Filters

​

@g33konaut

53 of 125

Composite only

  • Rotate, translate, scale
  • Blending*
  • Filters*, **

​

*) only if the element is on its own layer

**) not all of them, not in all browsers

​

@g33konaut

54 of 125

Composite-only filters

@g33konaut

55 of 125

Composite-only​ f​ilters

  • grayscale
  • blur
  • contrast
  • hue-rotate
  • invert
  • opacity
  • saturate
  • sepia

​

@g33konaut

56 of 125

Composite-only​ f​ilters

  • grayscale
  • blur
  • contrast
  • hue-rotate
  • invert
  • opacity
  • saturate
  • sepia

​

@g33konaut

57 of 125

Composite-only​ f​ilters

  • grayscale
  • blur
  • contrast
  • hue-rotate
  • invert
  • opacity
  • saturate
  • sepia

​

  • drop-shadow

@g33konaut

58 of 125

Composite-only​ f​ilters

  • grayscale
  • blur
  • contrast
  • hue-rotate
  • invert
  • opacity
  • saturate
  • sepia

​

  • drop-shadow

@g33konaut

59 of 125

Composite-only​ f​ilters

  • grayscale
  • blur
  • contrast
  • hue-rotate
  • invert
  • opacity
  • saturate
  • sepia

​

  • drop-shadow

@g33konaut

60 of 125

Compositing:

let shader = (x, y, sprites, blend, filter) => {� return filter((blend(x, y, sprites)) // colour�}

​

@g33konaut

61 of 125

Return of the pipeline

Numbers

Numbers

Composite

Numbers

@g33konaut

62 of 125

Recap Part 1:

  • Pixels are a bunch of numbers in memory

​

@g33konaut

63 of 125

Recap Part 1:

  • Pixels are a bunch of numbers in memory
  • Useful to be separate in “sprites” (layers)

@g33konaut

64 of 125

Recap Part 1:

  • Pixels are a bunch of numbers in memory
  • Useful to be separate in “sprites” (layers)
  • Compositing = combining layers

@g33konaut

65 of 125

Recap Part 1:

  • Pixels are a bunch of numbers in memory
  • Useful to be separate in “sprites” (layers)
  • Compositing = combining layers
  • Blending = how we combine

@g33konaut

66 of 125

Recap Part 1:

  • Pixels are a bunch of numbers in memory
  • Useful to be separate in “sprites” (layers)
  • Compositing = combining layers
  • Blending = how we combine
  • Compositing can be done concurrently

@g33konaut

67 of 125

Part 2:

Browser rendering

CC-BY-NC Google Developers

Illustrations: @pandatology

@g33konaut

68 of 125

The art of performance

“Performance is the art of avoiding work”

  • Paul Lewis

@g33konaut

69 of 125

Different paths through

CC-BY-NC Google Developers

@g33konaut

70 of 125

Parsing

<style>

.box { … }

</style>

<h2>Warning</h2>�<div class=”box”>� Hello world�</div>

​

CSSOM entry

DOM entry

DOM entry

@g33konaut

71 of 125

Layouting

<h2>

HTMLHeadingElement

<div>

HTMLDivElement

@g33konaut

72 of 125

Layouting

<h2>

HTMLHeadingElement

<div>

HTMLDivElement

<h2>

HTMLHeadingElement

<div>

HTMLDivElement

CSSOM

@g33konaut

73 of 125

Layouting

@g33konaut

74 of 125

Paint

@g33konaut

75 of 125

But now: JavaScript

var elem = document.querySelector(‘div’)�elem.style.width = ‘100%’;

​

​

​

@g33konaut

76 of 125

But now: JavaScript

var elem = document.querySelector(‘div’)�elem.style.width = ‘100%’;

​

​

​

  1. Layout
  2. Paint

@g33konaut

77 of 125

Let’s play a game...

#transform {

transform: translateX(0);

}

​

@g33konaut

78 of 125

Let’s play a game...

#transform {

transform: translateX(0);

}

setTimeout(() => { � el.style.transform = 'translateX(200px)' �}, 2000)

@g33konaut

79 of 125

Will it paint?

@g33konaut

80 of 125

Yes.

@g33konaut

81 of 125

WHY U NO COMPOSITE?

​

@g33konaut

82 of 125

Why does it paint?

  • Textures (“layers”) are expensive

@g33konaut

83 of 125

Why does it paint?

  • Textures (“layers”) are expensive
  • Engine relies on hints for layers

@g33konaut

84 of 125

Why does it paint?

  • Textures (“layers”) are expensive
  • Engine relies on hints for layers, such as
    • <video> & <canvas>

@g33konaut

85 of 125

Why does it paint?

  • Textures (“layers”) are expensive
  • Engine relies on hints for layers, such as
    • <video> & <canvas>
    • 3D transforms

@g33konaut

86 of 125

Why does it paint?

  • Textures (“layers”) are expensive
  • Engine relies on hints for layers, such as
    • <video> & <canvas>
    • 3D transforms
    • composite-only animations

@g33konaut

87 of 125

Why does it paint?

  • Textures (“layers”) are expensive
  • Engine relies on hints for layers, such as
    • <video> & <canvas>
    • 3D transforms
    • composite-only animations
    • will-change property

@g33konaut

88 of 125

Let’s play again...

#transform { transform: translate3d(0, 0, 0); }

​

​

setTimeout(() => { � el.style.transform = 'translate3d(200px, 0, 0)' �}, 2000)

@g33konaut

89 of 125

Will it paint?

@g33konaut

90 of 125

No. 3D transform

@g33konaut

91 of 125

Round 3:

#transform { animation: move 2s infinite; }�@keyframes move {� 0% { transform: translateX(0); }� 100% { transform: translateX(200px); }�}

@g33konaut

92 of 125

Will it paint?

@g33konaut

93 of 125

No. Composite-only anim

@g33konaut

94 of 125

Yea, I know...

@g33konaut

95 of 125

Using will-change

@keyframes move {� 0% { left: 0; }� 100% { left: 200px; } }

@g33konaut

96 of 125

Using will-change

@keyframes move {� 0% { left: 0; }� 100% { left: 200px; } }

@g33konaut

97 of 125

Using will-change

@keyframes move {� 0% { left: 0; }� 100% { left: 200px; } }

#transform {

will-change: left;

animation: move 2s infinite; }

@g33konaut

98 of 125

Using will-change

@keyframes move {� 0% { left: 0; }� 100% { left: 200px; } }

#transform {

will-change: left;

animation: move 2s infinite; }

@g33konaut

99 of 125

Using will-change

@g33konaut

100 of 125

Using will-change

This only works if the element

isn’t in the layout flow

(e.g. position: absolute)

@g33konaut

101 of 125

Browser summary

  • Layout & paint are expensive

​

@g33konaut

102 of 125

Browser summary

  • Layout & paint are expensive
  • Compositing is only enabled when there is a layer

​

@g33konaut

103 of 125

Browser summary

  • Layout & paint are expensive
  • Compositing is only enabled when there is a layer
  • Layers have a cost, do not overuse!

@g33konaut

104 of 125

Browser summary

  • Layout & paint are expensive
  • Compositing is only enabled when there is a layer
  • Layers have a cost, do not overuse!
  • Signals in CSS lead to layer creation
    • 3D transforms
    • will-change
    • composite-only animations

@g33konaut

105 of 125

Browser summary

  • Layout & paint are expensive
  • Compositing is only enabled when there is a layer
  • Layers have a cost, do not overuse!
  • Signals in CSS lead to layer creation
    • 3D transforms
    • will-change
    • composite-only animations
  • Check out csstriggers.com

@g33konaut

106 of 125

Measure

then

optimise

@g33konaut

107 of 125

Thank you!

Slides: bit.ly/ft17-renderperf

Web: spaces.archilogic.com

50linesofco.de

Twitter: @g33konaut

Illustrations: @pandatology

@g33konaut

108 of 125

Part 3:

Canvas2D & WebGL

@g33konaut

109 of 125

So <canvas> is a layer...

@g33konaut

110 of 125

Canvas 2D test drive

for(var i=0; i<NUM_OBJECTS; i++) {� var x = Math.random() * HD_WIDTH, � y = Math.random() * HD_HEIGHT,� size = Math.random() * 512� ctx.drawImage(img,

x - HALF_SIZE, y - HALF_SIZE, size, size

)�}

@g33konaut

111 of 125

Canvas 2D result

@g33konaut

112 of 125

Canvas 2D result

Yay

Meh

Bah!

@g33konaut

113 of 125

What about WebGL?

​

​

<!DOCTYPE html>�<html>�<body>� <canvas width="250" height="250"></canvas>� <script type="text/plain" id="vs">� attribute vec3 avPos;� void main(void) {� gl_Position = vec4(avPos, 1.0);� }� </script>� <script type="text/plain" id="fs">� void main(void) {� gl_FragColor = vec4(1.0, 1.0, 1.0, 1.0);� }� </script>� <script>� var gl = document.querySelector('canvas').getContext('webgl')�� gl.viewport(0, 0, 250, 250)� gl.clearColor(0.0, 0.0, 0.0, 1.0)� gl.clear(gl.COLOR_BUFFER_BIT)�� var vertices = [-1.0, -1.0, 0.0, 1.0, -1.0, 0.0, 0, 1.0, 0.0]��

var vertexBuf = gl.createBuffer()� gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuf)� gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW)�

​

var program = gl.createProgram()� var vs = gl.createShader(gl.VERTEX_SHADER)� var fs = gl.createShader(gl.FRAGMENT_SHADER)�� gl.shaderSource(vs, document.getElementById('vs').textContent)� gl.shaderSource(fs, document.getElementById('fs').textContent)�� gl.compileShader(vs)� console.log('Vertex shader compilation:', gl.getShaderInfoLog(vs))�� gl.compileShader(fs)� console.log('Fragment shader compilation:', gl.getShaderInfoLog(fs))�� gl.attachShader(program, vs)� gl.attachShader(program, fs)�� gl.linkProgram(program)� if(!gl.getProgramParameter(program, gl.LINK_STATUS)) {� alert('Linking has failed.')� }�

gl.useProgram(program)�� var vertexPosAttrib = gl.getAttribLocation(program, 'avPos')� gl.enableVertexAttribArray(vertexPosAttrib)� gl.vertexAttribPointer(vertexPosAttrib, 3, gl.FLOAT, false, 0, 0)�� requestAnimationFrame(function render() {� gl.drawArrays(gl.TRIANGLES, 0, 3)� requestAnimationFrame(render)� })� </script>�</body>�</html>

@g33konaut

114 of 125

What about WebGL?

<!DOCTYPE html>�<html>�<body>� <canvas width="250" height="250"></canvas>� <script type="text/plain" id="vs">� attribute vec3 avPos;� void main(void) {� gl_Position = vec4(avPos, 1.0);� }� </script>� <script type="text/plain" id="fs">� void main(void) {� gl_FragColor = vec4(1.0, 1.0, 1.0, 1.0);� }� </script>� <script>� var gl = document.querySelector('canvas').getContext('webgl')�� gl.viewport(0, 0, 250, 250)� gl.clearColor(0.0, 0.0, 0.0, 1.0)� gl.clear(gl.COLOR_BUFFER_BIT)�� var vertices = [-1.0, -1.0, 0.0, 1.0, -1.0, 0.0, 0, 1.0, 0.0]��

var vertexBuf = gl.createBuffer()� gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuf)� gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW)�

​

var program = gl.createProgram()� var vs = gl.createShader(gl.VERTEX_SHADER)� var fs = gl.createShader(gl.FRAGMENT_SHADER)�� gl.shaderSource(vs, document.getElementById('vs').textContent)� gl.shaderSource(fs, document.getElementById('fs').textContent)�� gl.compileShader(vs)� console.log('Vertex shader compilation:', gl.getShaderInfoLog(vs))�� gl.compileShader(fs)� console.log('Fragment shader compilation:', gl.getShaderInfoLog(fs))�� gl.attachShader(program, vs)� gl.attachShader(program, fs)�� gl.linkProgram(program)� if(!gl.getProgramParameter(program, gl.LINK_STATUS)) {� alert('Linking has failed.')� }�

gl.useProgram(program)�� var vertexPosAttrib = gl.getAttribLocation(program, 'avPos')� gl.enableVertexAttribArray(vertexPosAttrib)� gl.vertexAttribPointer(vertexPosAttrib, 3, gl.FLOAT, false, 0, 0)�� requestAnimationFrame(function render() {� gl.drawArrays(gl.TRIANGLES, 0, 3)� requestAnimationFrame(render)� })� </script>�</body>�</html>

Hahaha, no, don’t try to read this!

http://bit.ly/298HnXB

@g33konaut

115 of 125

WebGL result

@g33konaut

116 of 125

Accelerated Canvas2D

“But did you measure it with GPU acceleration?”

  • Men on the internet

@g33konaut

117 of 125

Yes.

@g33konaut

118 of 125

Yes.

“Canvas 2D turns my computer into a hovering hot stove at 50k objects, btw”

  • Me

@g33konaut

119 of 125

WebGL all the things?

@g33konaut

120 of 125

No.

@g33konaut

121 of 125

Use the right tool

  • HTML + CSS
    • Semantic & accessible content
    • UI primitives

​

@g33konaut

122 of 125

Use the right tool

  • HTML + CSS
    • Semantic & accessible content
    • UI primitives
  • SVG
    • Custom, responsive 2D shapes & declarative animations

​

@g33konaut

123 of 125

Use the right tool

  • HTML + CSS
    • Semantic & accessible content
    • UI primitives
  • SVG
    • Custom, responsive 2D shapes & declarative animations
  • Canvas2D
    • Custom 2D shapes & simple 2D graphics

​

@g33konaut

124 of 125

Use the right tool

  • HTML + CSS
    • Semantic & accessible content
    • UI primitives
  • SVG
    • Custom, responsive 2D shapes & declarative animations
  • Canvas2D
    • Custom 2D shapes & simple 2D graphics
  • WebGL
    • Fully accelerated 2D graphics & 3D content

@g33konaut

125 of 125

Thank you!

Slides: bit.ly/ft17-renderperf

Web: spaces.archilogic.com

50linesofco.de

Twitter: @g33konaut

Illustrations: @pandatology

@g33konaut