1 of 37

Surface Aggregation

jbauman@

2 of 37

Overview

  • What are Surfaces?
  • History
  • Surfaces in Chrome
  • Compositor Frames
  • Surface Aggregation
  • Future work

3 of 37

Surfaces

4 of 37

What are Surfaces?

Surfaces are a concept to allow graphical embedding of heterogeneous untrusting clients efficiently into one scene

Still in flux

5 of 37

What are they used for?

6 of 37

Surface

  • Has an ID
  • Represents instructions about how to render an area.
  • Can embed other surfaces
  • Represent a sequence of frames
  • Updated independently
  • (Conceptually) have size and scale factor

7 of 37

Surface classes (in cc/Surfaces)

  • Surface represents a single surface, has draw callback
  • Can put CopyOutputRequests on surfaces
  • Has one active frame and one pending frame
  • SurfaceSequence handles refcounting

8 of 37

Surface classes

SurfaceManager

SurfaceFactory

SurfaceFactory

Surface

Surface

Display

Aggregator

Renderer

Scheduler

owns

owns

client

9 of 37

Surface classes

Renderer

Compositor

CompositorFrameSink

SurfaceFactory

MojoCompositorFrameSink IPC

CompositorFrameSinkSupport

10 of 37

Surfaces in Chrome

11 of 37

History

Originally started for mojo - many untrusted embedders

Added in Chrome

Previously child frames went into the browser compositor, and browser compositor spit out CompositorFrames directly into final Renderer

12 of 37

Surfaces in Mus

Move Display to GPU process.

Mus window server takes place of browser compositor.

13 of 37

What’s a Frame?

14 of 37

What’s in a Frame?

  • Metadata
  • A list of resources (with IDs)
    • GPU mailbox ID
    • Texture type
    • etc
  • A list of RenderPasses

15 of 37

Render passes

​

​

​

RenderPass 3

RenderPass 2

Copy Request

RenderPass 2

Surface 1

16 of 37

Quad

Rectangle

References SharedQuadState

  • Clip rect
  • Transform to RenderPass
  • Opacity

Resource references

c

c

17 of 37

Complicated quad transforms

18 of 37

Damage Rect

.

19 of 37

Metadata

Latency info

Top controls info

Child Surfaces

20 of 37

Surface Aggregation

21 of 37

Surface Aggregation

RenderPass 3

RenderPass 2

Copy Request

RenderPass 2

Surface 1

RenderPass 2

Copy Request

Surface 1

Surface 2

RenderPass 5

RenderPass 4

Copy Request

RenderPass 4

Render Pass 3

RenderPass 3

Copy Request

22 of 37

Optimizations

RenderPass 3

RenderPass 2

Copy Request

RenderPass 2

Surface 1

RenderPass 2

Surface 1

Surface 2

RenderPass 5

RenderPass 4

Copy Request

RenderPass 4

23 of 37

Damage Rectangle

Calculate minimal damage rect

RenderPass filters and CopyOutputRequest make complicated

Output one per Renderpass - tells renderer what to draw

Calculated using frame index

Try to avoid aggregating outside of it.

24 of 37

First pass

Start at root surface

Walk tree

Calculate damage rect

Map child to parent resource ids

25 of 37

Second pass

Walk tree again

Keep track of transform to current RenderPass (multiple Surfaces)

Aggregate everything within damage rect.

Add quads to list.

26 of 37

Hidden CopyOutputRequests

Some Surfaces may not actually be referenced by a quad, but have a CopyOutputRequest. Track them in first pass, and, aggregate them completely first.

27 of 37

Side note: overlays

28 of 37

Draw scheduling

29 of 37

Damage-triggered

If Surface gets new frame, propagate to Displays containing it.

If it’s added to parent, parent will cause draw.

Smart timer scheduling

Draw callback

​

30 of 37

Future work

31 of 37

Move damage rect calculation later

32 of 37

Occlusion culling

33 of 37

Improve synchronization

For fsamuel’s talk@ soon

Current summary - surfaces draw as soon as they get there - missing surfaces don’t draw anything.

Minimal synchronization to prevent surfaces from dying before they’re used

​

34 of 37

Combine with final rendering

Now

SurfaceAggregator

GLRenderer

OverlayProcessor

Future

CompositorFrame

CompositorFrame

SurfaceAggregator

GLRenderer

Call into

35 of 37

CompositorFrame diffing

IPC of update to entire frame is expensive - attempt to diff it

May aggregation can process only diff sometimes?

Maybe a better format that allows finding RenderPassId->RenderPass mappings

36 of 37

Headless displays

Used only for consistent CopyOutputRequests. Looks like a real display.

Doesn’t require artificial references

Tries to piggyback off of other compatible display (software vs. GPU)

37 of 37

Questions?