1 of 52

The Interop Update

The Chrome Gap

Philip Jägenstedt, Robert Ma, Mike Taylor

April 10, 2019, BlinkOn 10

2 of 52

Interop as measured by web-platform-tests: improving

3 of 52

The WPT Update�since last year

4 of 52

wpt.fyi - more runs

More runs from heterogeneous platforms; new in the past year:

  • Mozilla Taskcluster
    • Chrome & Firefox on Linux
      • Dev/Nightly at every commit
      • Stable daily
      • Beta weekly

5 of 52

wpt.fyi - more runs

More runs from heterogeneous platforms; new in the past year:

  • Azure Pipelines
    • Edge on Windows
      • Stable only
    • Safari on macOS
      • Stable & Technology Preview

6 of 52

wpt.fyi - new features

  • Structured search
    • e.g. find tests that fail only Chrome
    • See “insights” tab for more examples
  • Screenshots
    • View screenshots of failing reftests

7 of 52

Other WPT stuff

  • web_tests/wpt_internal
  • WIP
    • WPT via CDP
    • wptrunner in Chromium
    • Triage tests on wpt.fyi

8 of 52

Call for action

  • Let’s triage & fix Chrome-only failures!
    • Structured search: filter Chrome-only failures
    • Reftest screenshots: investigate reftest failures
    • Triage metadata (coming!)

9 of 52

Call for action

  • Let’s write many more tests in WPT! Like, 90%?
    • wpt_internal: also enable quick, early iteration
    • wptrunner in Chromium (coming!): reduce the inconsistencies between WPT in Chromium and upstream

10 of 52

90%

Let’s do it

11 of 52

Long term trends

​

12 of 52

Data from browser-compat-data (BCD)

13 of 52

What is MDN’s browser-compat-data?

14 of 52

What is MDN’s browser-compat-data?

  • Hand curated over many years
  • 50-74% complete for Chrome, Edge, Firefox and Safari desktop
  • Less complete for mobile
  • Likely biased towards what’s most useful to document

15 of 52

Data from browser-compat-data (BCD)

16 of 52

Interop as measured by browser-compat-data subset: improving?

1/4 up 171

2/4 up 174

3/4 up 213

4/4 up 498

17 of 52

Data from BCD

CSS Typed OM

18 of 52

Data from BCD

Incomplete data?

19 of 52

Data from BCD

Incomplete data?

20 of 52

Does Confluence agree?

21 of 52

Data from Confluence

22 of 52

Data from Confluence

23 of 52

Does wpt.fyi agree? Yes

24 of 52

Quantifying the gap

25 of 52

Platform Gap Metric

Proposal: Create a measurement—weighted by feature usage (Chrome use counter data)—to track the difference in web platform support between Gecko and Blink.

For each feature that Chrome supports, whether we implement it or not, the absolute gap would be:

∑iIi * Pi

Where Ii is 0 if we support the feature, and 1 if we do not and Pi is the percentage of the feature served to Chrome.

26 of 52

Platform Gap Metric

Sample DOM gap measurement

Feature

Percentage of page loads in Chrome using the feature

Gap score

MouseEvent.toElement

42.04%

0.42049603

MouseEvent.fromElement

37.29%

0.37290945

Navigator.connection

35.31%

0

Document.all

22.20%

0

​

Total Gap Score:

0.79340548

27 of 52

Is it meaningful?

Can we gauge the impact (in terms of bug counts) and weight it accordingly?

28 of 52

29 of 52

CSS gap (gap_weight)

Note: The sizable decrease in the absolute gap is attributed to shipping support for -webkit-appearance in Firefox 63 (which appears in 53% of Chrome page loads).

Shipped -webkit-appearance

30 of 52

CSS gap (gap_size)

Scroll snapping?

31 of 52

CSS Gap

​

​

DOM Gap

Complete coverage for CSS properties is possible today, because Chrome has use counters for all of them.

💣 Problem: there’s no coverage for property values.

​

💣 Problem: Blink only has use counters for 5.6% of the DOM features that Gecko does not implement. We need to come up with a proxy for features used in sites served to Chrome.

​

32 of 52

Platform Gap Metric, revised

Proposal: Create a measurement—weighted by feature usage—to track the difference in web platform support between Firefox (Gecko) and Chrome (Blink).

For each feature that Chrome supports, whether we implement it or not, the absolute gap would be:

∑iIi * Pi

Where Ii is 0 if we support the feature, and 1 if we do not and Pi is our estimate of the popularity of the feature.

​

​

33 of 52

What’s in the gap?

34 of 52

Chrome-only APIs

  • BatteryManager
  • Bluetooth
  • CSS Typed OM
  • Feature Policy
  • Generic Sensors
  • Keyboard Lock
  • Keyboard Map
  • MIDI
  • Network Information API
  • Picture in Picture
  • Presentation API
  • Remote Playback
  • ResizeObserver
  • SharedArrayBuffer
  • Speech Recognition
  • USB
  • Web Locks
  • Worklets
  • Some webkit-prefixed APIs
  • … and many smaller things

35 of 52

No longer Chrome-only!

  • <details> element
  • navigator.languages
  • Notification API
  • Permissions API
  • requestIdleCallback
  • Scroll restoration
  • Shadow DOM
  • Streams

36 of 52

Edge-only

Safari-only

Firefox-only

  • AbstractRange
  • GamepadPose
  • WebRTC / ORTC APIs
  • Some VR-related things
  • Lots of ms-prefixed stuff

​

  • Parts of Web Audio
  • Parts of TextMetrics
  • ApplePay
  • webkitmouseforce* events
  • Many old and new prefixed APIs and CSS properties

​

  • HTTP Public Key Pinning
  • Many smaller things like RTCPeerConnection’s peerIdentity attribute
  • A variety of CSS properties/values
  • Some VR-related things (yep)
  • Lots of moz-prefixed stuff

37 of 52

So...

  • The gap is real
  • It’s growing (update: see extra slides)
  • Many big new things as opposed to many smaller/older things
  • Many more are coming (Fugu)

38 of 52

4 engines became 3

EdgeHTML

39 of 52

Data from BCD

40 of 52

What to do?

  • Keep investing in specs+tests
  • Better data (BCD)
  • Cross-vendor tools for prioritization (like wpt.fyi)
  • Understand Chrome-only set
  • Understand all-but-Safari set

WebKit is key?

41 of 52

Extra slides added after BlinkOn

42 of 52

Data from caniuse

43 of 52

Data from caniuse

44 of 52

Data from caniuse

45 of 52

Data from caniuse

46 of 52

So???

  • caniuse.com lists fewer features, but more complete version information for those
  • API size doesn’t affect weight
  • The gap is still there
  • However, not growing as dramatically

47 of 52

Backup slides

48 of 52

Data from BCD

49 of 52

Data from BCD

50 of 52

Confluence features/quirks

  • Reflects what a script can see by traversing the global object
  • Misses stuff with no “interface object” exposed
  • Treats standard and non-standard APIs the same

51 of 52

Data from Confluence

52 of 52

Data from Confluence