1 of 19

Improving interop with web-platform-tests

Philip Jägenstedt <foolip@chromium.org>

TPAC 2017, November 8

​

​

These slides: bit.ly/tpac-wpt

2 of 19

3 of 19

4 of 19

This is not inevitable!

5 of 19

6 of 19

wpt.fyi

Running all tests on 4 browsers daily.

Will be more browsers, more often.

wpt.fyi is URL:

7 of 19

wpt.fyi

Will soon change the default view to focus on interoperability.

​

Mockup:

8 of 19

Test automation now much easier��web-platform-tests.org/writing-tests/testdriver.html

9 of 19

Test automation decision tree

  1. If the WebDriver API required to test this is small:
    1. Add it to WebDriver (main spec or extension spec)
    2. Wrap that API with testdriver.js
    3. Write tests!
  2. Otherwise, ask public-test-infra@w3.org

10 of 19

Part of the

standards process

11 of 19

Asking for tests for normative changes

Accelerometer, Ambient Light Sensor, Battery Status API, Beacon, mature CSS specs, Clipboard API and events, Compatibility, Console, Content Security Policy, Cooperative Scheduling of Background Tasks, DOM Parsing and Serialization, DOM, Encoding, Feature Policy, Fetch, File API, File and Directory Entries API, Fullscreen API, Gamepad, Generic Sensor API, Geometry Interfaces, Gyroscope, HTML Media Capture, HTML,High Resolution Time, Indexed Database API, Intersection Observer, Keyboard Lock, Long Tasks API, MIME Sniffing, Magnetometer, Media Queries, Navigation Timing, Notifications API, Orientation Sensor, Page Visibility, Paint Timing, Payment Request API, Performance Timeline, Pointer Events, Pointer Lock, Preload, Presentation API, Quirks Mode, Resize Observer, Resource Hints, Resource Timing, SVG, Selectors, Server Timing, Service Workers, Static Range, Storage, Streams, Touch Events, UI Events, URL, User Timing, Vibration API, Web Animations, Web Audio API, Web MIDI API, WebRTC, WebUSB API, WebVR, WebVTT, XMLHttpRequest

12 of 19

100+ specs are doing this

Pointer Events�+ HTML

WHATWG

CSSWG

Lots!

Web Performance WG

13 of 19

How to do it: just copy+paste this

From Gamepad’s CONTRIBUTING.md:

For normative changes, a corresponding web-platform-tests PR is highly appreciated. Typically, both PRs will be merged at the same time. Note that a test change that contradicts the spec should not be merged before the corresponding spec change. If testing is not practical, please explain why and if appropriate file a web-platform-tests issue to follow up later. Add the type:untestable or type:missing-coverage label as appropriate.

14 of 19

Next 3 steps of the master plan

  1. Learn which implementations fail those tests.
  2. File issues for those implementations.
  3. Track those issues over time, revisiting spec issues if necessary.

15 of 19

Part of the

implementer workflow

16 of 19

2-way sync

web-platform-tests is a separate repository on GitHub. 2-way sync means:

  • Importing the tests into browsers’ source code repos
  • Making changes locally and exporting them back
  • As automatic and frequent as possible

17 of 19

2-way sync

Gecko has had it for a long time

Chromium:

  • WPT changes imported within 12 hours
  • Chromium changes exported to WPT within 1 hour
  • Chromium contributions to WPT more than tripled

EdgeHTML and WebKit working on it

18 of 19

Come to our

breakout session!

19 of 19

Resources

Documentation:

Get in touch: