1 of 13

Media Timed Events Task Force Update

Chris Needham, W3C TPAC, 16 September 2019

chris.needham@bbc.co.uk

2 of 13

Topics

  • In-band timed metadata and timed event support
    • MPEG-DASH emsg events
    • HLS timed metadata (e.g., ID3 tags)
  • Out-of band timed metadata
    • MPEG-DASH MPD events
    • Other applications, such as WebVMT (Video Map Tracks)
  • Improving synchronisation of DOM events triggered on the media timeline
    • TextTrackCue onenter and onexit
  • MPEG Carriage of Web Resources in ISO BMFF

3 of 13

History

  • Jan 2018: Giri Mandayam (Qualcomm) presented work at ATSC and MPEG on eventing - Minutes
  • Feb 2018: Cyril Concolato (Netflix) presented MPEG work on Carriage of Web Resources in ISO-BMFF - Minutes
  • Feb 2018: Media Timed Events Task Force started
  • Jan 2019: WICG DataCue repository created
  • May 2019: Use cases and Requirements v1 document published
  • Aug 2019: Discussions with implementers about DataCue

​

4 of 13

Use cases for timed metadata and in-band events

  • MPEG-DASH specific use cases
    • Player event signalling (DASH manifest update / refresh)
    • Media consumption metrics (DASH callback events)
  • Timed metadata for media streams
    • ID3 tags: Title, artist, image URLs
  • Ad insertion cues (SCTE35, SCTE214-1, -2, -3)
  • Application generated cues
    • Example use case: WebVMT Video Map Tracks�(Collaboration with W3C Spatial Data on the Web IG)

​

5 of 13

Recommendations

  • Allow web application to subscribe to event streams by event type
  • Allow web applications to create timed event or timed metadata cues, including: start time, end time, data payload
  • DOM event triggering:
    • When cues are parsed from the media container by the UA
    • When the current playback position reaches the cue start/end on the media timeline
  • Support for MPEG-DASH emsg and MPD events
  • Allow cues with unknown end time
  • Improve synchronisation (within 20 milliseconds on media timeline)

​

6 of 13

Current status

  • Task Force Use cases and Requirements: almost complete
  • WICG DataCue explainer: in progress
  • API specification: not started

7 of 13

MPEG Carriage of Web Resources in ISO-BMFF Containers

  • May 2018: Input received from MPEG via liaison (member-only link)
  • May 2018: TAG input requested: (GitHub issue)
  • Since then: Under development at MPEG as ISO/IEC FDIS 23001-15
  • Next: Continue discussion within Media & Entertainment IG

8 of 13

Browser support for DataCue

Edge (pre-Chromium)

HTML 5.1 DataCue

attribute ArrayBuffer data;

Chrome

No support

Safari

Supported, for HLS timed metadata, extended to allow different data types. Additional properties:

attribute any value;

readonly attribute DOMString type;

Firefox

No support

HbbTV

HTML 5.1 (8 Oct 2015 ED) DataCue with native handling of player specific events (e.g., manifest refresh), one TextTrack per event type. HbbTV has native (type 1) DASH playback

9 of 13

Player support for DASH and HLS events

shaka.Player.EmsgEvent No internal handling of manifest refresh events. No HLS timed metadata support?

Internal handling of manifest refresh (urn:mpeg:dash:event:2012) and tracking callback (urn:mpeg:dash:event:callback:2015) events.

​

Other event types are surfaced to the web app

HLS timed metadata (ID3 in MPEG2-TS)

HLS timed metadata (ID3 in MPEG2-TS)

10 of 13

Next steps

  • Breakout session “DataCue API and time marches on in HTML”,�Wednesday at 11:00
  • Raise issues against WHATWG HTML to propose changes to time marches on
  • Input needed from media player implementers, DASH-IF to develop DataCue API proposal

11 of 13

References (1/2)

12 of 13

References (2/2)

13 of 13

Thank you!