Media Timed Events / DataCue API

Chris Needham, 16 November 2020

Agenda

1. Add TextTrackCue end time representing end of media (HTML #5297)
2. Review feedback from TPAC meeting (minutes)
3. Mapping of emsg boxes to the media timeline (#26, MSE #189)
4. Feature detection for supported event types (#24)
5. AOB?

​

TextTrackCue end time

• HTML issue #5297 and pull request #5953
• Browser bugs:
• Safari: https://bugs.webkit.org/show_bug.cgi?id=218567
• Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=1145620
• Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1675317
• WebVTT pull request #493
• Next steps?

Feedback from TPAC meeting

• Meeting minutes from TPAC 15 October 2020 meeting
• Overall:
• Questions about the detail from MSE spec editors
• Capability detection
• Mapping emsg boxes to media timeline in MSE
• Browser parsing of inband event schemes
• No objections expressed
• Should timed metadata / event resources be fetched out of band?

​

Mapping emsg boxes to media timeline

• See DataCue issue #26 and MSE #189
• emsg as top-level boxes only?
• What needs to be specified wrt parsing of media segments in MSE, and interaction with setTimestampOffset()?
• Proposal
• Initially, draft in the DataCue API spec. Later, use this to update Sourcing Inband Media Resource Tracks from Media Containers into HTML, or (possibly) in MSE ISO BMFF Byte Stream Format, not in DataCue API spec (MTE #1)
• Help wanted! Moving this forward needs contributors

Feature detection

• Feature detection of DataCue API
• Capability detection for supported event types (#24)
• Which in-band events does a UA support?
• Which in-band event schemas does a UA support?
• Does the UA support native parsing of cue data?
• Proposal
• Initially, draft in the DataCue API spec. Later, use this to propose updates to Media Capabilities API
• Help wanted! Moving this forward needs contributors

Native vs application parsing of in-band cues (#16)

• Should the DataCue API expose emsg boxes as structured data, or as binary blobs for the web application to parse?
• Want to avoid requiring UAs to update to accommodate new event message schemas. Implies application-level parsing
• Is there a common set of schemas where UA parsing makes sense?