1 of 11

W3C

Media & Entertainment IG

Monthly Meeting

1 February 2022

2 of 11

MEIG Meeting, 1 February 2022

Logistics and Agenda

Date and time�1 February, 15:00 UTC

IRChttps://irc.w3.org/?channels=#me

IRC Guidehttps://www.w3.org/wiki/IRC

Code of Conduct https://www.w3.org/Consortium/cepc/

Agenda

  • DataCue and SEI event messages
  • Picture in Picture API for arbitrary content
  • Media Source Extensions V2 origin trials
  • TPAC 2021 outcomes and next step planning
    • Web application UI performance
  • AOB
  • Next meeting

3 of 11

DataCue and video SEI event messages

  • DataCue
    • API based on TextTrackCue for arbitrary timed metadata
    • Proposed mapping for DASH emsg events
  • H.264 / H.265 SEI message support
    • Open question: Integrate with DataCue proposal or separate API?
    • WebCodecs integration (see WebCodecs#198)
  • Help wanted
    • Input on use cases, demonstrate wider need
    • Editor to develop the proposals
  • Next meeting: Monday 21 February at 16:00 UTC / 08:00 PST / 01:00 JST (Tuesday)

4 of 11

Picture in Picture API for arbitrary content

  • Picture in Picture API (spec) currently works with video elements
  • A proposed v2 feature is to enable Picture in Picture for arbitrary content (see issue 113)
  • Proposal: https://discourse.wicg.io/t/proposal-document-picture-in-picture/5736
  • Explainer: https://github.com/steimelchrome/document-pip-explainer/blob/main/explainer.md
    • “A new method window.requestPictureInPictureWindow() opens a picture-in-picture (always-on-top) window with a blank document that can be populated with arbitrary HTML elements instead of only a single HTMLVideoElement”
  • Contribute use cases:
    • Video with custom player controls and captions
    • Others?

5 of 11

Media Source Extensions v2:�Chrome Origin Trials

“MSE requires applications to provide fragments of containerized media (such as fragmented MP4, WebM, or MP3) to be able to buffer and play that media. If the application already has the media in a parsed and structured form, it can cost extra latency and code complexity to package that media into a container and feed it to MSE.

As the web platform is evolving to give applications lower-level abstractions for efficiently encoding and decoding media via the WebCodecs API, MSE can use WebCodecs media structures to let applications more efficiently feed their media to MSE.

Use cases include simplifying and improving performance of transmuxing HLS-TS into fMP4 for buffering with MSE, and low-latency streaming with a seekable buffer.”

6 of 11

Media Source Extensions v2:�Chrome Origin Trials

“Enable Media Source Extensions (MSE) API usage from DedicatedWorker contexts to enable improved performance of buffering media for playback by an HTMLMediaElement on the main Window context. By creating a MediaSource object on a DedicatedWorker context, an application may then create an ObjectURL for it and postMessage that URL to the main thread for use in attaching to an HTMLMediaElement. The context that created the MediaSource object may then use it to buffer media.

When the main context is busy servicing other application logic, restricting MSE to the main Window context can significantly impair the user experience due to delayed media playback start or stalled playback”

7 of 11

Web application UI performance on TVs

  • Topic raised at our TPAC 2021 meeting - minutes
  • Increasing use of Web Assembly:
    • Amazon Prime Video - blog post
    • Disney+ Application Development Kit - blog post
    • BBC R&D experiments - blog post
  • Increasing use of Canvas based rendering, e.g. LightningJS
  • Next steps for MEIG?
    • Gather practical developer experience on HTML based TV app development - which scenarios or browser features cause performance issues
    • Create developer guidance for building performant applications
    • Propose APIs, e.g, for cache management (suggestion from TPAC 2021)
    • Develop performance test framework

8 of 11

W3C/SMPTE Media Production Workshop

  • Workshop held 8-19 November 2021 - see workshop page
  • 24 talks submitted:
    • Google, Mozilla, Adobe, Grass Valley, Bilibili, 5th Kind, Grabyo, Clipchamp (Microsoft), Bytedance, Disney Research, Soundtrap (Spotify)
  • Topics covered: WebCodecs, WebAssembly, Web Audio, media synchronization, file system access, media metadata, accessibility
  • A draft report is available
  • Next steps:
    • Survey presenters and participants
    • Possible new activity in MEIG to coordinate use cases, requirements, spec issues, feature proposals (if needed)

9 of 11

Next meeting

  • Tuesday 1 March 2022 at:
    • 07:00 US Pacific
    • 15:00 UK
    • 16:00 France
    • 23:00 China
    • 00:00 Japan (Wednesday)

10 of 11

Resources

11 of 11

Thank you!