Media & Entertainment IG at TPAC 2019

Chris Needham / chris.needham@bbc.co.uk

5 Nov 2019

Agenda

• Brief summary of media topics at TPAC 2019
• Outcomes and next steps for
• 360° video and captioning
• Bullet chatting
• Media applications of Web of Things
• Media timed events Task Force
• Frame accurate seeking, rendering
• Media production use cases
• M&E IG priorities for the next year

Media groups at TPAC 2019

• Media & Entertainment IG (minutes)
• Audio WG (minutes: day 1, day 2)
• Immersive Web WG (minutes)
• Second Screen WG / CG (minutes: day 1, day 2)
• Web & Networks IG (minutes)
• Color on the Web CG (minutes)
• Media related breakout sessions
• Media WG (minutes: day 1, day 2)
• Timed Text WG (minutes: day 1, day 2)

Media related breakout sessions

• DataCue and “time marches on” in HTML (minutes)
• Efficient audio/video processing (minutes)
• Bullet chatting (minutes)
• Introducing Audiobooks (minutes)
• Next Generation TextTrackCue (minutes)

• Web Transport (minutes)
• HTML 3D element and Native glTF (minutes)
• Web Codecs (minutes)
• Web GPU (minutes)
• Images on the Web (minutes)
• Standardizing 360 video (minutes)

360° Video

360° Video

• Previous IG discussions:
• Jan 2018: 360° Video (minutes)
• Feb 2019: Subtitles in 360° Video and VR Experiences on the Web (minutes)
• Main conclusion from TPAC:
• The need 360° video with adequate captioning support is recognised
• WebXR focusing on low level input and rendering, but inefficient for video
• No consensus to extend <video> element with projection data
• Continue discussion in Immersive Web CG (issue #55)
• For captioning, further analysis is needed: follow up in issue #40 and the new Immersive Captions CG

Bullet Chatting / Danmaku

Bullet chatting

• Previous IG discussions:
• January 2019: Minutes
• August 2019: Minutes
• Main conclusion from TPAC:
• Further gap analysis work to be done to identify new or changed APIs
• Standardisation of a delivery data format? Does TTML support all requirements?
• Bullet Chatting CG created, initial conference call on Nov 8 at 9am UTC
• Continue work in the M&E IG?

Web of Things

Media applications for Web of Things

• NHK demo of Web of Things for Hybridcast control API at TPAC
• Web of Things Interest Group are interested in developing media
• Next steps:
• Discussion within Web of Things IG
• Possible topic for next M&E IG call on December 3rd

Media Timed Events

Media Timed Events

• For in-band events, UA vs application level parsing, which event types should be handled where?
• For synchronisation accuracy, add a note to the HTML spec
• Add web platform tests for cue event triggering
• Complete the Interest Group Note
• Feedback: Make more specific recommendations, clarify scope
• Combine effort with Next Generation TextTrackCue and look at cue processing architecture

Media Timed Events:

Possible TextTrack architecture

Container Parsing�Demuxing

​

Media data

Presentation Buffer (Rendering Objects)

Synchronized Rendering

Buffer (e.g. movie fragments)

Web App

JS Parsing & Rendering Object Construction

JS Object

Metadata

​

same time

Reference: Presentation from Media WG / TTWG joint meeting

Media Timed Events: Next steps

• Help wanted!
• Input needed on DataCue API, Sourcing In-band Tracks spec, architecture, etc.
• Next steps
• Continue the Media Timed Events Task Force, next call Monday 18th November

Frame accurate seeking and rendering

Frame accurate seeking and rendering

• Seeking use cases
• Non-linear video editing in the browser
• Browser-controlled cloud based media editing
• Seamless stream switching for dynamic content insertion
• Rendering use cases
• Graphic overlays on video (e.g., player annotations in sports)
• Subtitle rendering synchronized with frame changes
• Synchronized playback of multiple media streams (possibly across multiple devices)
• See M&E IG issue #4, and Slides from TPAC
• Related TPAC breakout: Efficient audio/video processing (slides, minutes)

Media production use cases

Professional media workflows on the web

• Media production workflows are moving to the cloud
• Users expect to interact via web applications
• API gaps exist:
• Sample accurate playback
• High dynamic range and wide color gamut support
• Improved subtitle and caption support
• etc.
• Next steps: Should we catalog and address these gaps?
• Presentation from M&E IG meeting: Professional media workflows on the web

Professional media workflows on the web: Next steps

• Propose creating a Task Force to identify and document API gaps
• May lead to new API proposals or changes to existing APIs
• Who could be Task Force moderator?
• Who is interested to participate?

M&E IG priorities

Discussion

• What are your priorities for the IG for next 12 months?
• Input to Media Working Group specs
• Professional media workflows