1 of 42

W3C ORTC Community Group Meeting #10

November 20, 2015 10:00AM-11:30AM PDT

Chair: Erik Lagerway (erik@hookflash.com)

2 of 42

W3C CG IPR Policy

  • See the Community License Agreement for details.
  • Goals are
    • Enable rapid spec development
    • Safe to implement via royalty-free commitments from participants+employers
    • Comfort for committers by limiting scope to OWN contributions
    • Transparency about who is making commitments
  • How it works in practice
    • Anyone can post to public-ortc@w3.org
    • CG members who have signed CLA can post to public-ortc-contrib
    • Editor should ensure that spec includes only “contributions”, CC-ing public-ortc-contrib makes that easier on the editor.

3 of 42

Welcome!

  • Welcome to the 10th meeting of the W3C ORTC Community Group!�Now 107 members!
  • During this meeting, we hope to:
    • Bring you up to date on the status of the ORTC specification
    • Make progress on some outstanding issues
    • Organize/plan for implementation feedback

4 of 42

About this Virtual Meeting

Information on the meeting:

  • Hangouts Meeting
    • Web Broadcast Link (view only)
    • Participatory Hangout Link (when meeting starts)
  • Link to Slides has been published on CG home page & ORTC.org
  • Scribe? IRC http://irc.w3.org/ Channel: ORTC
  • The meeting is being recorded.

​

​

5 of 42

W3C ORTC Community Group Basics

  • W3C ORTC CG website:
  • Public mailing list: public-ortc@w3.org
    • Join Here - link on the right hand side
    • Non-members can post to this list.
    • Non-member contributions are problematic.
  • Contributor’s mailing list: public-ortc-contrib@w3.org
    • Join Here - link on the right hand side
    • Members only, preferred list for contributions to the specification.

6 of 42

Associated Sites

  • ORTC developer website: http://ortc.org/
    • Editor’s drafts, pointers to github repos, etc.
  • ORTC API Issues List: https://github.com/openpeer/ortc/issues?state=open

​

7 of 42

Editor’s Draft Changes

  • November 2015 Editor’s draft:

​

Changes from the 22 June Editor’s Draft:

​

  • RTCWEB/WebRTC 1.0 compatibility (10)
  • Codec capabilities and settings (9)
  • Data channel clarifications (5)
  • RtpListener (1)
  • Miscellaneous (6)

8 of 42

RTCWEB/WebRTC 1.0 Compatibility

​

  • Issue 195: Add support for the WebRTC 1.0 certificate management API
  • Issue 218: Add certificate argument to the RTCDtlsTransport constructor
  • Issue 219: Add the "failed" state to RTCDtlsTransportState
  • Issue 220: Change getNominatedCandidatePair to getSelectedCandidatePair
  • Issue 222: Add support for WebRTC 1.0 RTCIceCredentialType
  • Issue 225: Change spelling from "iceservers" to "iceServers" for consistency with WebRTC 1.0
  • Issue 229: Change "outbound-rtp" to "outboundrtp" within the Statistics API
  • Issue 236: Add getContributingSources() method
  • Issue 249: Change double priority to RTCPriorityType priority
  • Issue 262: Substitute degradationPreference for framerateBias and move it to RTCRtpParameters

9 of 42

Adapter.js: ORTC as a WebRTC Shim

WebRTC Application

adapter.js

ORTC

Can adapter.js be modified to run WebRTC 1.0 examples on ORTC?

Yes: https://github.com/webrtc/adapter/pull/136/

10 of 42

Codec Capabilities and Settings

​

  • Issue 252: Add support for Opus capabilities and settings
  • Issue 254: Add payloadType attribute to RTCRtpRtxParameters
  • Issue 255: Clarify meaning of unset RTCRtpCodecCapability.clockRate
  • Issue 258: Update VP8 and H.264 capabilities and add VP8 and H.264 settings
  • Issue 259: Add RTX codec parameters
  • Issue 260: Add RED codec parameters
  • Issue 273: Change codec parameter and option names to camelCase
  • Issue 274: Add section on (Opus) codec options
  • Issue 275 and Issue 277: Clarify meaning of codec capabilities and options

11 of 42

Data Channel Issues

​

  • Issue 227: Added support for SCTP port numbers
  • Issue 231: Change maxPacketLifetime and maxRetransmits from unsigned short to unsigned long
  • Issue 233: Clarify DataChannel negotiation
  • Issue 240: Clarify behavior of RTCDataChannel.send()
  • Issue 242: Add text relating to RTCDataChannel exceptions and errors

12 of 42

RtpListener issue

  • Issue 243: RtpListener / mux id and RTP header extension issues

​

Scenario:

  • Developer creates an RtpListener object to receiver the RtpUnhandledEvent for incoming streams.
  • Developer has not yet created an RtpReceiver object.
  • Incoming RTP packets arrive with MID / RID header extensions.

​

How can the RtpListener fill in the muxId / rid values in the RtpUnhandledEvent dictionary if the RtpHeaderExensions are not yet known?

​

dictionary RTCRtpUnhandledEventInit : EventInit {� DOMString muxId; // <-- how is this value obtained??� DOMString rid; // <-- how is this value obtained??� payloadtype payloadType;� unsigned long ssrc;�};

13 of 42

RtpListener issue

Solution:

Add optional sequence<RTCRtpHeaderExtensionParameters> headerExtensions to RtpListener's constructor.

​

[Constructor(� RTCDtlsTransport transport,� optional sequence<RTCRtpHeaderExtensionParameters> headerExtensions // adding this results issue�)]�interface RTCRtpListener {� readonly attribute RTCDtlsTransport transport;� attribute EventHandler? onunhandledrtp;�};

​

14 of 42

Miscellaneous

​

  • Issue 223: Clarify behavior of createAssociatedGatherer()
  • Issue 237 and Issue 239: Fix to Examples 5 and 6
  • Issue 241 and Issue 248: Fix typos in Example 11
  • Issue 265: Add RID support to the unhandledrtp event
  • Issue 268: Update references
  • Issue 278: Clarified behavior in RTCRtpSender constructor and setTrack() when track.readyState is "ended"

15 of 42

For Discussion Today (Video Theme)

  • Basic video
    • Issue 258: VP8 and H.264 capabilities and settings
    • Issue 249 and Issue 262: RTCPriorityType and RTCDegradationPreference
  • Simulcast & RIDs { FINALLY: true } :)
    • Issue 265: RID support in the unhandledrtp event
    • encodingId and RIDs
    • Simulcast scenarios (in WebRTC 1.0 and ORTC)

16 of 42

VP8 and H.264 Capabilities

  • VP8 capabilities:
  • ​
  • ​
  • H.264 capabilities:

Note: H.264 (RFC 6184) does not have a max-fr SDP parameter.

17 of 42

VP8 and H.264 Settings

  • VP8 settings:
  • ​
  • ​
  • H.264 settings:

18 of 42

RTCPriorityType and RTCDegradationPreference

  • In the 22 June Editor’s draft RTCRtpEncodingParameters looked like this:

dictionary RTCRtpEncodingParameters {� unsigned long ssrc;� payloadtype codecPayloadType;� RTCRtpFecParameters fec;� RTCRtpRtxParameters rtx;� double priority = 1.0;� unsigned long maxBitrate;� double minQuality = 0;� double framerateBias = 0.5;� double resolutionScale;� double framerateScale;� boolean active = true;� DOMString encodingId;� sequence<DOMString> dependencyEncodingIds;�};

19 of 42

RTCPriorityType and RTCDegradationPreference (cont’d)

  • WebRTC 1.0 has introduced RTCPriorityType and RTCDegradationPreference enums:

20 of 42

RTCPriorityType and RTCDegradationPreference (cont’d)

  • Problem: Is it possible to have a distinct degradationPreference for each layer (in simulcast or SVC)?
  • Does it make sense to say:
    • “Where congestion is experienced, reduce resolution in extension layers, but not in the base layer?”
      • In temporal scalability, can’t do this because temporal layers have the same resolution; if lower framerate is desired, drop an extension layer.
      • In both simulcast and SVC, conflict with framerateScale.
  • Conclusion: degradationPreference is an RtpSender property.
  • WebRTC 1.0 PR: https://github.com/w3c/webrtc-pc/pull/397

21 of 42

RTCPriorityType and RTCDegradationPreference (cont’d)

  • With the changes, RTCRtpEncodingParameters and RTCRtpParameters look like this:

dictionary RTCRtpEncodingParameters {� unsigned long ssrc;� payloadtype codecPayloadType;� RTCRtpFecParameters fec;� RTCRtpRtxParameters rtx;� RTCPriorityType priority;

unsigned long maxBitrate;� double minQuality = 0;� double resolutionScale;� double framerateScale;� boolean active = true;� DOMString encodingId;� sequence<DOMString> dependencyEncodingIds;�};

dictionary RTCRtpParameters {� DOMString muxId = "";� sequence<RTCRtpCodecParameters> codecs;� sequence<RTCRtpHeaderExtensionParameters> headerExtensions;� sequence<RTCRtpEncodingParameters> encodings;� RTCRtcpParameters rtcp;� RTCDegradationPreference degradationPreference = "balanced";�};

22 of 42

things go

better

with

RIDS!

For a refreshing simulcast...

RID

Put a RID on it!

23 of 42

Purpose of RID

MID [aka "muxId"] identifies the RtpSender / RtpReceiver pairing by tagging SSRCs with a unique identifier [see MID].

​

An RTP packet tagged with MID will arrive at the correct RtpReceiver...

​

BUT how does the RtpReceiver know...

  • which SSRCs identify which simulcast stream?
  • which RTX/FEC packets belong to which simulcast stream?
  • which SVC layer goes to which simulcast stream?
  • where to route an RTX/FEC packets after SSRC changes [see Issue #246]?

​

Proposal: Source RTP Stream Identifier (“RID”)

[see https://tools.ietf.org/html/draft-pthatcher-mmusic-rid-02]

​

24 of 42

RID Support in unhandledrtp events

dictionary RTCRtpUnhandledEventInit : EventInit {� DOMString muxId;� DOMString rid; // <-- added rid to event dictionary� payloadtype payloadType;� unsigned long ssrc;�};

​

​

Proposal: Event the RID value to the application

​

The developer can use this information to handle the different simulcast streams appropriately.

25 of 42

RID Handling

dictionary RTCRtpEncodingParameters {

//... other stream or layer parameters ...� DOMString encodingId;� sequence<DOMString> dependencyEncodingIds;�};

​

Rules for handling RID inside RtpEncodingParameters:

​

  • If dependencyEncodingIds is empty:��RID = encodingId��NOTE: This encoding is known as the the "base" simulcast (or normal) encoding.�
  • If dependencyEncodingIds has data:��RID = encodingId from related "base" RtpEncodingParameters.

26 of 42

Does Encoding Parameters need a RID attribute?

Not really… but it could have one:

​

dictionary RTCRtpEncodingParameters {

//... other stream or layer parameters …

� DOMString rid; // <-- do we really need this???

� DOMString encodingId;� sequence<DOMString> dependencyEncodingIds;�};

​

The rules for encodingId already support RID without the need to have a redundant identifier in the RtpEncodingParameters.

​

27 of 42

Simulcast Scenarios

  1. Screen share with mobile device and a PC.
    • Mobile device has smaller screen size as well as lower maximum framerate capability.
    • Mobile device screen and PC screen might not be an exact ratio of one other.
    • In a screen share, frames/second can vary. PC and mobile framerate do not have to maintain a fixed ratio (unlike temporal scalability), as long as framerate is within decoder capability.

28 of 42

What do we need in ORTC API?

  • Currently in ORTC API
    • Codec settings (e.g. maxFr/maxFs) and resolutionScale/framerateScale
    • maxFr is the total maximum framerate for *all* layers (only applicable to VP8/VP9, not H.264)
  • Should parameters from draft-pthatcher-mmusic-rid be added to RTCRtpEncodingParameters?
    • max-width, max-height, max-fps

​

29 of 42

Question for Community Group:

Are there any comments regarding known deficiencies of the ORTC API at this time?

​

Good time to speak before implementation is too far along!

30 of 42

Thank you

Special thanks to:

Bernard Aboba - Microsoft

Michael Champion - Microsoft

Justin Uberti - Google

Peter Thatcher - Google

Robin Raymond - Hookflash

Erik Lagerway - Hookflash

​

​

31 of 42

For More Information

ORTC Community Group

http://www.w3.org/community/ortc/

​

ORTC Developers & API Drafts

http://ortc.org

32 of 42

Further Background...

​

The following slides are details on implementation statuses and will not be covered in Community Group meeting.

​

33 of 42

Organization / Call for implementation feedback

Mobile C++ ORTC implementation:

https://github.com/openpeer/ortc-lib-sdk

​

ORTC JS "shims" (i.e. downshim and upshim to / from WebRTC 1.0)

https://github.com/openpeer/ortc-js-shim (vacant repo)

​

ORTC specification:

https://github.com/openpeer/ortc

​

ORTC Node JS implementations:

https://github.com/openpeer/ortc-node

​

Browser Implementations:

Requested at this time (status.modern.ie lists ORTC as "In Development")

34 of 42

ORTC Update

  • WebRTC and ORTC specs significantly converged (objects)
  • WebRTC Working Group and ORTC Community Group converging (charter, editors)
  • Edge ORTC available in Windows 10 November update.
  • PeerConnection implementation in JS already working
  • Google has begun work on implementing WebRTC objects and ORTC for Chrome and mobile apps
    • Basic RtpSender/RtpReceiver already available for mobile apps at webrtc.org.
    • More to come in Q4 and 2016.

35 of 42

What is in Microsoft Edge ORTC?

36 of 42

What is in Edge ORTC (cont’d)?

  • Audio codecs: G.711, G.722, Opus, SILK, CN, DTMF, RED/external FEC
  • Video codecs: H.264UC + ULPFECUC (supports simulcast + SVC)
    • On the roadmap: interoperable H.264
  • Statistics: getStats() + msGetStats() supplemental call metrics
  • Audio/Video multiplexing (BUNDLE), RTP/RTCP mux
  • IPv4 and IPv6
  • Limitations:
    • DTLS 1.0 (1.2 on the roadmap)
    • No data channel
    • RTP/RTCP mux required with DtlsTransport (non-mux supported for SRTP/SDES)
    • No forking
    • Limited support for ICE aggressive nomination.

​

37 of 42

Edge ORTC Test Drive

https://dev.modern.ie/testdrive/

38 of 42

For More Information

39 of 42

What is ORTC-lib?

  • Open Source real world usable ORTC reference implementation
  • C++ API implementation
  • Focus is on mobile
  • Wrapped in Objective-C, CX (WinRT), Java
  • Applications can be written for WinRT, iOS, Mac, Android
  • Applications can be written in C++, Obj-C, C#, Java
  • Uses lower level WebRTC media components in core
  • Main API implementation done by Hookflash
  • Ready by end of year 2015

40 of 42

What is in ORTC-lib?

  • Audio codecs: Opus, G.711, G722, Isac, CN, DTMF, RED, RTX, FEC
  • Video codecs: VP8, VP9, H.264, RED, RTX, FEC
  • Simulcast support
  • Forking support
  • ICE:
    • Aggressive and regular nomination
    • Continuous nomination
    • Extended ICE gather options
    • Controllable ICE candidate search order (faster setup)
  • IPv6 support
  • Limitations:
    • No SVC (at this time)

41 of 42

What is in ORTC-lib?

Data Channel

Sctp Transport

RtpSender

Track

Track

RtpSender

DtlsTransport

IceTransport

IceTransport

DtlsTransport

RtpReceiver

Internet

Native C++ application

C# application

WinRT CX API wrapper

WinJS application*

Objective-C API wrapper

Java API wrapper

iOS application

Android application

Mac application

IceGatherer

IceGatherer

IceTransportController

Data Channel

Sctp Transport

Sdes Transport

Sdes Transport

Track

RtpReceiver

Track

Sctp Transport

Data Channel

Sctp Transport

Data Channel

IceTransportController

Sending Application

Receiving Application

RtpListener

* not officially supported

42 of 42

Learnings from ORTC Lib

No major red flags in the object model.

While minor issues in the specification were found, the object model appears to be sound.

​

Some issues found were related to:

  • SCTP ports
  • DTLS certificate creation / management
  • RtpListener header extensions needed before RtpReceiver attached
  • ICE gatherer pruning when forking is a challenge
  • General object lifecycle race conditions (e.g. getOrCreateTransport)