1 of 9

Improving Server and Client-Side Algorithms for Adaptive Streaming of Non-Immersive and Immersive Media

MMSys’21 Doctoral Symposium

Mehmet N. Akcay

2 of 9

Research Items

Low-latency ABR for live content (LoL+)
Evaluation of Apple's LL-HLS
Content-aware playback speed control
Common Media Client Data (CMCD): initial findings
Adaptive streaming for content-aware-encoded videos
Adaptive viewport margins for immersive media

2

In this research, new methods are proposed for both client and server side along with enhanced Adaptive Bitrate algorithms that are applicable to both non-immersive and immersive content.

In live streaming especially when the content is not ready yet, traditional ABRs are having difficulty to pick higher representations. We have designed an algorithm that solves that problem using CMAF chunks which we call LoL+.

On the other hand, recently Apple has made an enhancement for low latency called LL-HLS protocol and we made an evaluation of this new protocol and compared it with Dash.js equivalent representation.

During video playback, controlling of playback speed also plays an important role to catch up with the target latency. With our new playback controller module, we also check the calculated event density value to control the playback speed smoothly and in a content-aware fashion.

In September 2020, the Consumer Technology Association (CTA) published a new specification called CTA-5004 that allows sharing information from the clients to the content delivery servers. We showed that better user experiences can be achieved that way in our Common Media Client Data Initial Findings paper.

In our Adaptive Streaming for content-aware-encoded videos paper, we have focused on traditional ABR problems on variable bitrate encoded contents. We developed a new abr to deal with video stall problems while playing VBR contents.

And lastly streaming of immersive video is also being studied within this research. The latency plays an important role in immersive video as well, and it is more complicated since the required bandwidths are much higher compared to 2d HD contents. In order to solve that problem we are still working on ABR enhancements.

3 of 9

Low-Latency ABR for Live Content (LoL+)

3

4 of 9

Evaluation of Apple’s LL-HLS

4

AVPlayer

LL-HLS Beta v64

2 Mbps, 30 f/s

Segments: 2s

Parts: 34ms

HTTP/2

w/ LL-HLS support

In 60 seconds:

- 30 requests for 30 segments

~15 MB media download

In 60 seconds:

- 1800 requests for parts

~15 MB media download

- 160 requests for playlist updates

~150 KB non-media download

dash.js v3.1.1

2 Mbps, 30 f/s

Segments: 2s

Chunks: 34ms

HTTP/1.1

CTE

5 of 9

Content-Aware Playback Speed Control

5

6 of 9

Common Media Client Data (CMCD): Initial Findings

Avg. BR: Average bitrate across all clients (Mbps)
Min. BR: Average bitrate for the client that consumed the lowest average bitrate (Mbps)
Avg. RD: Average total rebuffering duration across all clients (s)
Max. RD: Total rebuffering duration for the client that suffered from the longest rebuffering duration (s)
Avg. RC: Average rebuffering count across all clients
Avg. SC: Average bitrate switching count across all clients

6

7 of 9

Adaptive Streaming for Content-Aware-Encoded Videos

7

Bandwidth Profile	ABR Rule	TD (MB)	TRD (s)	LRC	SRC	HD (%)
Cascade	Rate-based	184.00	4.92	6	11	88.33
	Dynamic	165.88	3.17	10	8	80.67
	SARA-Basic	178.56	1.26	3	5	65.33
	SARA-RLS	196.10	0.40	1	1	65.00
	SARA-MPC	198.06	0.80	2	1	66.67
Twitch	Rate-based	180.88	2.69	6	8	61.67
	Dynamic	186.57	2.54	10	6	68.00
	SARA-Basic	231.48	1.40	3	7	74.00
	SARA-RLS	199.01	1.00	3	4	55.00
	SARA-MPC	203.62	1.02	4	5	51.33
LTE	Rate-based	188.78	1.90	5	12	87.00
	Dynamic	130.19	1.30	6	7	53.00
	SARA-Basic	140.38	1.27	5	6	40.67
	SARA-RLS	146.10	1.00	3	7	43.67
	SARA-MPC	161.47	0.95	4	5	62.33

With our experiment dash.js standard rate-based and dynamic ABR experience rebufferings and their HD performance drops whereas SARA flavors increase their HD shares

8 of 9

Adaptive Viewport Margins for Immersive Media (In Progress)

8

9 of 9

List of Publications and Source Codes

ACM MMSys’20: When they go high, we go low: low-latency live streaming in dash.js with LoL (https://dl.acm.org/doi/abs/10.1145/3339825.3397043)
IEEE Trans. Multimedia: Catching the moment with LoL+ in Twitch-like low-latency live streaming platforms (https://ieeexplore.ieee.org/document/9429986)

Source: https://github.com/NUStreaming/LoL-plus

IEEE MMSP’20: Evaluating the performance of Apple's low-latency HLS (https://ieeexplore.ieee.org/document/9287117)
ACM MMSys’21: Content-aware playback speed control for low-latency live streaming of sports (https://doi.org/10.1145/3458305.3478437)

Demo url: http://streaming.university/demo/mmsys21-capsc/

ACM NOSSDAV’21: Common media client data (CMCD): initial findings (https://dl.acm.org/doi/abs/10.1145/3458306.3461444)

Source: https://github.com/NUStreaming/CMCD-DASH

IBC’21: Adaptive streaming of content‐aware‐encoded videos in dash.js

Demo url: http://streaming.university/demo/ibc21-sara/

9