1 of 15

Gemini: An Ensemble Framework for Bandwidth Estimation in Web Real-Time Communications

Tianrun Yin, Hongyu Wu, Runyu He, Shushu Yi, Dingwei Li

Adviser: Jiaqi Zheng

Department of Computer Science and Technology， Nanjing University�

2 of 15

Grand Challenge on�Bandwidth Estimation for Real-Time Communications�

Organized and sponsored by

3 of 15

Grand challenge

Goal: Design and implement a bandwidth estimator on the receiver side of a provided RTC system——AlphaRTC, a fork of Google's WebRTC codebase.
Given:

AlphaRTC and gym based on it
A limited number of network traces

Evaluation:

Perceptual video quality metric based on the VMAF model
Perceptual audio quality metric based on the DNSMOS model
Average frame drop rate
Average frame delay
Variance of frame delay
Network performance metrics (e.g., packet loss rate)

3

4 of 15

Web Real-time Communication widely used in APPs

4

Enterprise Communications

Health care

Social app

Education

Real-time video applications have never played a more critical role in our lives as they enable us to live and work remotely while staying connected with the rest of the world. In fact, Web Real-time Communication is widely used in areas such as enterprise communications, Social app, Health care , Education. It's used in everyday apps like Slack Messenger.

However, the rapid increase in the use of real-time video also poses an unprecedented challenge for consistently delivering high quality of experience (QoE) — such as high video and audio quality, low delay— to all users.

A pivotal/ˈpɪvətl/ algorithm to optimize the QoE for real-time video communications is bandwidth estimation. It runs on the endpoint of a real-time video application and aims at adapting the video bitrate dynamically to stay within the available network capacity.

5 of 15

Learning-based algorithms help improve congestion and bandwidth estimation

5

Remy

[SIGCOMM’13]

PCC

[NSDI’15]

Pensieve

[SIGCOMM’17]

PCC-vivace

[NSDI’18],

Indigo

[ATC’18]

Qflow

[MOBIHOC’19]

OnRL

[MOBICOM'20]

2013

2018

2015

2017

2019

2020

6 of 15

Rule-based algorithm can help learning-based one

6

Rule-based algorithm

Key idea: Predict bandwidth based on aggregated RTP feedback packet information
Strength:

Good convergence
Low overhead

Weakness:

Fail to achieve consistent high performance in complex and diverse network environment

Learning-based algorithm

Key idea: Predict bandwidth based on neural networks trained by huge amounts of data
Strength:

Strong adaptability to network

Weakness:

Difficult to guarantee convergence
High overhead
Not very robust

In our work, we consider that rule-based algorithm can help learning-based one.

Rule-based algorithm predict bandwidth based on aggregated RTP feedback packet information. These algorithms tend to elaborate state mechanisms and therefore have good convergence. Compared to learning-based algorithms, the overhead of state calculation is much lower. However, they fail to achieve consistent high performance in complex and diverse network environment.

Learning-based algorithm predict bandwidth based on neural networks trained by huge amounts of data. Deep neural networks tend to give it strong adaptability to network. However, training neural networks requires a lot of data and takes a lot of time and difficult to converge. Deep neural networks require matrix computation and bring high overhead. In fact, bandwidth prediction should not take up too much computing resources. At the same time, reinforcement learning is not so safe, and is not very robust in an untrained environment.

It is not difficult to find that the shortcomings of the learning-based approach can be remedied by rule-based algorithms. Based on this idea, we proposed our framework, Gemini.

7 of 15

GEMINI overview

7

GEMINI’s core idea: Take advantage of the DRL’s excellent performance for a given environment while keeping the DRL stable inherited from GCC.

The core idea of Gemini is that we can take advantage of the DRL’s excellent performance for a given environment while keeping the DRL stable inherited from GCC. In our framework, we mainly predict bandwidth through two modules: DRL part and GCC part. We used PPO algorithm for DRL training and refered to webRTC source code to implement the Python version of GCC algorithm. After receiving the RTP packet, we will predict the bandwidth through these two parts. And then through the overload detector, we can judge whether the current network environment is safe for DRL. If the environment is safe, we use the estimates from the DRL part as the result, otherwise, We mixed the results of the DRL and GCC parts, the exact ratio is determined by the trust window. We use the hybrid estimation as the result.

8 of 15

DRL part of GEMINI

State:

receiving rate (kb/s),𝑅
packet loss rate (%), L
average RTT (ms), D
Latest prediction(kb/s)
the difference between the current estimated bandwidth and the previous one (kb/s), Δ.

Action:

We use a log-based function to map the input rate of (0,20) Mbps to (0,1).

Reward function:

𝑅−1.5𝐷 −1.5𝐿−0.02∗ |Δ|

8

9 of 15

Compute bandwidth in real time

We use lightweight actor-critic neural network.

9

10 of 15

Robust Assurance

DRL is not safe because of its “black-box” neural network
Solution: Detect overload signal and combined with GCC results
Core issue: how to use the performance of DRL as much as possible in a secure environment, and avoid the unstable performance caused by repeated switching in an unsafe scenario.
Overload detector based on the delay gradients based mechanism and dynamic threshold updating
Trust window mechanism:

10

DRL							GCC

DRL							GCC

Pure DRL is not safe because of its “black-box” neural network. It may have unstable performance under the condition of inconsistent training environment and operational environment.

Our solution is that we detect the overload signal or the “unsafe” signal and combine the DRL result with the GCC result.

The core issue now becomes how to take advantage of the performance of DRL as well as avoid the unstable performance.

We use an overload detector, which is based on the delay gradients based mechanism and dynamically update the threshold. This design has similar implementations in both the sender version of GCC and OnRL .

In addition, we used the trust window mechanism to adjust the proportion of DRL and GCC results.

When the state is unsafe, we use a trust window control the hybrid estimation. If the latest m SIGNALs are safe, the trust window will be increased by one, otherwise, it will be decreased by one. By doing this, we can gradually use DRL data as the environment becomes safer. At the same time, when the environment is not safe, we can gradually shift the results towards GCC.

11 of 15

Generate more traces

A limited number of network traces cannot train DRL Model —— We generate more traces with similar sample. Some data cleaning is also done.

11

Origin data

Generated data

12 of 15

Implementation details

12

We develop a prototype of our algorithms based on gym and alpha-RTC developed by OpenNetLab.
We implement the DRL part of Gemini using actor-critic framework.
We implement the delay-based bandwidth prediction of GCC using Python based on WebRTC source code.
We generate data for a variety of environments，such as WIRED_35mbps, 4G_700kbps, WIRED_900kbs, WIRED_200kbps, 5G_12mbps, 4G_500kbps, 4G_3mbps and random data with gaussian distribution.

13 of 15

Evaluation

13

4G networks with 700 Kbps capacity

We verify the Gemini framework by conducting experiments in trace of multiple environmental samples.

14 of 15

Summary

Challenge Goal: Design and implement a bandwidth estimator on the receiver side of a provided RTC system——AlphaRTC.
GEMINI: We developed the Gemini framework by combining DRL and GCC algorithms.

DRL part —— We use lightweight actor-critic neural network.
GCC part —— We implemented a Python version of GCC’s delay-based part.
Robust Assurance

Beyond the challenge:

We plan to further improve the overload estimation mechanism in the future, and combine some methods of packet processing with traditional algorithms during DRL operation

14

15 of 15

15

Thanks

You can find our project in https://gitee.com/tyler-ytr/Gemini

My email is ytrpossible@gmail.com