1 of 12

A Hybrid Receiver-side Congestion Control Scheme for �Web Real-time Communication

Bo Wang | Yuan Zhang | Size Qian | Zipeng Pan | Yuhong Xie

Communication University of China

2 of 12

System Overview

2

Sender-side controller

Receiver-side controller

Network

Decisions

Gym

Bandwidth

estimator

Video encoder

Packet pacer

Receiver buffer

Video decoder

Display

RTCP feedback

RTP packet

Network Trace

Observations

3 of 12

Problem

For loss-based TCP:

Probing Mechanism: Filling and Draining

For WebRTC:

Smaller buffer while still expect considerable media quality

Most congestion control schemes designed for TCP are not suitable for WebRTC.

3

Why is it challenging to design a congestion control scheme for Web Real-time Communication?

Bandwidth limited

Bottleneck

bandwidth

Receiving rate

One way delay

Start point of

packet loss

Sending rate

Propagation delay

App limited

Optimal operating point

Buffer limited

Before talking about the model, let’s focus on the challenge of designing a congestion control scheme for Web Real-time Communication.

[click]

As shown in this figure, the objective of congestion control is to keep network links operating at the optimal point where inflight data equals to BDP. But measuring bottleneck bandwidth and propagation delay abide by the uncertainty principle.

[click]

Existing congestion control schemes for loss-based TCP are mainly based on filling and draining logics to address the problem.

[click]

But Web Real-time Communication apps have stricter requirement on latency than loss-based TCP network scenarios while still expect high-quality media delivery. It treats the excessively delayed packets as losses, which may bring severe damage to media quality. A typical tradeoff.

[click]

Therefore, most congestion schemes designed for TCP are not suitable for WebRTC.

4 of 12

Problem

Heuristic schemes:

robustness & weak adaptability

Learning-based schemes:

adaptability & weak robustness

4

Why is it challenging to design a congestion control scheme for Web Real-time Communication?

Heuristic + Learning-based = HRCC

5 of 12

HRCC Design

5

RTP Received

RTCP sender

Heuristic

RL-Agent

State Generator

Guide Interval

The framework includes two components:

First, [click] we implement a receiver-side version of GCC as the heuristic scheme.

In our experiments, we notice that the heuristic scheme decreases its estimation quickly to sudden bandwidth drop or high loss rate but increases estimation slowly to sudden bandwidth rising. Which means it tends to under-utilize link capacity when the network suddenly got better or just highly flutuating.

Based on this feature, we implement

[click] an RL-Agent that periodically tunes the heuristic scheme’s output estimate based on its observation on network link. By combining two methods, HRCC aims to achieve the robustness of classic heuristic schemes and adaptibility of learning-based schemes at the same time.

The workflow of HRCC is quite simple, with received RTP packets, the heuristic scheme takes the related statistics of the last decision-making interval as inputs. And generates bandwidth estimation and bandwidth usage state. Afterwards, the state generator takes states mentioned above to generate state input for the RL-Agent. At the end of every guide interval, the RL-Agent outputs a gain coefficient 𝜇 to tune bandwidth estimate.

6 of 12

Formulation

6

7 of 12

HRCC Design——Heuristic Scheme

7

trendline filter

overuse detector

AIMD controller

loss rate calculation

min

Loss-based bitrate controller

Delay-based bitrate controller

8 of 12

HRCC Design——RL-Agent

8

State	Description
	the receiving rate of the last 10 decision-making intervals
	the network delay measurements for the last 10 decision-making intervals
	the packet loss rate of the last 10 decision-making intervals
	the bandwidth estimation given by heuristic scheme for the last 10 decision-making intervals
	the most current bandwidth estimation given by HRCC that caused overuse for the last 10 decision-making intervals
	the time so far since the last overuse for the last 10 decision-making intervals

9 of 12

Evaluation

9

Fully Heuristic: the heuristic scheme adapted from GCC (HRCC without the RL-Agent)
Fully RL-based: a fully RL-based scheme whose DNN structure is similar to HRCC, it continuously outputs a gain coefficient between 0.5 and 2 to tune the previous bandwidth estimate

Training Set: 700 network traces ranges from [700kbps, 2.6Mbps], 200 of them specify [1%, 10%] random loss
Test Set: 100 network traces ranges from [500kbps, 7Mbps], 25 of them specifying [1%, 10%] random loss

Measurement	Bandwidth Utilization (%) ↑	Queueing Delay (ms) ↓			Packet Loss Rate (%) ↓				QoE ↑
Measurement	Bandwidth Utilization (%) ↑	Average	50th	95th	Packet Loss Rate (%) ↓				QoE ↑
HRCC	78.9	40.5	9.4	198.4	1.5	78.9	66.3	98.5	58.6
Fully Heuristic	67.9	17.2	3.1	93.3	1.4	67.9	72.6	98.6	57.7
Fully RL-based	75.7	263.2	226.2	527.5	7.4	75.7	54.4	92.6	53.8

Now I’m going to give the simulation test results.

[click]

To show the performance of HRCC, we train a fully RL-based bandwidth estimator with a structure similar to the RL-agent of HRCC, we remove the state information given by the heuristic scheme from state input. It outputs a gain coefficient between 0.5 and 2 to tune the previous bandwidth estimate.

[click]

We conduct the simulations under 100 network traces to compare the hybrid scheme with both of its individual components(in other words, the fully heuristic scheme and the fully RL-based scheme), each lasting 100 seconds. The average bandwidth of these traces ranges from 500kilobps to 7Megabps, while 25% of them specifying [1%, 10%] packet loss rate.

As provided in the table, HRCC achieves the highest bandwidth utilization with similar delay and loss performance compared with the fully heuristic scheme and outperforms the other 2 schemes on overall QoE.

10 of 12

Evaluation

10

more robust & higher bandwidth utilization

without random loss

with random loss

And now I’m going to give 3 specific test cases to take a glimpse of the performance of HRCC.

[click]

These figures show 2 cases of running simulations under different traces without random loss. As you can see in the left figure, in relatively stable conditions, the fully RL-based scheme performs well enough. However, in the right figure, the fully RL version lacks constraints on bandwidth estimate under the other traces that are completely different from the training data, the estimated bandwidth is totally off the trail and causes fatal fluctuations of queuing delay.

[click]

Furthermore, we evaluate the performance of the 3 schemes in bad network environments with random loss rates. As we can see, the fully heuristic scheme cannot reach the available bandwidth with its loss-based mechanism, while the fully RL-based scheme causes extremely high queuing delay though achieves better bandwidth utilization. In this case, HRCC reaches the best compromise.

By catching a glimpse of the 3 cases, we may see that HRCC realizes a [click] more robust performance than the fully RL-based scheme and achieves higher bandwidth utilization than the fully heuristic scheme.

11 of 12

Summary & Future Work

11

The proposed hybrid receiver-side congestion control scheme - HRCC consists of:

A robust heuristic scheme to continuously make bandwidth estimate;
An RL-agent to periodically tune the bandwidth estimate made by the heuristic scheme.

Simulation test run verified that HRCC outperforms the fully heuristic scheme and the fully RL-based scheme on overall performance.

In the future…

Prove & Improve: Interpret HRCC (generating decision tree)

Modification: Implement sender-side logic

As a summary, HRCC outperforms both of its individual components on bandwidth utilization and overall QoE.

[click]

In the following studies,

[click]First, There’s related research shows that by interpreting deep learning-based models using decision tree method, the control logic behind the DNN could be dug out. In our case, we may find out why HRCC in such way and perhaps improve it in a precise way. Besides, overweight model could bring dramatic decision delay, and that’s not tolerable for real-time communication. Transferring RL model into a decision tree may efficiently address the problem and make the model more applicable.

[click] Second, in actual application scenarios, there’re nonnegligible propagation delays for receiver-side estimation sending back to the sender. So, the feedback latency should be considered for deciding sender-side sending rate. We will work on the sender-side logic.

12 of 12

Thank you！

Bo Wang | Yuan Zhang | Size Qian | Zipeng Pan | Yuhong Xie

Communication University of China