1 of 58

Applying Multipath Principles to a Cleaner Transport Protocol: Multipath QUIC

Olivier Bonaventure (UCLouvain, Belgium)�Quentin De Coninck (UCLouvain, Belgium)

https://qdeconinck.github.io/ �10 August 2020

Multipath Transport Protocols’ SIGCOMM 2020 Tutorial

1

2 of 58

So We Have Middleboxes...

2

Access Control System

Network Admission Control

Centri Firewall

Protocol translator

NetRanger

Content Switching Module

SSL Terminator

Streamer

3 of 58

… And They Hinder (Multipath) TCP Extensions

3

http://blog.multipath-tcp.org/blog/html/2018/12/07/multipath_tcp_encounters_its_first_middlebox.html

How can we innovate in networks with middleboxes?

SYN + MP_CAPABLE

SYN

4 of 58

Limited TCP Extensibility

4

How can we get rid of such engineering effort?

TCP Header

Max 15

Max 40 bytes

MP_JOIN Option in SYN/ACK

Of SHA-256

5 of 58

Outline

​

  • The motivations behind (MP)QUIC
  • A brief overview of QUIC
  • Bringing Multipath to QUIC
  • Exploring Use Cases
  • Open challenges and opportunities

5

6 of 58

A Brief Overview of QUIC

  • Finalization of IETF QUIC v1 in progress

​

  • Providing TCP/TLS1.3 service atop UDP
    • (Nearly) everything is encrypted
    • Reliable, in-order, multistream data transfer

​

  • Frame-based protocol

​

  • Large companies are deploying both client and server sides

6

IP

TCP

TLS

HTTP/2

IP

UDP

QUIC

HTTP/3

TLS

H

F

F

F

Today, from 1:30 pm to 5:00 pm

​

Langley, Adam, et al. "The quic transport protocol: Design and internet-scale deployment." Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 2017.

​

7 of 58

QUIC Handshake

7

CLIENT HELLO (with QUIC Transport Parameters)

SERVER HELLO (with QUIC Transport Parameters)

Establish pre-shared keys + data exchange

CLIENT FINISHED

Client

Server

The PSK can then be used in subsequent connections to send data in 0-RTT

Unilateral declarations �(features, initial receiving window,...)

8 of 58

A QUIC (Short) Packet

8

Flags

Sequence of control/data frames

Destination Conn. ID

Packet Number

Does not depend on network 4-tuple

Monotonically increasing

Easier deployment

Encrypted Payload

http://www.iconarchive.com/show/mono-general-4-icons-by-custom-icon-design/eye-icon.html

Cleartext Header

9 of 58

A QUIC Packet

9

TCP

QUIC

https://www.meretmarine.com/fr/content/le-petrolier-francais-antarctica-entre-en-service

https://www.meretmarine.com/sites/default/files/styles/mem_846_article_content/public/new_objets_drupal/11210.jpg

10 of 58

A QUIC Packet

10

TCP

QUIC

Payload

Header

Header

Frame

Frame

Frame

Frame

Frame

Frame

Not bound to a given packet

Core QUIC messages

11 of 58

QUIC Is Frame-Based

11

Flags

Destination Conn. ID

Packet Number

Cleartext Header

Does not depend on network 4-tuple

Monotonically increasing

Easier deployment

Encrypted Payload

ACK

Stream 0

“GET /index”

Stream 4

“GET /image”

Control frame acknowledging packets

Data multiplexing over a single connection

Theoretically, up to 2^62 possible (encrypted) frames!

12 of 58

Monotonically increasing packet numbers

TCP sequence numbers ambiguity

12

0

10

20

30

40

10

ACK(50)

Was I right to retransmit 10 ?

13 of 58

Monotonically increasing packet numbers

With QUIC packet numbers

13

0

1

2

3

4

5

ACK(0->5)

I should not have retransmitted !

-> Update dupack threshold

Contains frames that were in packet 1

14 of 58

Monotonically increasing packet numbers

TCP RTT estimation

​

-> RTT samples MUST NOT be made using segments that were retransmitted (if timestamps are not used) [RFC6298]

14

0

What is the RTT ?

0

RTO

ACK(10)

15 of 58

Monotonically increasing packet numbers

QUIC RTT estimation

​

15

0

Not ambiguous anymore !

1

RTO

ACK(only 1)

16 of 58

Unidirectional Connection IDs

16

Not bound to a 4-tuple anymore

CID A

Content

CID B

Content

Want to change CID?

CID B

NEW CONNECTION ID (C)

CID C

Content

Let’s change now!

Connection IDs can change during the connection’s lifetime

Proposed by the packet’s receiver

17 of 58

Connection Migration

17

18 of 58

Connection Migration

18

19 of 58

Connection Migration

19

20 of 58

Connection Migration

20

21 of 58

Connection Migration

21

22 of 58

Connection Migration

22

23 of 58

Connection Migration

23

24 of 58

Connection Migration

Change the IP/port 4-tuple of a given QUIC connection

  • Can change Connection IDs in use (privacy concerns)
  • Need to ensure that new remote address is still the same host
  • After performing path validation, the connection can be re-used normally.
    • The congestion control must however be reset

24

25 of 58

Connection Migration

25

Data exchange

WiFi becomes out-of-scope

Reduced data exchange

New path has been validated

Data exchange

{

Path validation

CID A

NEW CONNECTION ID (D)

CID B

NEW CONNECTION ID (C)

Propose alternative destination CID to the peer

CID C

PING

CID D

PATH CHALLENGE (X)

CID C

PATH RESPONSE (X)

26 of 58

Simple QUIC Data Transfer

26

F

CID A

PN = 4

STREAM(id=4, off=0): “Some data in my long frame”

F

CID B

PN = 8

ACK(4)

MAX_DATA(for stream=4): 1024

F

CID A

PN = 5

STREAM(id=4,off=26):“.”

ACK(8)

STREAM(id=6,off=0):“J”

F

CID B

PN = 9

ACK(5)

STREAM(id=4,off=0):“OK”

Authorize offset of stream 4 up to 1024

27 of 58

QUIC and Packet Losses

27

F

CID A

PN = 6

STREAM(id=4,off=26):“.”

STREAM(id=6,off=0):“J”

F

CID B

PN = 9

ACK(Largest=6, Block=->4)

F

CID A

PN = 5

STREAM(id=4,off=26):“.”

ACK(8)

STREAM(id=6,off=0):“J”

28 of 58

Outline

​

  • The motivations behind (MP)QUIC
  • A brief overview of QUIC
  • Bringing Multipath to QUIC
  • Exploring Use Cases
  • Open challenges and opportunities

28

29 of 58

Revealing QUIC Unidirectional Flows (“Uniflows”)

29

From IPC1 to IPS1

From IPS1* to IPC1*

Probed flow IPC2 to IPS1 with Path Validation

Connection path = both flows

Flow client to server (CID A)

Flow server to client (CID B)

Flows’ 4-tuples on client and server are not necessarily the same (e.g., NAT)

Only one uniflow per direction can be used to send useful frames

30 of 58

What Do We Want?

Enable simultaneous usage of multiple network paths while

​

  • Working in all networks where regular QUIC works

30

Let’s advertise a target address!

Monitoring uniflows?

31 of 58

Design Goals

Enable simultaneous usage of multiple network paths while

​

  • Working in all networks where regular QUIC works
  • Handling network asymmetry
  • Keeping control on path management
  • Not introducing new privacy / security concerns

31

De Coninck, Q., & Bonaventure, O. (2020, March). Multipath Extensions for QUIC (MP-QUIC). IETF Internet draft deconinck-quic-multipath-04. https://datatracker.ietf.org/doc/draft-deconinck-quic-multipath/

​

32 of 58

Using Multiple Uniflows with MPQUIC

32

2 sending uniflows

3 receiving uniflows

3 sending uniflows

2 receiving uniflows

CID C - Uniflow ID 1

CID A - Uniflow ID 0

CID B - Uniflow ID 0

CID D - Uniflow ID 1

CID E - Uniflow ID 2

  • Internal ID for each uniflow in use
  • Each uniflow has its (set of) Connection ID(s)

33 of 58

Multipath QUIC

  • Managing multiple uniflows
    • How to create uniflows and keep control on their number?
  • Transporting data
    • How can we spread data over several uniflows?
  • Congestion control
    • How to control congestion over multiple network bottlenecks?
    • (Similar to Multipath TCP)

33

34 of 58

Negotiating the Multipath Extensions

During connection handshake, use a QUIC transport parameter

34

CLIENT HELLO (max_sending_uniflow_id = 1)

SERVER_HELLO (max_sending_uniflow_id = 2)

  • Put a limit on the number of desired sending uniflows
    • Limit number of peer’s receiving uniflows
  • Initial path has (implicit) uniflow ID 0
  • If one does not advertise the TP, extensions are disabled

Don’t want to send on more than 3 uniflows

2 sending uniflows are enough

35 of 58

Proposing Uniflows

Receiver-initiated process using MP NEW CONNECTION ID frame

35

MP NEW CID – Uniflow ID 1 – CID A

Receiving uniflow 1�ready

Sending uniflow 1 usable

  • Host determines the number of sending uniflows of its peer
    • = number of different Uniflow IDs sent in MP NEW CID frames
  • Migration of a given Uniflow still possible

36 of 58

Advertising Host Addresses

36

Start connection on IPv4 server address

ADD ADDRESS(Address ID 1, 0, IPv6 address)

  • Address is encrypted, no more IP address leak in cleartext (like Multipath TCP)!

​

  • Address ID events ordered by a sequence number to handle reordering

ADD ADDRESS(1, 0, IP1)

ADD ADDRESS(1, 2, IP2)

REMOVE ADDRESS(1, 1)

Time

Relies on ADD ADDRESS and REMOVE ADDRESS frames

I want to advertise my IPv6

This is the newest one

37 of 58

Establishing Sending Uniflows

37

  • Each host runs its own path manager (unlike Multipath TCP)
    • Client can link a uniflow to pair of local address / remote validated address
    • Server should first wait for client packets to learn client addresses
      • Then link uniflows to pair of local address / seen and validated client address

IP A

IP B

IP C

IP D

Full-mesh example

ADD ADDRESS (IP B)

ADD ADDRESS (IP D)

MP NEW CIDs

MP NEW CIDs

​

38 of 58

Sending Data on Multiple Uniflows

38

  • STREAM frame is self-contained
    • No need to define additional sequence number!
    • Packet independence: frames not constrained over a given uniflow!
  • Similar for MAX_DATA, MAX_STREAM_DATA,...
    • Frames are idempotent
  • Packet scheduler schedules all frames (not only data)

How can I handle data reordering?

STREAM (offset 0)

STREAM (offset 50)

Uniflow 0 - PN 12

Uniflow 1 - PN 19

Need to send data

https://www.iphoned.nl/apps/beste-formule-1-apps/

https://www.leroygauthier.be/fr/produit=tracteur-kubota-4x4-diesel-18cv-b1181-avec-roues-agr-av-4.50-10-ar.7-16&id=tracfb4df1

http://www.esa.int/ESA_Multimedia/Images/2011/10/Galileo_In-Orbit_Validation_satellite

https://new.siemens.com/global/en/company/stories/industry/industrial-5g-the-wireless-network-of-the-future.html

​

39 of 58

Acknowledging Packets Sent on Uniflows

  • One packet number space per uniflow
  • (MP)ACK frame indicates the receiving Uniflow ID it acknowledges
    • ACK frames acknowledge the Uniflow 0

39

CID A, PN 58 -- Frames...

Sending uniflow 1 - CID A

Receiving uniflow 1 - CID D

Receiving uniflow 1 - CID A

Sending uniflow 1 - CID D

CID D, PN 87 -- MP ACK(Uniflow 1, 58)

40 of 58

An Multipath QUIC Connection Example

40

A

4

MP NEW CID (UID 1, D)

B

3

MP NEW CID (UID 1, C)

ACK(4)

C

0

PATH CHALLENGE(X)

A

5

STREAM(id:0, off:0, “GET”)

ACK(3)

D

0

PATH RESPONSE(X)

PATH CHALLENGE(Y)

D

1

ACK(6)

STREAM(id:0, off:2, “me da”)

C

1

ACK(5)

MP ACK(UID 1, 1)

B

4

STR(id:0, off:0, “So”)

ACK(5)

MP ACK(UID 1, 0)

A

6

PATH RESP(Y)

ACK(4)

MP ACK(UID 1, 0)

B

5

STREAM(id:0, off:7, “ta”)

41 of 58

Multipath QUIC Implementations

41

  • Multipath plugin of PQUIC [SIGCOMM19]
    • Enable to have different multipath algorithms with different plugins
    • See https://pquic.org
  • Other efforts for Multipath QUIC are ongoing
  • Initial mp-quic implementation from [CoNEXT17]
    • https://github.com/qdeconinck/mp-quic
    • Old version of QUIC, has a different design on the wire

​

[SIGCOMM19] De Coninck, Quentin, et al. "Pluginizing quic." Proceedings of the ACM Special Interest Group on Data Communication. 2019. 59-74.

[CoNEXT17] De Coninck, Q., & Bonaventure, O. (2017, November). Multipath quic: Design and evaluation. In Proceedings of the 13th international conference on emerging networking experiments and technologies (pp. 160-166).

42 of 58

Outline

​

  • The motivations behind (MP)QUIC
  • A brief overview of QUIC
  • Bringing Multipath to QUIC
  • Exploring Use Cases
  • Open challenges and opportunities

42

43 of 58

Exploring Use Cases

43

  • Smartphones
  • Specific Applications
  • Multipath VPN
  • ATSSS

44 of 58

Smartphones

44

<

$

$

De Coninck, Quentin, and Olivier Bonaventure. "MultipathTester: Comparing MPTCP and MPQUIC in Mobile Environments." 2019 Network Traffic Measurement and Analysis Conference (TMA). IEEE, 2019.

45 of 58

Smartphones

45

Wi-Fi loss may not be abrupt, can be useful to duplicate traffic during handover

<

$

$

De Coninck, Quentin, and Olivier Bonaventure. "MultipathTester: Comparing MPTCP and MPQUIC in Mobile Environments." 2019 Network Traffic Measurement and Analysis Conference (TMA). IEEE, 2019.

46 of 58

Specific Applications

Don’t hesitate to share your use case!

46

47 of 58

Multipath VPN

47

V

V

QUIC Connection

Decrease TCP Bulk Transfer Time

https://datatracker.ietf.org/doc/draft-piraux-quic-tunnel-tcp/

https://datatracker.ietf.org/doc/draft-piraux-quic-tunnel/

De Coninck, Quentin, et al. "Pluginizing quic." Proceedings of the ACM Special Interest Group on Data Communication. 2019. 59-74.

48 of 58

Access Traffic Steering Switching and Splitting

48

ATSSS

5G

Similar to a Multipath VPN, but for 5G networks

3GPP (3rd Generation Partnership Project). "Technical Specification Group Services and System Aspects; System Architecture for the 5G System; Stage 2 (Release 16)", 2019, https://www.3gpp.org/ftp/Specs/archive/23_series/23.501/ .

Bonaventure, O. et al. “3GPP Access Traffic Steering Switching and Splitting (ATSSS) - Overview for IETF Participants”. IETF Internet draft draft-bonaventure-quic-atsss-overview-00. https://datatracker.ietf.org/doc/draft-bonaventure-quic-atsss-overview/ .

​

​

49 of 58

Outline

​

  • The motivations behind (MP)QUIC
  • A brief overview of QUIC
  • Bringing Multipath to QUIC
  • Exploring Use Cases
  • Open challenges and opportunities

49

50 of 58

Packet Scheduling with Multistream

Alexander Rabitsch, Per Hurtig, and Anna Brunstrom. 2018. A Stream-Aware Multipath QUIC Scheduler for Heterogeneous Paths. In Proceedings of the Workshop on the Evolution, Performance, and Interoperability of QUIC (EPIQ’18). Association for Computing Machinery, New York, NY, USA, 29–35. DOI:https://doi.org/10.1145/3284850.3284855

Shi, X., Wang, L., Zhang, F., & Liu, Z. (2019, December). Fstream: Flexible stream scheduling and prioritizing in multipath-quic. In 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS) (pp. 921-924). IEEE.

Wang, J., Gao, Y., & Xu, C. (2019, August). A multipath QUIC scheduler for mobile HTTP/2. In Proceedings of the 3rd Asia-Pacific Workshop on Networking 2019 (pp. 43-49).

Mogensen, R. S., Markmoller, C., Madsen, T. K., Kolding, T., Pocovi, G., & Lauridsen, M. (2019, April). Selective Redundant MP-QUIC for 5G Mission Critical Wireless Applications. In 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring) (pp. 1-5). IEEE.

50

51 of 58

Coupled Congestion Control, Multiple Streams

51

52 of 58

Multipath QUIC Design

De Coninck, Q., & Bonaventure, O. (2017, November). Multipath quic: Design and evaluation. In Proceedings of the 13th international conference on emerging networking experiments and technologies (pp. 160-166).

Viernickel, T., Froemmgen, A., Rizk, A., Koldehofe, B., & Steinmetz, R. (2018, May). Multipath QUIC: A deployable multipath transport protocol. In 2018 IEEE International Conference on Communications (ICC) (pp. 1-7). IEEE.

De Coninck, Q., & Bonaventure, O. (2020, March). Multipath Extensions for QUIC (MP-QUIC). IETF Internet draft deconinck-quic-multipath-04. https://datatracker.ietf.org/doc/draft-deconinck-quic-multipath/

De Coninck, Q. (2020, March). Flexible Multipath Transport Protocols. Ph.D. Thesis. https://inl.info.ucl.ac.be/publications/flexible-multipath-transport-protocols.html

​

52

53 of 58

Conclusion

  • QUIC is clean enough to easily enable multipath usage
    • Everything encrypted, no more middlebox interference
    • Multipath goes further than connection migration
    • Easier to deploy
  • Next steps?
    • Getting Multipath QUIC standardized
    • Improving Multipath QUIC performance
      • Scheduling (including multistreaming)
      • Congestion control

53

54 of 58

Backup Slides

54

55 of 58

The different types of frames

  • STREAM frame
    • Contains application data
  • ACK frame
    • Acknowledges packets
  • CRYPTO frame
    • Exchange cryptographic handshake messages (TLS record stream)
  • PATH_CHALLENGE / PATH_RESPONSE frames
    • Used for path validation

55

56 of 58

The different types of frames

  • MAX_DATA / MAX_STREAM_DATA frames
    • Similar to the TCP receive window at connection/stream level

56

200

300

700

800

STREAM(offset=200, len=100)

700

1300

ACK, MAX_STREAM_DATA(1300)

Receive buffer

Missing data

Data received but cannot process

Free slot

Absolute offset

57 of 58

The different types of frames

  • NEW_CONNECTION_ID frame
    • Propose new CID to use for the connection
  • MAX_STREAMS frame
    • Limit the number of streams IDs to use
  • PING frame
    • Check that the peer is still alive
  • CONNECTION_CLOSE frame
    • Close the connection and provide the error cause
  • … and many more!

57

58 of 58

Why Do We Need Multipath Extensions?

58

Is current QUIC sufficient to use multiple network paths simultaneously?

​

What about using a single Connection ID with several network paths?

  • How to distinguish connection migration?
  • Linkability / privacy concerns?

​

What about using a single packet number space (with different CIDs)?

  • ACK complexity/size with heterogeneous uniflows?
  • Performance estimation for each uniflow?