1 of 20

Ruiyang Zhu, Xiao Zhu, Anlan Zhang^†, Xumiao Zhang, Jiachen Sun, Feng Qian^†, Hang Qiu^‡, Z. Morley Mao, Myungjin Lee*

University of Michigan ^†University of Southern California

^‡University of California Riverside *Cisco Research

Boosting Collaborative Vehicular Perception on the Edge with Vehicle-to-Vehicle Communication

ACM SenSys’24, Hangzhou, China.

Presenter: Bo Chen, University of Illinois at Urbana-Champaign

2 of 20

Outline

Background and Motivation
System Design
Evaluation Results
Conclusion

2

3 of 20

Background: Collaborative Vehicular Perception

3

Sensors

Perception

LiDAR 3D Point Clouds,

2D Camera Images, etc.

Drivable space detection,

Object detection, etc.

Limited range and occlusion

Collaborative Vehicular Perception (CVP)

Exchange sensor data among CAVs to extend sensing capabilities

Extended Perception in occluded area

Connected and autonomous vehicles rely on various on-board sensors for the perception of the environment such as vehicles, pedestrians, and cyclists to make driving decisions. For example, the cameras capture 2D images while the LiDAR units output 3D point clouds.

However, there are limitations of on-board sensors. One important limitation is that sensors like LiDAR suffer from occlusion. Because of the linear propagation of light, they cannot "see through" non-transparent objects and can only provide line-of-sight information. In the middle figure, we can find that there are blindspot areas created by the occlusion of objects, which means the ego vehicle cannot get any information of those areas. Therefore, there could be false detections like missed objects, which increases road risks and affects driving efficiency for autonomous vehicles.

To overcome this limitation, collaborative vehicular perception, or CVP for short, has been proposed. The key idea of CVP is to let CAVs communicate about their sensor data, such as LiDAR point clouds in many cases. As shown in the example on the right, although one single CAV’s LiDAR has limited capability, LiDAR data from combining another CAV covers almost all corners of this intersection. �

4 of 20

Limitations of Existing V2V-only CVP

Lack of scalability for larger number of vehicles^{[1, 2, 3]}

AVR and Cooper^{[1, 2]} focus only on a 2 vehicles scenario
Creates additional overheads by sharing N-1 copies or use WiFi Broadcast mode^[3], which creates congestion in V2V network

WiFi broadcast does not have MAC layer ACKs (no congestion control by default)^[3]

4

[1] AVR: Augmented vehicular reality. MobiSys 2018.

[2] Chen, Qi, et al. "Cooper: Cooperative perception for connected autonomous vehicles based on 3d point clouds." IEEE ICDCS, 2019.

[3] CarSpeak: A Content-Centric Network for Autonomous Driving. SIGCOMM 2012.

(1) 2 vehicles

(2) >= 3 vehicles, N * (N-1) bandwidth sharing

Broadcast

(3) >= 3 vehicles, through wireless broadcast

5 of 20

Limitation of Existing V2I-only CVP

V2I (cellular) network conditions^[1] can vary temporally and spatially

Ideally, C-V2X communication expects fast and stable network performance
Different carriers have different performance coverages

[1] EMP. MobiCom 2021.

The other type of communication paradigm is to use vehicle-to-infrastructure/edge (V2I) communications to merge different vehicles’ view at an edge server for computation. While existing work demonstrate that V2I based solutions scale better in reducing latency by reducing the amount of data transfer and offloading computation to an edge server, its performance is highly dependent on the V2I links, or cellular network performance for most of the cases. By performing a measurement in a major U.S. city, we find that onboard cellular (uplink) connectivity is highly heterogeneous for different carriers and locations. Vehicles may experience very poor performance during urban driving such as 8 to 20 seconds of low uplink bandwidth less than 1 Mbps, which makes CVP application unapplicable in those poor V2I network conditions.

6 of 20

Need for a hybrid V2V+V2I architecture

Harbor: A Hybrid architecture for collaborative vehicular perception that adaptively uses V2V and V2I connectivity
Key idea: bridge V2I-disconnected vehicles (helpees) by strategically pairing them with V2I-connected vehicles (helpers) through V2V

6

Mode switch

ISP A

ISP C

ISP B

Pure V2I Mode

ISP A

V2V link

ISP C

V2V+V2I Mode

The limitations of single-mode communication methods like V2V and V2I motivate us to use a combination of both to improve the robustness and performance of CVP applications.

To this end, we proposed to use a hybrid V2V and V2I architecture to tackle the problem. Our key idea for the hybrid architecture is to bridge V2I disconnected vehicles (which we refer them as helpees) by strategically pairing them with V2I-connected vehicles (or helpers) through V2V connectivity.

[click] A motivating illustration would be like the following: a red vehicle experience very bad V2I performance or lost its V2I connection, [click] [click] it can then switch to use V2V communication and leverage another vehicle (in this case, the black vehicle) to help upload its data and continue performing the collaboration.

7 of 20

Outline

Background and Motivation
System Design
Evaluation Results
Conclusion

7

8 of 20

Harbor design

A hybrid architecture for collaborative vehicular sensing

Leverages both V2V and V2I network access
Flexibly manages V2V and V2I connections

Strategic helper assignment

Timely detection result delivery

8

Here is a design diagram of Harbor. As shown, there are two types of vehicles in Harbor: helper vehicles, or helpers, which have V2I connectivity, as well as helpee vehicles, or helpees, which don’t have V2I connectivity. Helpers directly upload sensor data to the server through their V2I links [click]. Helpees leverage V2V links to first send sensor data to helpers [click], which then forward helpees’ data to the server through V2I.

�[click] To assign helper vehicles to helpee vehicles for data forwarding, Harbor runs a helper assignment module on the server side, which leverages the state information of each vehicle by letting helpee vehicles broadcast different types of control messages through the V2V medium, which are eventually forwarded to the server through V2I links.

�On receiving the sensor data from vehicles, the server performs detection on the merged data and sends the results back to the vehicles. To ensure timely result delivery [click], Harbor employs deadline-aware view merging at the server side and MAC-layer message prioritization at the vehicle side.

�Next, I’ll describe these key design decisions of Harbor in more detail [click].

�

9 of 20

Strategic helper assignment - Goal

Goal: Efficiently assign helpers to helpees to speedup sensor data upload

Different helpers have different V2I and V2V conditions

End-to-end upload performance depends on network performance on both the V2V path and the V2I path
Key idea: Identify performance impacting factors for V2V and V2I and combine these factors to score the assignments

Direct measures: V2V & V2I bandwidth
Indirect measures: factors that affects V2V & V2I bandwidth (e.g. distance, interference)

9

Let’s start with Harbor’s helper assignment algorithm. The problem [click] is to assign a helper vehicle for each helpee vehicle, and we want to do this in an efficient manner to optimize the collaborative sensing performance through faster sensor data upload. This is not an easy problem as different helpers can have different V2I network conditions [c]. [c]

The end-to-end upload performance depends on the network performance on both V2V path and V2I path as helpee vehicles first send their data to helpers and relay it to the server.

Our key ideas is to identify the performance-impacting factors for V2V and V2I path and merge these impacting factors to score the possible assignment decisions.

Such impacting factors include direct network performance measures and indirect network performance measures.

[c] The most direct & obvious measure of the network performance is the V2V and V2I network bandwidth.

[c] While Indirect measures include factors that affect the V2V & V2I bandwidth, which I will introduce shortly.

10 of 20

Identify performance impacting factors

Both V2V and V2I paths’ performance matter

V2I bandwidth can be measured from V2I path

→ Measured from ongoing data transfer to the edge server

However, V2V bandwidth is hard to quantify

Measured bandwidth by establishing connections? → Large overhead, can be inaccurate

→ Using indirect measures and heuristics

Performance impacting factors

V2I path: V2I bandwidth and load of helper
V2V path: Physical distance between helper and helpee, V2V network interference

10

As we have seen the performance of both V2V and V2I path are important. We start thinking about what factors to use for assigning helpers to helpees [c].

[c] For the V2I path, the V2I bandwidth is measurable as vehicles with V2I connectivity constantly transmit data to the server [c]. [c] However, for the V2V path, this is not the case. [c] V2V bandwidth is hard to quantify because not all vehicles are actively transmitting data to each other and their performance will be impacted by the choice of other helpee to helper links. [c] Using probing or active measurements by establishing connections between vehicles creates too much overhead.

[c] Therefore, to reduce the overhead of active measurement of V2V bandwidth, we apply heuristics and use indirect measures in the V2V channel to quantify its network performance.

To summarize, we consider different performance impacting factors [c]. For the V2I path: we use the the measured V2I bandwidth and the load of a helper. You could imaging that using pairing a helper with higher bandwidth is better to reduce the data upload latency. [c] For the V2V path, we use the physical distance between a helper and a helpee, as well as the V2V network interference caused by multiple wireless transmitters and receivers. The intuition behind those factors is that we prefer helpers w/ closer distances and assignments that create less wireless interference.

11 of 20

Strategic helper assignment - Overview

Harbor jointly considers different factors and selects an assignment with the analytical model

A heuristic score function to merge effects caused by different factors
Normalize each factor to avoid a single factor becoming dominant
Run periodically to adapt to vehicle mobility & network state changes

11

Assignment

Assignment pair

helpee

helper

Harbor uses a heuristic score function to combine result of the above factors, by summing up distance score and interference score under V2I bandwidth constraint [click]. We define an assignment as a set of assignment pairs [click], with each pair representing a helpee and the corresponding helper [click]. We also normalize the score of each factor to avoid that a single factor being too large and dominant. Bad assignments are penalized by getting lower scores and will not be selected. We are doing assignments in a centralized manner for all the helpees together and run this process periodically so that we can update the assignment decision when the system state changes.

Please refer to our paper for more details about the analytical models and how we optimize the helper assignments calculation.

12 of 20

Timely delivery of detection results

It is vital to deliver remote detection results back to vehicles in time

Key idea: Application-layer deadline awareness

12

Application-layer deadline awareness

Server deadline for frame merging and detection
Vehicles use existing time synchronization methods (NTP) to sync with the edge

13 of 20

Timely delivery of results - MAC layer prioritization

In V2V network, sensor data transmission (by helpee) and detection result delivery (from helper) contend for the shared wireless medium
Key Idea: Prioritize small but latency sensitive traffic over bulk data transfer

13

V2V Wireless Network

Higher Priority

Lower Priority

Higher Priority

Lower Priority

Sensor data stream: bandwidth intensive

TX Queues

Helpee

Helper

Detection results: small but latency sensitive

Another problem that affects the detection results delivery is the resource contention at the MAC layer. [click] In V2V network, bulk sensor data transmission (by helpee) and detection result delivery (from helper) contend for the shared wireless medium. How to account for the bandwidth requirement for sensor data stream and latency requirement for small detection result delivery?

[click] Our key idea is straightforward yet effective: Prioritize small but latency sensitive traffic over bulk data transfer. To prioritize the traffic at the MAC layer, we create a higher priority queue in the 802.11 Mac transmission (TX) queue for transmitting the detection result messages. We adjust the contention window parameter of the transmission queue in the OS kernel so that packets that contain detection results have a higher possibility of getting the medium.

14 of 20

Outline

Background and Motivation
System Design
Evaluation Results
Conclusion

14

15 of 20

End-to-end evaluation - Emulation

Evaluation Metrics

End-to-end detection latency (s) and detection accuracy
Compare with different baseline schemes

15

Harbor outperforms all V2V/V2I sharing schemes

Harbor reduces mean latency by 38.0% on average

16 of 20

End-to-end evaluation – Real-world Driving

Harbor outperforms EMP, AVR and CarSpeak by reducing 18% - 37% of detection latency and improving 8.0% - 11.0% on accuracy.

16

Harbor outperforms all V2V/V2I sharing schemes in real-world driving experiments

17 of 20

End-to-end Evaluation – Mcity Testbed

Harbor achieves the best perception accuracy compared with EMP^[1] and AVR^[2]

Harbor delivers better autonomous driving outcome by increasing the driving reaction time and reducing crash scenarios

17

18 of 20

Outline

Background and Motivation
System Design
Evaluation Results
Conclusion

18

19 of 20

Conclusion

Develop Harbor, a hybrid system architecture for cooperative vehicular perception
Harbor strategically assigns helpers to helpees

Harnesses performance impacting factors from different network layers

Harbor performs timely detection result delivery

App-layer deadline awareness and MAC-layer message prioritization

Harbor outperforms V2V and V2I collaboration solutions

Reduces up to 57.1% in end-to-end latency and improves up to 12% in detection accuracy
Result in significantly fewer collisions under dangerous driving scenarios

19

20 of 20

20

Thank You!

Our Team