1 of 11

Privacy Leakage Study and Protection for Virtual Reality Devices

​

Dirk Catpo Risco and Brandon (Jinu) Son

​

13 June 2024

​

Project Advisor: Dr. Chen

Mentors: Changming Li, Honglu Li, and Tianfang Zhang

1

2 of 11

Introducing the Team

Team members:

​

​

Mentors:

​

​

Advisor:

2

Dirk Catpo Risco

RU ECE MS

Changming Li

RU ECE PhD

​

​

Dr. Yingying (Jennifer) Chen

​

​

Brandon (Jinu) Son

RU CS UG

​

​

Honglu Li

RU ECE PhD

​

​

Tianfang Zhang

RU ECE PhD

​

​

3 of 11

Project Overview

  • Augmented reality (AR)/virtual reality (VR) devices are becoming more popular
  • Used in many applications (e.g. healthcare, communication, tourism)
  • Privacy concerns arise due to zero-permission sensors
  • We study activity privacy leakage in AR/VR devices
  • Our study focuses on activity recognition based on motion sensors

3

4 of 11

Week 2 Recap

  • Read research paper [1] regarding large language models (LLMs) comprehending the physical world

​

  • Build a connection between research paper and also privacy concerns of AR/VR devices

​

[1] Xu, H., Han, L., Yang, Q., Li, M. and Srivastava, M., 2024, February. Penetrative ai: Making llms comprehend the physical world. In Proceedings of the 25th International Workshop on Mobile Computing Systems and Applications (pp. 1-7).

4

5 of 11

Week 3 Progress

  • Set up host computer and android studio environment

  • Started extracting data from the inertial measurement unit (IMU)

​

  • Recorded and ran trials of varying head motions

5

6 of 11

Host Computer Setup

  • Objective: We try to access the IMU sensor in AR/VR headset
  • Downloaded appropriate application, software, and tools
    • MATLAB
    • IDE: Android Studio Arctic Fox | 2020.3.1
    • OS: Android 8.0 (Oreo)
    • SDK: 28.0.3
    • NDK: 20.0.5594570
  • Android studio is used to collect sensor data
  • MATLAB is used to output graphs using sensor data

6

7 of 11

Test Motions and Data Collection

  • Varying head motions
    • Forward and back
    • Rotations

​

  • Data processing using MATLAB
    • Graphs showing accelerometer and gyroscope

​

  • Collecting different activities using VR device
    • We collect two activities in one trial to demonstrate their differences

Head moving forward

Head rotating

7

8 of 11

Accelerometer Data

8

Head moving forward

Head rotating

9 of 11

Gyroscope Data

9

Head rotating

Head moving forward

10 of 11

Data Interpretation

  • Accelerometer
    • Between ~ 3 and 11 seconds
    • Graphs corresponds with forward-back head motion
  • Gyroscope
    • Between ~ 11 and 18 seconds
    • Graphs corresponds with rotational head motion
  • Conclusion
    • Graphs show accurate correlation with head motions

​

10

11 of 11

Week 4 Goals

  • Run more tests to collect more data
  • Design more motions for data collection
    • Different head motions
      • Rotational
      • Linear
    • Combinations of head motions
      • Looking left and then right
      • Looking up and then down

​

11