Control Interactions and Feedback in Virtual and Augmented Reality

Tania Khan

Introduction

• COGS: Cognition and Brain Stream
• Supervisor: Dr.Patrick Pennefather
• Collaborated With: Emerging Media Lab
• Project Type: User Testing
• End Goal: Design Document with Recommendations for VR & AR controls

The Problem

• Interacting with elements and objects in VR and AR is not intuitive
• Lack of guidance for user - motion controller interactions, not enough research conducted
• Current “best options” on the market are expensive and wear easily

Current Literature and Technology

• Immersive virtual embodiment (feeling of one’s own body and sensation in VR) can be shaped by vestibular and tactile information (Kilteni et al., 2012)
• Co-located haptic feedback improves immersion and depth perception (Swapp et al., 2006)
• While a VR or AR headset is important for audio-visual immersion, the controls are what allow users to actually interact with the environment (Circuit Stream, 2022)

User Testing

• Controls for the Oculus Rift and HTC Vive were tested by completing certain tasks in the Fruit Ninja, Ikea Pancake Kitchen and Open Brush apps
• Controls for the Hololens 2 were tested through the Graffiti 3D app
• Users shared their feedback through a qualitative interview (total 5 users)

Results & Insights

What did the user testing reveal?

Oculus Rift Controllers

• 3/5 users found that there were too many input buttons to navigate

​

• 2/5 users pressed the middle and index finger triggers accidentally when focusing on the Fruit Ninja and Pancake Kitchen tasks

​

• 3/5 users preferred the joystick because it was easier for directional input and to receive feedback

Right Hand Controllers

​

Left Hand Controllers

​

HTC Vive Controllers

• 3/5 users found the controllers to be too big for their hands, length wise

​

• 4/5 users liked the simplistic design and lack of input buttons

​

• 2/5 users preferred the trackpad because of its scrolling feature and resemblance to laptop pads and touch screens

Index Finger Triggers

Middle Finger Triggers

Joysticks

Main Menu

Letter Buttons (X, Y, B, A) are predominantly used for specific gaming functions like jumping, etc

Hololens 2 Gesture Controls

• All users struggled with basic gestures such as pressing buttons for the menu

​

• Only 1 user used the “pinch” gesture to draw while everyone else used a one finger gesture to try and draw

​

• All users were frustrated by the lack of tactile/haptic feedback available to them

Accessing Menu Gesture

​

Drawing Gesture

​

VR Design Recommendations

• Limit the number of input buttons to avoid misclicks and confusion (aka SIMPLE)
• Increase controller capacity for haptic feedback and vibrations
• Incorporate joysticks as the input has stronger tactile response

AR Design Recommendations

• Considering transferring touch screen gestures for AR controls
• Adding haptic feedback and tactile elements for depth perception
• External element that allows users to input controls e.g. touch screen remote

​

Citations

Circuit Stream. (2022, January 5) Overview of VR Controllers: The Way of Interacting With the Virtual Worlds. https://circuitstream.com/blog/vr-controllers-the-way-of-interacting-with-the-virtual-worlds/

​

Nanjappan, V., Liang, H., Lu, F., Papangelis, K., Yue, Y., & Man, K. L. (2018). User-elicited dual-hand interactions for manipulating 3D objects in virtual reality environments. Human-Centric Computing and Information Sciences, 8(1), 1-16. https://doi.org/10.1186/s13673-018-0154-5

​

Kilteni, K., Groten, R., & Slater, M. (2012). The sense of embodiment in virtual reality. Presence (Cambridge, Mass.), 21(4), 373-387. https://doi.org/10.1162/PRES_a_00124

​

SWAPP, D., PAWAR, V., & LOSCOS, C. (2006). Interaction with co-located haptic feedback in virtual reality. Paper presented at the , 10(1) 24-30. https://doi.org/10.1007/s10055-006-0027-5

​

​

Thanks!

Tania Khan

Email: tanzk00@gmail.com

Phone: +1 858-308-2574

Website: https://taniamkhan.wixsite.com/portfolio

​

Product Designer

​

Tania Khan

16

https://taniamkhan.wixsite.com/portfolio