1 of 20

PSEUDO-HAPTIC FEEDBACK USING AUDIO IN VR WATERCRAFT NAVIGATION

Harshita Shetye

CMP504: Masters Project

Supervisors: Niall Moody, Naman Merchant

MSc, Computer Games Technology

Abertay University

2 of 20

OVERVIEW

A brief overview of the project – Research gap, implementation, test results, and conclusion.

�

2

9/19/2025

3 of 20

WHY THIS PROJECT?

Started off with an idea of building a virtual instrument for learning purposes
Haptic hardware is extremely costly
Existing work has been done for cross-modal perception

Audio, visual, touch
Solid surfaces

Gap: Tying these elements together

Procedural
Dynamic
Real-time physics interaction
Liquid surface

This project initially started off with an interesting idea of creating a virtual instrument for learning purposes. As a prior school band geek and a music lover, I was very interested in experimenting with some audio design.
I checked with some staff in ETC and unfortunately, haptic gloves were not available for student research purposes. This lead me to think how much it would cost indie studios or developers to purchase such a hardware. The cost went beyond 600 pounds just for the gloves.
And so, I researched if there is any cheaper alternative to technology. That's when I got to know about cross-modal perception – which basically works when the brain combines inputs from different senses into a single experience.
When looking at previous studies, I saw that researchers often combined visuals and sound to create these touch-like illusions. I wanted to see if sound, alone, could do that work.
I found a few studies that focused only on sound, but they were mostly for simple interactions, like feeling the texture of a virtual block. The sound wasn't directly tied to the complex, physics-based actions.
That's where I found my research gap: Whether procedural sound alone could create the feeling of force in a dynamic vehicle simulation. My project aimed to connect those missing pieces.

4 of 20

�

4

9/19/2025

5 of 20

RESEARCH QUESTION & ARTEFACT

Proposed Solution: Use carefully engineered audio as the medium for cross-modal perception to create an illusion of touch which is a technique known as "pseudo-haptics".
Research Question: Can procedurally modified water audio create convincing audio-tactile illusions of hydrodynamic resistance during VR boat maneuvers, particularly during sharp turns and water collisions?
Artefact: A VR boat simulation was developed in Unreal Engine 5.5.4.

Capture the physics parameters in real-time...
And pass those parameters to MetaSounds...
To dynamically affect the Procedural Audio output

6 of 20

EXPERIMENT

Additionally, to test how effectively real-time physics could influence the audio, two conditions were created.
13 participants

Condition A: Procedural & Dynamic

The sound constantly reacted to the boat's physics taken from the blueprint in real-time.

Condition B: Procedural & Static

Default variables were assigned instead of taking input parameters from the blueprint.

7 of 20

EXPERIMENT

Additionally, to test how effectively real-time physics could influence the audio, two conditions were created.
13 participants

Condition A: Procedural & Dynamic

The sound constantly reacted to the boat's physics taken from the blueprint in real-time.

Condition B: Procedural & Static

Default variables were assigned instead of taking input parameters from the blueprint.

8 of 20

RESULTS

Open-Ended Question: In which variations, if any, did you feel illusions of physical touch while in the environment? Was there a specific place where you felt this?

Out of 13 -> 9 reported feeling some physical resistance while 4 reported none.

"Splash/drift during sharp turns"
"Approaching and falling off the waterfall"
Additional: Boat engine, wood creaks

9 of 20

CONCLUSION

The responsiveness of the dynamically modulated procedural audio, driven by real-time physics, is an effective method to create pseudo-haptic illusions of physical resistance in a watercraft VR.
This gives the developers a practical solution to create believable virtual worlds without the use of costly hardware.
The research bridges a gap: To extend the pseudo-haptic theory from simple and static interaction to a complex and dynamic interaction in a liquid surface.

10 of 20

PROJECT STRUCTURE

YouTube Link: https://youtu.be/IcuQs0I1cAY

�

10

9/19/2025

11 of 20

BLUEPRINTS: PHYSICS

Parameters: RiverSpeed, AlignmentWithCurrent, EngineSound, RiverCurveIntensity, BoatFoam, Splash/DriftSound

�

11

9/19/2025

12 of 20

RIVERSPEED

�

12

9/19/2025

13 of 20

ALIGNMENTWITHCURRENT

�

13

9/19/2025

14 of 20

ENGINESOUND

�

14

9/19/2025

15 of 20

RIVERCURVEINTENSITY

Based on the boat's location, the following points on the water spline are taken...

D1: Boat's current location
D2: 5 meters ahead of the boat

River's direction at D1 and D2 is taken.
Dot product of D1 • D2 gives [-1, 1], with 0 for a sharp curve.
This is then ABS and inverted so that 1 = sharp curve.

… which interpolates the river curve intensity over a period of 0.5 seconds.

�

15

9/19/2025

16 of 20

BOATFOAM

This takes dot product output from the RiverCurveIntensity calculation, meaning 1 if it is a sharper curve.
This is multiplied by the river's speed and interpolated over a period of 1 second.
It is then clamped between [0.05, 1] and passed to MetaSounds.

�

16

9/19/2025

17 of 20

SPLASH/DRIFT SOUND

Dot product of Boat Velocity • Boat Right Velocity is taken to get the boat's "sideways" velocity.
If this speed exceeds 800, then the value is clamped between [0.1, 1], which is sent either to the right-side or the left-side audio placement, depending on which side the boat is drifting.
If the dot product in #1 is less than 0, then the boat is turning left = audio parameter is updated. Else, right audio parameter is updated.
If the speed is less than 800, then the audio parameter is set to 0 volume.

�

17

9/19/2025

18 of 20

METASOUNDS

�

18

9/19/2025

19 of 20

�

19

9/19/2025

20 of 20

ANY QUESTIONS?

�

20

9/19/2025