1 of 33

`Human Physiology, Perception, and Virtual Reality

Dr. Subhajit Sidhanta

​

​

​

​

Department of Industrial Engineering & Management

Indian Institute of Technology (IIT) Kharagpur

West Bengal, India

1

2 of 33

vir·tu·al re·al·i·ty

vərCH(əw)əl rē‘alədē

the computer-generated simulation of a three-dimensional image or environment that can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors.

2

3 of 33

vpl research

4 of 33

5 of 33

6 of 33

These should motivate you to appreciate the amount of work that our sense organs and neural structures are doing to fill in missing details and make interpretations based on the context of our life experiences and existing biological structures. Interfering with these without understanding them is not wise!

7 of 33

remote control of vehicles, e.g. drones

architecture walkthroughs

virtual travel

education

a trip down the rabbit hole

4

8 of 33

Vision treatment in VR

  • Treatment of amblyopia
    • Training the brain to use the “lazy” eye

Images courtesy of

9 of 33

Exciting Engineering Aspects of VR/AR

6

  • sensors & imaging
  • computer vision
  • scene understanding

haptic, …

  • photonics / waveguides
  • human perception
  • displays: visual, auditory, vestibular•,
  • VR cameras
  • cloud computing
  • shared experiences
  • HCI

applications

  • compression, streaming
  • CPU, GPU
  • IPU, DPU?

images by microsoft, facebook

10 of 33

Where We Want It To Be

7

image by ray ban

11 of 33

Personal Computer

8

e.g. Commodore PET 1983

Laptop

e.g. Apple MacBook

Smartphone

e.g. Google Pixel

???

​

AR/VR

e.g. Microsoft Hololens

12 of 33

A Brief History of Virtual Reality

9

1838

1968

2012-2018

Stereoscopes

Wheatstone, Brewster, …

VR & AR

Ivan Sutherland

VR explosion

Oculus, Sony, HTC, MS, …

Nintendo Virtual Boy

1995

???

13 of 33

Ivan Sutherland’s HMD

  • optical see-through AR, including:
    • displays (2x 1” CRTs)
    • rendering
    • head tracking
    • interaction
    • model generation

​

  • computer graphics
  • human-computer interaction

10

I. Sutherland “A head-mounted three-dimensional display”, Fall Joint Computer Conference 1968

14 of 33

Nintendo Virtual Boy

11

  • computer graphics & GPUs were not ready yet!

Game: Red Alarm

15 of 33

Where we are now

12

IFIXIT teardown

16 of 33

17 of 33

18 of 33

19 of 33

20 of 33

Virtual Image

13

1 + 1 = 1

d d ' f

d

d’

f

Problems:

  • fixed focal plane
  • no focus cues ☹
  • cannot drive accommodation with rendering!
  • limited resolution

21 of 33

A dual-resolution display

14

  • High resolution image in the centre, low resolution fills wide field-of-view
  • Two displays combined using a beam-splitter
  • Image from: https://varjo.com/bionic-display/

22 of 33

AUGMENTED REALITY�Pepper’s Ghost 1862

16

23 of 33

Google Glass

17

24 of 33

Google Glass

18

25 of 33

Meta 2

19

  • Larger field of view (90 deg) than Glass

​

  • Also larger device form factor

26 of 33

Microsoft HoloLens

20

27 of 33

Microsoft HoloLens

  • diffraction grating

​

  • small FOV (30x17), but good image quality

21

28 of 33

Microsoft HoloLens 2

  • Wider field of view (52 deg)
  • High resolution (47 pix per deg)
  • Improved ergonomics
  • Better hand tracking

29 of 33

Zeiss Smart Optics

23

  • great device form factor
  • polycarbonate light guide – easy to manufacture and robust
  • smaller field of view (17 deg)

30 of 33

Sony IMX-001

24

  • also great form factor
  • small FOV (9x6 deg)
  • monochrome

31 of 33

Video AR: ARCore, ARKit, ARToolKit, …

25

32 of 33

VR/AR challenges

  • Latency (next lecture)
  • Tracking
  • 3D Image quality and resolution
  • Reproduction of depth cues (last lecture)
  • Rendering & bandwidth
  • Simulation/cyber sickness
  • Content creation
    • Game engines
    • Image-Based-Rendering

26

33 of 33

Simulation sickness

27

  • Conflict between vestibular and visual systems
    • When camera motion inconsistent with head motion
    • Frame of reference (e.g. cockpit) helps
    • Worse with larger FOV
    • Worse with high luminance and flicker