1 of 53

VR and Image Sensors

Computer Graphics and Imaging

UC Berkeley CS 184/284A

Discussion 13

2 of 53

Worksheet 13 link!

https://tinyurl.com/yxh82fvp

3 of 53

Week 13 Announcements

ASSIGNMENT 1

Homework 4

Deadline

Code / write-up:

Friday (11/27)!

ASSIGNMENT 2

Final Project

Deadline

Milestone:

Friday (11/22)!

LOCATION & TIMES

Office Hours

@ Cory 531!

  • Kevin: M 12-2pm
  • Stephane: Tu 10-12am
  • Yoko: W 3-5pm
  • Joshua: Th 11am-1pm

4 of 53

Virtual Reality

5 of 53

Terminology

  • Augmented reality adds on to view of the real world.
  • Virtual reality is immersive.
  • User sees only what is emitted by the headset light display.

6 of 53

Question 1

7 of 53

Vergence and Accommodation

Vergence (eye rotation):

  • Convergence: Both eyes rotate inward when looking at something close.
  • Divergence: Both eyes rotate outward when looking at something far away.

Accommodation (lens adjustment):

  • Near objects: Ciliary muscles contract → lens becomes thicker and more curved → greater refractive power.
  • Far objects: Ciliary muscles relax → lens becomes thinner and flatter → lower refractive power.

8 of 53

Q1 Virtual Reality Ethan

9 of 53

Q1.1 Virtual Reality Ethan

10 of 53

Q1.1 Virtual Reality Ethan

11 of 53

Q1.2 Virtual Reality Ethan

12 of 53

Q1.2 Virtual Reality Ethan

13 of 53

Q1.3 Virtual Reality Ethan

14 of 53

Q1.3 Virtual Reality Ethan

15 of 53

Q1.4 Virtual Reality Ethan

16 of 53

Vergence-Accommodation Conflict

  • Usually, accommodation and vergence are in sync.
  • VR causes a possible mismatch.
  • The conflicting signals that your brain receives causes discomfort.

17 of 53

Q1.4 Virtual Reality Ethan

18 of 53

Q1.5 Virtual Reality Ethan

19 of 53

Need for Low-latency in VR

  • If you turn your head → how does the VR know what to display next?

20 of 53

Need for Low-latency in VR

  • If you turn your head → how does the VR know what to display next?
  • Measure user’s head movement.

​

21 of 53

Need for Low-latency in VR

  • If you turn your head → how does the VR know what to display next?
  • Measure user’s head movement.
  • Calculate updates to the scene.

​

22 of 53

Need for Low-latency in VR

  • If you turn your head → how does the VR know what to display next?
  • Measure user’s head movement.
  • Calculate updates to the scene.
  • Render the new image!

23 of 53

Q1.5 Virtual Reality Ethan

24 of 53

Q1.6 Virtual Reality Ethan

25 of 53

Q1.6 Virtual Reality Ethan

26 of 53

Q1.7 Virtual Reality Ethan

27 of 53

Q1.7 Virtual Reality Ethan

28 of 53

Q1.8 Virtual Reality Ethan

29 of 53

Q1.8 Virtual Reality Ethan

30 of 53

Sensor

31 of 53

Recall Cameras…

  • During the shutter duration, the open camera lens collects light.
  • This light is focused onto the image sensor.
  • Photons are particles of light.
  • They enter the camera according to the Poisson distribution.

32 of 53

Poisson Distribution

  • Models the occurrence of independent events over a fixed time interval.

33 of 53

Poisson Distribution

  • Models the occurrence of independent events over a fixed time interval.
  • For example,
    • The arrival of buses while waiting at the stop for an hour. (For CS 70 enjoyers.)

34 of 53

Poisson Distribution

  • Models the occurrence of independent events over a fixed time interval.
  • For example,
    • The arrival of buses while waiting at the stop for an hour. (For CS 70 enjoyers.)
    • The arrival of photons over a shutter duration.
  • The average rate of arrival is λ.

35 of 53

Poisson Shot Noise

  • In low-light conditions, fewer photons arrive.
  • The signal received by the image sensor has more variability.
  • This variability is Poisson shot noise.

36 of 53

37 of 53

ISO/Gain

38 of 53

Signal to Noise Ratio

39 of 53

[Gaussian] noised signal vs true signal

Distribution of [Gaussian] noise

40 of 53

Signal to Noise formula for images

Signal power (mean)

Noise power (std dev)

μ

σ

=

  • For non-negative random variables
  • Applies to photon counts per pixel when taking a photo!
  • Wow so clean

41 of 53

42 of 53

More photons -> higher SNR

(better quality image)

Signal power (mean)

Noise power (std dev)

μ

σ

=

43 of 53

Question 2

44 of 53

Q2.1 Sensors

45 of 53

Q2.2 Sensors

46 of 53

Q2.3 Sensors

MORE PHOTONS!!!!

47 of 53

Question 3

48 of 53

 is the average number of photons captured by a pixel.

49 of 53

Q3.1 Poisson Shot Noise

50 of 53

Q3.2 Poisson Shot Noise

51 of 53

Q3.3 Poisson Shot Noise

52 of 53

Q3.4 Poisson Shot Noise

53 of 53

Let’s Take Attendance.

  • The attendance form is in the header of the CS 184/284A course website: cs184.eecs.berkeley.edu/sp26/. You can also use this QR code:
  • Be sure to select put today’s date and �input your TA’s secret word: VR 😉
  • Any feedback? Let us know!