1 of 53

CSE 5524: �Image formation

2 of 53

Course information

Carmen/GitHub:

Will add people by today!
For announcement, posting course materials (slides), and homework submission

Piazza:

For discussion. Please register!
Link: To set up by today!
Please use name.#@osu.edu
Access code: osu-cse-5524-SP25-chao

Detailed syllabus (pdf):

can be found on Carmen and the course website

2

3 of 53

Textbook

Required

https://mitpress.mit.edu/9780262048972/foundations-of-computer-vision/
OSU PDF access: https://library.ohio-state.edu/record=b11145305
Purchasable on MIT Press Bookstore or Amazon

3

Foundations of Computer Vision

We have PDF access now!

4 of 53

1/23 next Thursday

I will be traveling.

The current plan is that the TA will give a lecture about “PyTorch,” which will be very useful for the final project and potentially for homework as well.

5 of 53

Today

Recap
Image formation (continued)
Camera
Color

5

6 of 53

From lights to world interpretation

To understand our world from the lights

We need to “associate” the reflected light with the surface in the world.
We need to know which light rays come from which direction in space.

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

7 of 53

Images & cameras

Forming an image = identifying which rays coming from which directions

Camera: organizing rays

Pinhole camera:

One location on the wall
Light from one direction

Projection surface

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

8 of 53

Perspective projection equations

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

9 of 53

Orthographic (parallel) projection equations

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

10 of 53

From pinholes to lenses

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Light needs to be concentrated/ bent!

11 of 53

Snell’s law

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

12 of 53

A lens

A specifically “shaped” piece of transparent material, positioned to focus light from a surface point onto a sensor

Ideally …

Need: numerical optimization!

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

13 of 53

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Lensmaker’s formula:

14 of 53

Generalization to the “whole plane”

Points of the optical axis

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

15 of 53

Questions?

16 of 53

Imaging with lenses

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Lenses impose a “perspective” projection, like pinhole cameras

17 of 53

Imaging with lenses

Two points on the opposite sides of the lens at distance a and b are conjugate

f: focal length

n: material index

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

18 of 53

Imaging with lenses

Lights from one conjugate points will focus on the other conjugate points

Parallel rays will focus at distance “f”
Rays through the center will be straight, like a pinhole camera
Magnification of a lens is a/b

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

19 of 53

Depth of field

If the lens is part of a low-end, non-adjustable camera, what is fixed?

Sensor plane

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Focal plane

20 of 53

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Depth of field:

region around the focal plane whose blur effect is within the tolerance

21 of 53

Depth of field

[Figure credit: https://www.google.com/url?sa=i&url=https%3A%2F%2Ffstoppers.com%2Fnews%2Fchange-your-depth-field-app-9507&psig=AOvVaw1mooNsqleZkvCtSeA28kSk&ust=1737132367428000&source=images&cd=vfe&opi=89978449&ved=0CBQQjRxqFwoTCICPo8bY-ooDFQAAAAAdAAAAABAR]

22 of 53

Is Depth of field adjustable?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

A: aperture

A = f/N, N is a camera’s f-number

U: object plane

23 of 53

Is Depth of field adjustable?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

24 of 53

Consumer cameras

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

25 of 53

Questions?

26 of 53

Lens is not necessarily convex

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

27 of 53

Lenses in telescope

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

28 of 53

Consumer camera

https://i1.adis.ws/i/canon/pro-infobank-aperture-lens-cutaway_3823a2b43d46401999eb0c4886f78d99?$media-collection-full-dt-jpg$

29 of 53

Questions?

30 of 53

Camera as linear system (section 7)

So far, we talked about “lens-based” pinhole cameras

Immediate captured!

There are other imaging systems:

Medical
Astronomical

The intensity recorded (data) may look nothing like an interpretable images

Need one more step, e.g., through linear algebra, to recover an image

31 of 53

Cameras as linear systems

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Inverse problem

32 of 53

Cameras as linear systems

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

33 of 53

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

34 of 53

Other imagers: edge camera

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

35 of 53

Other imagers: pinspeck camera

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

36 of 53

Fun read

37 of 53

38 of 53

Light field camera

39 of 53

Questions?

40 of 53

Color

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

41 of 53

Color

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

42 of 53

Why?

Light is a mixture of electromagnetic waves of different wavelengths/frequencies

Visible lights

Refraction bends light

Diffraction separate lights

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

43 of 53

Light power spectrum

Light intensity at each wavelength = power spectrum (like a distribution)

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

44 of 53

Light power spectrum

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

45 of 53

Color appearance

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

46 of 53

Light reflectance from surfaces

Surface characteristics will change the light spectrum

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

47 of 53

Light reflectance from surfaces

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

48 of 53

Color perception

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Cone receptor in huma retina

49 of 53

What does it mean?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

50 of 53

Color reproduction

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

51 of 53

Color reproduction

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

52 of 53

Spatial resolution and color

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

53 of 53

Spatial resolution and color

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]