1 of 55

CSE 5524: �Image formation

2 of 55

Course information, grading, reading, policy, etc.

  • Please check slide decks 1 & 2, course website, Carmen, and syllabus.

​

  • Course website:

https://sites.google.com/view/osu-cse-5524-sp25-chao/home

​

  • Office hours start this week!
    • Dr. Chao (DL587): Tuesday 3 – 4 pm & Friday 9 – 10 am
    • Zheda Mai (BE 406, # 6): Monday 11 am - 12 pm & Wednesday 2 pm - 3 pm

​

  • Linear algebra quizzes: released last week, due 9/16

2

3 of 55

Textbook

​

  • Required

​

​

​

​

​

​

​

​

​

​

​

​

​

3

Foundations of Computer Vision

You have PDF access!

4 of 55

Homework release and due (tentative)

​

  • HW 1: 9/4 – 9/18
  • HW 2: 9/16 – 9/23
  • HW 3: 9/23 – 10/7
  • HW 4: 10/7 – 10/21
  • HW 5: 10/28 – 11/13
  • HW 6: 11/13 – 12/2

​

  • Final project presentation: 12/16

4

5 of 55

Today

​

  • Recap
  • Image formation (continued)
  • Camera

5

6 of 55

From lights to world interpretation

  • To understand our world from the lights
    • We need to “associate” the reflected light with the surface in the world.
    • We need to know which light rays come from which direction in space.

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

7 of 55

Images & cameras

  • Forming an image = identifying which rays coming from which directions

​

  • Camera: organizing rays

​

  • Pinhole camera:
    • One location on the wall
    • Light from one direction

Projection surface

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

8 of 55

Perspective projection equations

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

9 of 55

Orthographic (parallel) projection equations

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

10 of 55

Can we really have orthographic projection?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

11 of 55

Today

​

  • Recap
  • Image formation (continued)
  • Camera

11

12 of 55

What’s wrong with pinhole cameras

Projection surface

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Images are dime …

Limited lights ...

13 of 55

From pinholes to lenses

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Light needs to be concentrated/ bent!

14 of 55

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

15 of 55

Bending the light

  • From one material to the other, light changes its wavelength and speed

​

​

​

​

​

  • The changes at the surface will cause light to bend, i.e., refraction
    • Depend on the change of speed and orientation

16 of 55

Snell’s law

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

17 of 55

A lens

  • A specifically “shaped” piece of transparent material, positioned to focus light from a surface point onto a sensor

​

  • Ideally …

​

​

​

  • Need: numerical optimization!

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

18 of 55

Simplified optical system

  •  

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

19 of 55

  • Assumptions:
    • Paraxial: the angle is small
    • Thin lens: negligible thickness

​

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

20 of 55

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

  • Assumptions:
    • Paraxial: the angle is small
    • Thin lens: negligible thickness

​

 

21 of 55

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

  • Assumptions:
    • Paraxial: the angle is small
    • Thin lens: negligible thickness

​

  • Lensmaker’s formula:

​

22 of 55

Generalization to the “whole plane”

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

23 of 55

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

24 of 55

Today

​

  • Recap
  • Image formation (continued)
  • Camera

24

25 of 55

Imaging with lenses

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

​

Lenses impose a “perspective” projection, like pinhole cameras

​

26 of 55

Imaging with lenses

  • Two points on the opposite sides of the lens at distance a and b are conjugate

f: focal length

n: material index

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

27 of 55

Imaging with lenses

  • Lights from one conjugate points will focus on the other conjugate points

​

​

​

​

​

​

    • Parallel rays will focus at distance “f”
    • Rays through the center will be straight, like a pinhole camera
    • Magnification of a lens is a/b

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

28 of 55

Depth of field

  • If the lens is part of a low-end, non-adjustable camera, what is fixed?

Sensor plane

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Focal plane

29 of 55

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

​

Depth of field:

region around the focal plane whose blur effect is within the tolerance

​

30 of 55

Depth of field

[Figure credit: https://www.google.com/url?sa=i&url=https%3A%2F%2Ffstoppers.com%2Fnews%2Fchange-your-depth-field-app-9507&psig=AOvVaw1mooNsqleZkvCtSeA28kSk&ust=1737132367428000&source=images&cd=vfe&opi=89978449&ved=0CBQQjRxqFwoTCICPo8bY-ooDFQAAAAAdAAAAABAR]

31 of 55

Is Depth of field adjustable?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

A: aperture

​

​

​

​

​

​

​

A = f/N, N is a camera’s f-number

U: object plane

32 of 55

Is Depth of field adjustable?

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

33 of 55

Consumer cameras

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

34 of 55

Questions?

35 of 55

Lens is not necessarily convex

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

36 of 55

Lenses in telescope

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

37 of 55

Consumer camera

https://i1.adis.ws/i/canon/pro-infobank-aperture-lens-cutaway_3823a2b43d46401999eb0c4886f78d99?$media-collection-full-dt-jpg$

38 of 55

Questions?

39 of 55

Camera as linear system (section 7)

  • So far, we talked about “lens-based” pinhole cameras
    • Immediate captured!

​

  • There are other imaging systems:
    • Medical
    • Astronomical

​

  • The intensity recorded (data) may look nothing like an interpretable images
    • Need one more step, e.g., through linear algebra, to recover an image

40 of 55

Cameras as linear systems

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

Inverse problem

41 of 55

Cameras as linear systems

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

42 of 55

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

43 of 55

Other imagers: edge camera

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

44 of 55

Other imagers: pinspeck camera

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

45 of 55

Fun read

46 of 55

47 of 55

Light field camera

48 of 55

Homework 1

49 of 55

Problem overview

Given

Goal: Generate

Y: 3D height

Z: 3D depth

50 of 55

Convention

  • In this homework, given a map (or a matrix), say I
    • I[i, j] means the i-th horizontal index (left-right) and j-th vertical index (bottom-up)
    • i >= 0, j >= 0

i

j

51 of 55

For implementation, locations are indexed by [I, j]

Y: 3D height

Z: 3D depth

(i, j)

y: 2D vertical

 

 

52 of 55

Cue 1: edges (white pixels mean edges)

All edges

Contact edges

Vertical edges

Horizontal

You need to find edge locations!

53 of 55

Put all the constraints together

 

Least square solution!

For example,

 

 

 

 

A is like 2300-by-1681

54 of 55

Why linear system? Try this toy example

1

4

7

2

?

8

3

6

9

 

 

 

i

j

55 of 55

Self introduction