1 of 53

CSE 5524: �Geometry & Camera modeling

2 of 53

Self introduction

3 of 53

Today

​

  • Recap: camera
  • Image representation
  • Homogeneous & heterogeneous coordinate systems
  • Geometry transformation
  • Camera model & calibration

3

4 of 53

Images & cameras

  • Forming an image = identifying which rays coming from which directions

​

  • Camera: organizing rays

​

  • Pinhole camera:
    • One location on the wall
    • Light from one direction

4

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

5 of 53

From pinholes to lenses

5

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

6 of 53

Perspective projection equations

6

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

7 of 53

Perspective projection equations

[Figure credit: A. Torralba, P. Isola, and W. T. Freeman, Foundations of Computer Vision.]

8 of 53

Today

​

  • Recap: camera
  • Image representation
  • Homogeneous & heterogeneous coordinate systems
  • Geometry transformation
  • Camera model & calibration

8

9 of 53

Computer vision: data

9

RGB image (s): Three matrices

10 of 53

What is the data structure of images?

  • Ordered array (matrix) of pixel values:

​

​

​

​

  • A set of pixels and their locations:
    • Make geometry explicit!

10

Regular spatial grid

 

11 of 53

Example: image translation

11

12 of 53

Today

​

  • Recap: camera
  • Image representation
  • Homogeneous & heterogeneous coordinate systems
  • Geometry transformation
  • Camera model & calibration

12

13 of 53

Homogeneous and heterogeneous coordinates

13

heterogeneous

homogeneous

14 of 53

Properties of the homogeneous coordinate system

  • Scaling equivalence (non-zero scale)

​

​

​

​

  • Illustration

​

​

14

heterogeneous

homogeneous

15 of 53

Today

​

  • Recap: camera
  • Image representation
  • Homogeneous & heterogeneous coordinate systems
  • Geometry transformation
  • Camera model & calibration

15

16 of 53

2D image transformation

16

 

17 of 53

Translation

  • Heterogeneous coordinate:

​

​

​

​

  • Homogeneous coordinate:

17

Product-only operations

18 of 53

Translation: heterogeneous vs. homogeneous

18

heterogeneous

homogeneous

19 of 53

Translation: heterogeneous vs. homogeneous

19

homogeneous

 

20 of 53

Translation: heterogeneous vs. homogeneous

20

 

homogeneous

 

21 of 53

Scaling

21

 

22 of 53

Rotation

22

23 of 53

 

23

 

24 of 53

 

24

 

25 of 53

Shearing

25

 

 

26 of 53

Chaining transformations

  • Homogeneous coordinates allow us to combine transformations via products

26

27 of 53

Important properties

  • Matrix operations are noncommunicative
    • The order of operations is important

​

  • All the operations are relative to the “origin”
    • If you want to perform operations relative to an arbitrary central location, translate first, and then translate it back.

​

27

28 of 53

Generic 2D transformations

  • Affine transformations (translation, rotation, scaling, and shears)
    • 6 degrees of freedom
    • Parallel lines remain parallel
    • Lengths and angles can change

​

​

  • More general transformations
    • 8 degrees of freedom
    • including elations and projections

28

29 of 53

Questions?

30 of 53

Image warping

  • Transformations are “continuous”, but pixel locations are “discrete”

30

 

’

 

31 of 53

Forward mapping

  •  

31

32 of 53

Backward mapping

  •  

32

33 of 53

Forward vs. backward mapping

33

34 of 53

Today

​

  • Recap: camera
  • Image representation
  • Homogeneous & heterogeneous coordinate systems
  • Geometry transformation
  • Camera model & calibration

34

35 of 53

Recap: simple vision system & image formation

35

Where is the origin of the 3D world coordinate system?

What is the orientation of the word coordinate system?

36 of 53

How about now?

36

Is there a formulation to change the coordinate system easily?

37 of 53

Illustration

37

Image

Image formation

Know how to convert coordinates

Know how to convert coordinates back

38 of 53

Questions?

39 of 53

Perspective projection

  • Assumption:
    • The world coordinate origin is at the pinhole
    • Z-axis is perpendicular to the image plane

39

 

 

 

  • Division complicates the algebraic relations
  • Using “homogeneous” coordinates can resolve it

40 of 53

Recap: homogeneous coordinates

  • Homogeneous and heterogeneous coordinates

40

heterogeneous

homogeneous

41 of 53

Perspective projection in homogeneous coordinates

41

 

 

42 of 53

Camera-intrinsic parameters

42

(X, Y, Z)

n

m

43 of 53

Questions?

44 of 53

Camera-extrinsic parameters

44

R: 3D rotation between coordinate systems

​

T: 3D translation between coordinate systems

 

45 of 53

Camera-extrinsic parameters

45

heterogeneous

homogeneous

46 of 53

Camera-extrinsic parameters

46

homogeneous

 

 

 

 

47 of 53

Questions?

48 of 53

Full camera model (intrinsic + extrinsic)

48

 

49 of 53

Full camera model (intrinsic + extrinsic)

Camera

3D

Intrinsic

Rotation

Translation

50 of 53

Questions?

51 of 53

Glance at camera calibration

  •  

51

52 of 53

Questions?

53 of 53

Going from 2D to 3D

  • If we know Z of (x, y), can we recover (X, Y, Z)?

53