Introduction to computer vision II

Jean Ponce

jean.ponce@ens.fr

​

Zuhaib Akhtar za2023@nyu.edu

Ayush Jain aj3152@nyu.edu

​

​

​

Slides will be available after classes

Camera geometry�and calibration

• Intrinsic and extrinsic parameters
• Strong (Euclidean) calibration
• Degenerate configurations
• What about affine cameras?

​

​

Quantitative Measurements and Calibration

Euclidean Geometry

Pinhole Perspective Equation

Pinhole Perspective Equation

[

]

Homogeneous coordinates

Conversion

​

Converting to homogeneous coordinates

homogeneous image

coordinates

homogeneous scene

coordinates

Converting from homogeneous coordinates

Homogeneous coordinates

​

Invariant to scaling

​

​

​

​

​

​

A point in Cartesian coordinates is a ray in homogeneous ones

​

Homogeneous Coordinates

Cartesian Coordinates

Slide Credit: Savarese

Projection matrix

p: Image Coordinates: (u,v,1)

M: 3x4 projection matrix

K: Intrinsic Matrix (3x3)

R: Rotation (3x3)

t: Translation (3x1)

P: World Coordinates: (x,y,z,1)

O_w

i_w

k_w

j_w

R,t

p

f

K

Slide Credit: Savarese

Projection matrix

Intrinsic Assumptions

• Unit aspect ratio
• Image center at (0,0)
• No skew

Extrinsic Assumptions

• No rotation
• Camera at (0,0,0)

P

p

K

Slide Credit: Savarese

Projection matrix: accounting for pixel size

Intrinsic Assumptions

• Unit aspect ratio
• Image center at (0,0)
• No skew

Extrinsic Assumptions

• No rotation
• Camera at (0,0,0)

P

p

Remove assumption: known optical center

Intrinsic Assumptions

• Unit aspect ratio
• No skew

Extrinsic Assumptions

• No rotation
• Camera at (0,0,0)

Remove assumption: square pixels

Intrinsic Assumptions

• No skew

Extrinsic Assumptions

• No rotation
• Camera at (0,0,0)

Remove assumption: non-skewed pixels

Intrinsic Assumptions

Extrinsic Assumptions

• No rotation
• Camera at (0,0,0)

Oriented and Translated Camera

O_w

i_w

k_w

j_w

t

R

P

p

Allow camera translation

Intrinsic Assumptions

Extrinsic Assumptions

• No rotation

​

3D Rotation of Points

Rotation around the coordinate axes, counter-clockwise:

z

Slide Credit: Saverese

Allow camera rotation

Degrees of freedom

5

6

normalized coordinates

Explicit form of the projection matrix

Explicit Form of the Projection Matrix

Note:

M is only defined up to scale in this setting!!

Theorem (Faugeras, 1993)

Explicit Form of the Projection Matrix

Linear Camera Calibration

Linear Systems

A

x

b

=

Square system:

• unique solution

​

• Gaussian elimination

Linear Systems

A

A

x

x

b

b

=

=

Square system:

• unique solution

​

• Gaussian elimination

Rectangular system ??

• underconstrained:

infinity of solutions

Linear Systems

A

A

x

x

b

b

=

=

Square system:

• unique solution

​

• Gaussian elimination

Rectangular system ??

• underconstrained:

infinity of solutions

Minimize ||Ax-b||

2

• overconstrained:

no solution

How do you solve overconstrained linear equations ??

Homogeneous Linear Systems

A

x

0

=

Square system:

• unique solution: 0

​

• unless Det(A)=0

Homogeneous Linear Systems

A

A

x

x

0

0

=

=

Square system:

• unique solution: 0

​

• unless Det(A)=0

Rectangular system ??

• 0 is always a solution

Homogeneous Linear Systems

A

A

x

x

0

0

=

=

Square system:

• unique solution: 0

​

• unless Det(A)=0

Rectangular system ??

• 0 is always a solution

Minimize ||Ax||

under the constraint

||x|| =1

2

2

How do you solve overconstrained homogeneous

linear equations ??

The solution is e .

1

Example: Line Fitting

Problem: minimize

​

​

​

with respect to (a,b,d).

• Minimize E with respect to d:

• Minimize E with respect to a,b:

where

• Done !!

n

Note:

• Matrix of second moments of inertia

​

​

• Axis of least inertia

Linear Camera Calibration

Minimize ||Pm|| under the constraint ||m|| =1

2

2

Once M is known, you still got to recover the intrinsic and

extrinsic parameters !!!

This is a decomposition problem, not an estimation

problem.

• Intrinsic parameters

​

• Extrinsic parameters

ρ