Introduction to computer vision 3

Jean Ponce

jean.ponce@ens.fr

​

Zuhaib Akhtar za2023@nyu.edu

Ayush Jain aj3152@nyu.edu

​

​

​

Slides will be available after classes

​

​

The end of camera geometry and calibration

• Degenerate cases
• Affine cameras

Linear Camera Calibration

Minimize ||Pm|| under the constraint ||m|| =1

2

2

Degenerate Point Configurations

Are there other solutions besides M ??

• Coplanar points: (λ, μ, ν ) = (π,0,0) or (0,π,0) or (0,0,π )

• For given values of (λ, μ, ν ): points lying on two quadric

surfaces

= straight line + twisted cubic

Does not happen for 6 or more random points!

Analytical Photogrammetry

Non-Linear Least-Squares Methods

• Newton
• Gauss-Newton
• Levenberg-Marquardt

Iterative, quadratically convergent in favorable situations

Applications: Mobile Robot Localization (Devy et al., 1997)

(Rothganger, Sudsang, Ponce, 2002)

Affine models: Weak perspective projection

When the scene relief is small compared its distance from the

camera, m can be taken constant: weak perspective projection.

Weak-Perspective Projection

Paraperspective Projection

What about Affine Cameras?

Orthographic Projection

Parallel Projection

More Affine Cameras

Weak-Perspective Projection Model

(p and P are in homogeneous coordinates)

p = A P + b

(neither p nor P is in hom. coordinates)

p = M P

(P is in homogeneous coordinates)

Theorem: All affine projection models can be

represented by affine projection matrices.

Definition: A 2x4 matrix M = [A b], where A is a rank-2 2x3 matrix, is called an affine projection matrix.

General form of the weak-perspective

projection equation:

Theorem: An affine projection matrix can be written uniquely (up to a sign amibguity) as a weak perspective projection matrix as defined by (1).

(1)

​

Image processing

• Filters and convolution
• Derivatives and edge detection
• The Canny edge detector
• Denoising, sparsity and dictionary learning
• Super-resolution

​

Basic Filters

Convolution

Linear filters = Weighted averages

• Represent the weights by a rectangular array F.

​

• Applying the filter to an image G is equivalent to performing a convolution:

​

R_ij = (F*G)_ij = ∑_u,v F_{i-u, j-v} G_{u, v}

• In the continuous case:

​

(f * g) (x,y) = s_u,v f(x-u,y-v) g(u,v) du dv

​

• Note: f*g=g*f.

Kernel:

Kernel:

Kernel:

15 x 15 matrix of value 1/225

Basic Properties

• Commutativity: f * g = g * f
• Associativity: (f * g) * h = f * (g * h)
• Linearity: (af + bg) * h = a f * h + b g * h
• Shift invariance: f_t * h = (f * h)_t
• Only operator both linear and shift invariant
• Differentiation:

Practicalities (discrete convolution)

• MATLAB: conv (1D) or conv2 (2D)
• Border issues:
• When applying convolution with a KxK kernel, the result is undefined for pixels closer than K pixels from the border of the image
• Options:

Wrap around

Expand/Pad

Crop

I_ideal

I_real

​

Defocus = Convolution?

NO: Blur depends on depth

1-D:

2-D:

Slight abuse of notation:

We ignore the normalization

constant such that

Gaussian filters

Kernel:

Simple Averaging

Gaussian Smoothing

Image Noise

Gaussian Smoothing to Remove Noise

Bottom line: The standard deviation of white noise is divided by k*sigma

Increasing σ

Basic Properties

• Gaussian removes “high-frequency” components from the image 🡪 “low pass” filter
• Larger σ remove more details
• Combination of 2 Gaussian filters is a Gaussian filter:

​

​

​

​

• Separable filter:

​

​

​

​

• Critical implication: Filtering with a NxN Gaussian kernel can be implemented as two convolutions of size N 🡪 reduction quadratic to linear 🡪 must be implemented that way

​

Note about Finite Kernel Support

• Gaussian function has infinite support

​

​

​

• In actual filtering, we have a finite kernel size

Oriented Gaussian Filters

• G_σ smoothes the image by the same amount in all directions
• If we have some information about preferred directions, we might want to smooth with some value σ₁ in the direction defined by the unit vector [a b] and by σ₂ in the direction defined by [c d]

​

​

​

​

​

​

​

• We can write this in a more compact form by using the standard multivariate Gaussian notation:

​

​

​

​

​

• The two (orthogonal) directions of filtering are given by the eigenvectors of Σ, the amount of smoothing is given by the square root of the corresponding eigenvalues of Σ.

Image Derivatives

• Image Derivatives
• Derivatives increase noise
• Derivative of Gaussian
• Laplacian of Gaussian (LOG)

Image Derivatives

• We want to compute, at each pixel (x,y) the derivatives:
• In the discrete case we could take the difference between the left and right pixels:

​

​

• Convolution of the image by
• s

​

• Problem: Increases noise

Difference between

Actual image values

True difference

(derivative)

Sum of the noises

-1

0

1

Finite differences

Finite differences responding to noise

Increasing zero-mean Gaussian noise

Smooth Derivatives

• Solution: First smooth the image by a Gaussian G_σ and then take derivatives:

​

​

• Applying the differentiation property of the convolution:

​

​

• Therefore, taking the derivative in x of the image can be done by convolution with the derivative of a Gaussian:

​

​

​

• Crucial property: The Gaussian derivative is also separable:

G

G_x

Derivative + Smoothing

Better but still blurs away edge information

Without smoothing

With smoothing

Applying the first derivative of Gaussian

I

There is ALWAYS a tradeoff between smoothing and

good edge localization!

Image with Edge

Edge Location

Image + Noise

Derivatives detect edge and noise

Smoothed derivative removes noise, but blurs edge