CS5670: Computer Vision

Feature invariance

Reading

• Szeliski (2^nd edition): 7.1

Announcements

• Project 1 code due tomorrow (Friday), 2/7, at 8pm
• Project 1 artifact due Monday, 2/10, at 8pm to CMSX
• Project 2 (Feature Detection & Matching) will be released on Tuesday, due Friday, February 21
• To be done in groups of 2
• Please start forming teams now!
• Please plan to work on the project early
• Take-home midterm planned after February Break
• Release: Thursday, Feb 27, due Tuesday, March 4
• Slip days cannot be used for the take-home midterm

Local features: main components

1. Detection: Identify the interest points

​

​

​

​

​

​

• Description: Extract vector feature descriptor surrounding each interest point.

​

​

​

​

​

​

• Matching: Determine correspondence between descriptors in two views

Kristen Grauman

Harris features (in red)

Image transformations

• Geometric

​

Rotation�

​

Scale��

• Photometric

Intensity change

Invariance and equivariance

• We want corner locations to be invariant to photometric transformations and equivariant to geometric transformations
• Invariance: image is transformed and corner locations do not change
• Equivariance: if we have two transformed versions of the same image, features should be detected in corresponding locations
• (Sometimes “invariant” and “equivariant” are both referred to as “invariant”)
• (Sometimes “equivariant” is called “covariant”)

Harris detector invariance properties: image translation

• Derivatives and window function are equivariant

Corner location is equivariant w.r.t. translation

Harris detector invariance properties: image rotation

Second moment ellipse rotates but its shape (i.e. eigenvalues) remains the same

Corner location is equivariant w.r.t. image rotation

Harris detector invariance properties: �Affine intensity change

• Only derivatives are used to compute Harris scores 🡪 invariance to intensity shift I → I + b

• Intensity scaling: I → a I

Partially invariant to affine intensity change

I → a I + b

Harris detector invariance properties: scaling

All points will be classified as edges

Corner

Neither invariant nor equivariant to scaling

Scale invariant detection

Suppose you’re looking for corners

​

​

​

​

​

​

​

Key idea: find scale that gives local maximum of f

• in both position and scale
• One definition of f: the Harris operator

Slide from Tinne Tuytelaars

Lindeberg et al, 1996

Slide from Tinne Tuytelaars

Lindeberg et al., 1996

Implementation

• Instead of computing f for larger and larger windows, we can implement using a fixed window size with a Gaussian pyramid

(sometimes need to create in-between levels, e.g. a ¾-size image)

Feature extraction: Corners and blobs

Another common definition of f

• The Laplacian of Gaussian (LoG)

(very similar to a Difference of Gaussians (DoG) – i.e. a Gaussian minus a slightly smaller Gaussian)

Laplacian of Gaussian

• “Blob” detector

​

​

​

​

​

​

• Find maxima and minima of LoG operator in space and scale

*

=

maximum

minima

Scale selection

• At what scale does the Laplacian achieve a maximum response for a binary circle of radius r?

r

image

Laplacian

Characteristic scale

• We define the characteristic scale as the scale that produces peak of Laplacian response

characteristic scale

T. Lindeberg (1998). "Feature detection with automatic scale selection." International Journal of Computer Vision 30 (2): pp 77--116.

Find local maxima in 3D position-scale space

K. Grauman, B. Leibe

⇒ List of � (x, y, s)

Scale-space blob detector: Example

Scale-space blob detector: Example

Scale-space blob detector: Example

Scale Invariant Detection

covariant

Note: The LoG and DoG operators are both rotation equivariant

Questions?

Feature descriptors

We know how to detect good points

Next question: How to match them?

​

​

​

​

​

​

​

​

Answer: Come up with a descriptor for each point, find similar descriptors between the two images

​

​

​

​

​

​

​

?