1 of 37

Introduction to computer vision 10

Jean Ponce

jean.ponce@ens.fr

​

Zuhaib Akhtar za2023@nyu.edu

Ayush Jain aj3152@nyu.edu

​

​

​

Slides will be available after classes

2 of 37

Structure from motion

  • Ambitguities
  • The affine SFM problem
  • A geometric approach to two-view affine SFM
  • An analytical trick for two-view affine SFM
  • Multi-view affine SFM
  • Projective two-view SFM
  • Projective multi-view SFM
  • Auto-calibration

3 of 37

Euclidean (= similarity) ambiguity

4 of 37

If M is unconstrained:�Projective ambiguity

 

5 of 37

Projective ambiguity

6 of 37

Structure from motion

  • Let us now look at simpler, affine cameras

center at�infinity

7 of 37

The Affine Structure-from-Motion Problem

 

Problem: estimate the m 2x4 matrices M and

the n positions P from the mn correspondences p .

i

j

ij

2mn equations in 8m+3n unknowns

Overconstrained problem, that can be solved

using (non-linear) least squares!

8 of 37

The Affine Ambiguity of Affine SFM

If M and P are solutions,

i

j

So are M’ and P’ where

i

j

and

Q is an affine

transformation.

When the intrinsic parameters are unknown

9 of 37

Types of ambiguity

Projective

15dof

Affine

12dof

Similarity

7dof

Euclidean

6dof

Preserves incidence, tangency, cross-ratios

Preserves parallellism, ratios of lengths along parallel ßlines

Preserves angles, ratios of length

Preserves angles, lengths

  • With no constraints on the camera calibration matrix or on the scene, we get a projective reconstruction
  • Need additional information to upgrade the reconstruction to affine, similarity, or Euclidean

10 of 37

Affine Structure from Motion

Reprinted with permission from “Affine Structure from Motion,” by J.J. (Koenderink and A.J.Van Doorn, Journal of the Optical Society of America A,

8:377-385 (1990). © 1990 Optical Society of America.

Given m pictures of n points, can we recover

  • the three-dimensional configuration of these points?
  • the camera configurations?

(structure)

(motion)

11 of 37

Affine projections induce affine transformations from planes

onto their images.

Planar

affine

6dof

Preserves parallellism, ratios of lengths along parallel lines

12 of 37

 

barycentric coordinates

13 of 37

Affine coordinate are planar affine invariants

​

​

​

​

​

​

​

​

​

..and so are (signed) ratios of distances along

parallel lines

​

14 of 37

Affine coordinate are planar affine invariants

and thus affine projection invariants..

​

​

​

​

​

​

​

​

..and so are (signed) ratios of distances along

parallel lines

​

15 of 37

Geometric affine scene reconstruction from two images

(Koenderink and Van Doorn, 1991).

16 of 37

Geometric affine scene reconstruction from two images

(Koenderink and Van Doorn, 1991).

17 of 37

Geometric affine scene reconstruction from two images

(Koenderink and Van Doorn, 1991).

 

 

18 of 37

Geometric affine scene reconstruction from two images

(Koenderink and Van Doorn, 1991).

 

 

19 of 37

Geometric affine scene reconstruction from two images

(Koenderink and Van Doorn, 1991).

 

 

20 of 37

Geometric affine scene reconstruction from two images

(Koenderink and Van Doorn, 1991).

 

 

21 of 37

The Affine Epipolar Constraint

 

22 of 37

The Affine Epipolar Constraint

Note: the epipolar lines are parallel.

23 of 37

Affine Epipolar Geometry

24 of 37

The Affine Fundamental Matrix

where

25 of 37

Without normalization

With normalization

Mean errors:

10.0pixel

9.1pixel

Mean errors:

1.0pixel

0.9pixel

Perspective case..

26 of 37

Mean errors: 3.24 and 3.15pixel (without normalization

160.92 and 158.54pixel).

Affine case..

27 of 37

The Affine Epipolar Constraint

Note: the epipolar lines are parallel.

28 of 37

An Affine Trick..

29 of 37

An Affine Trick..

Algebraic Scene Reconstruction Method

30 of 37

Affine reconstruction. Mean relative error: 3.2%

31 of 37

The Affine Structure-from-Motion Problem

 

Problem: estimate the m 2x4 matrices M and

the n positions P from the mn correspondences p .

i

j

ij

2mn equations in 8m+3n unknowns

Overconstrained problem, that can be solved

using (non-linear) least squares!

32 of 37

Multiple affine images (Tomasi & Kanade, 1992)

Idea 1: pick one of the points (or their center of mass)

as the origin to reduce the problem to a bilinear one

(WP projection preserves center of mass)

33 of 37

Multiple affine images (Tomasi & Kanade, 1992)

Idea 1: pick one of the points (or their center of mass)

as the origin to reduce the problem to a bilinear one

 

Idea 2: Use SVD to decompose D into A and P factors

(WP projection preserves center of mass)

34 of 37

Singular Value Decomposition

 

 

35 of 37

Singular Value Decomposition

 

Theorem: When A has rank p<n, the SVD takes the form

 

 

36 of 37

Multiple affine images (Tomasi & Kanade, 1992)

 

 

37 of 37

Affine reconstruction. Mean relative error: 2.8%