1 of 21

Lab 2

Class Logistics & HW1 Review

2 of 21

Class and Lab timings

  • All classes and labs are in-person only, unless there is a change due to unexpected circumstances.

​

  • DS-GA 3001-009 (Lecture)�Mondays 4:55pm-6:35pm�Location: 208, Silver Center for Arts and Science, 32 Waverly Pl, New York, NY 10011 (SILV 208, location link)

​

  • DS-GA 3001.010 (Lab)�Thursdays 5:55pm-6:45pm�Location: 60 Fifth Avenue, New York, NY 10011 (60FA 150, location link)

3 of 21

Office Hours and Campuswire

  • Jean Ponce (jean.ponce@inria.fr)�Wednesdays, 2:00pm-3:30pm�Room 300, 60 5th Ave, New York, NY 10011

​

  • You can have additional OH with instructors or TA by appointment via mail.

​

  • Recommended to start discussions on Campuswire rather than waiting for OH.

​

  • If you are don’t have the invite, you can join using link. Group Join Code: 7367

4 of 21

Grading

​

  • Four programming assignments/homework (50% of the grade)

  • Final project (40% of the grade)

​

  • Class participation and attendance (5%)

​

  • Lab participation and attendance (5%)

5 of 21

Homework

​

  • Homework 1 on camera calibration. Link Due on Feb. 17.

​

  • Homework 2 on Canny edge detector. Due on Mar. 10.

​

  • Homework 3 on mean shift. Due on Apr. 11.

​

  • Homework 4 on neural networks. Due on May 5.

​

  • Homework will be released on Brightspace (under the Assignments tab) and the course website.

​

6 of 21

Project and Class Participation

​

  • Homework 1 on camera calibration. Link Due on Feb. 17.

​

  • Homework 2 on Canny edge detector. Due on Mar. 10.

​

  • Homework 3 on mean shift. Due on Apr. 11.

​

  • Homework 4 on neural networks. Due on May 5.

​

  • Homework will be released on Brightspace (under the Assignments tab) and the course website.

​

7 of 21

Homework 1 & Intrinsic/Extrinsic Parameters

Slides courtesy: Irmak Guzey

8 of 21

Homework 1

  • Find the projection matrix
  • Extract intrinsic and extrinsic matrices from projection matrix
  • Plot transformed image

9 of 21

Extrinsic and Intrinsic parameters

Image credits: https://towardsdatascience.com/what-are-intrinsic-and-extrinsic-camera-parameters-in-computer-vision-7071b72fb8ec

�

  • Extrinsic Parameters define the location and orientation of the camera with respect to the world frame.

​

  • Intrinsic Parameters: allow a mapping between camera coordinates and pixel coordinates in the image frame.

10 of 21

Finding the projection matrix

​

Here, m (3x4) is the projection matrix of the system

From Eq 1.20, we can have the image positions (xi, yi) of n fiducial points P i(i = 1, . . . , n)

with known homogeneous coordinate vectors Pi as,

​

11 of 21

Finding the projection matrix

Here, m (3x4) is the projection matrix of the system

12 of 21

Finding the projection matrix

Here, m (3x4) is the projection matrix of the system

We need to solve this equation where P and m can be represented as:

How?

13 of 21

Singular Value Decomposition for Linear Equations

Image taken from https://en.wikipedia.org/wiki/Singular_value_decomposition

https://numpy.org/doc/stable/reference/generated/numpy.linalg.svd.html

When we have a transformation (or any matrix), SVD is used to decompose this transformation matrix into 3 different parts.

Details are not in the scope of this course.

14 of 21

Extracting Parameters from Projection Matrix

Theta: Angle between the physical and normalized image place.

When there is a skew.

​

Wanted: Since we know the projection matrix, if we can write the projection matrix with the values of intrinsic and extrinsic matrix, we can extract these matrices.

15 of 21

Extracting Intrinsic Parameters from Projection Matrix

16 of 21

Extracting Intrinsic Parameters from Projection Matrix

If the principal center of the retina is not in the origin of the camera

17 of 21

Extracting Intrinsic Parameters from Projection Matrix

If the principal center of the retina is not in the origin of the camera

​

If the angle between physical and normalized image planes is not 90 degrees.

18 of 21

Extracting Intrinsic Parameters from Projection Matrix

K: Intrinsic Matrix With 5 parameters:

  • alpha
  • beta
  • theta
  • x_0
  • y_0

19 of 21

Extracting Extrinsic Parameters from Projection Matrix

Then Projection Matrix can be written as a function of its 5 intrinsic parameters (mentioned prev), 3 rows of the rotation matrix and 3 rows of the translation matrix.

20 of 21

Extracting Extrinsic Parameters from Projection Matrix

Then Projection Matrix can be written as a function of its 5 intrinsic parameters (mentioned prev),

3 rows of the rotation matrix and 3 values of the translation matrix. Where rs and ts represent the rows of these matrices.

21 of 21

Reading

  • https://ftp.cs.toronto.edu/pub/psala/VM/camera-parameters.pdf
  • https://towardsdatascience.com/what-are-intrinsic-and-extrinsic-camera-parameters-in-computer-vision-7071b72fb8ec