Published using Google Docs
Image and Video Computing, Spring 2022
Updated automatically every 5 minutes

Image and Video Computing, Spring 2022

Schedule

Syllabus

Slides

Policies

Time/Location: Tue/Thu  5:00pm-6:15pm in CAS 522

Course Number: CS 585

Instructors: 

Prof. Bryan Plummer (bplum@bu.edu) office hours: Tue/Thu 11:00am-12:00pm in MCS-206

Teaching Fellows: 

         Hao Yu (haoyu@bu.edu) office hours: Fri 10:30am-11:30am in MCS 227

Praneeth Bogineni (pranchan@bu.edu) office hours: Mon 1:30pm-2:30pm in MCS 103

Graders: Zhongping Zhang, Yiwen Gu, Siqi Wang

Labs:

Fri 1:25pm (CAS 204A); 2:20pm (CAS 225); 3:35pm (CAS 225); 4:40pm (EMA 304)

Piazza:  https://piazza.com/bu/spring2022/cs585  Please use Piazza, not email, for all questions, including grading or missing class etc. (use the private message to Instructors for such requests)

Gradescope: All assignments and regrade requests must be submitted through Gradescope.

Who should take this class?

If you are a PhD, Master’s or upper-class undergrad interested in doing research in computer vision, this class is for you! It is a math- and programming-intensive class geared primarily towards graduate students. If you are not comfortable with the prerequisites, please DO NOT take this class before taking them.

SCHEDULE*

Lectures will be recorded and posted later that day on piazza, slides will be available before class/lab at this link

Topic

Details

Assignments

Thu

Jan 20

Introduction

Why study IVC? Industry successes and current needs. Reading: Lecture 1 links, Wiki Intro or Horn Ch. 1.

Fri

Jan 21

NO LAB

Tue

Jan 25

Cameras

Depth of field, field of view, lens aberrations, digital sensors.  Reading: Depth of Field, Field of View

Thu

Jan 27

Light and Shading

In camera transformation of light, reflectance properties, diffuse and specular reflection, shape from shading.  Reading: Lambertian Reflectance, Diffuse and Specular Reflection, Photometric Stereo

Fri

Jan 28

LAB1: Practicing image and video input/output in Python

ps1 out

Tue Feb 1

Color

Image and video formats, color (RGB, HSV), Flood Fill Algorithm, Sequential Labeling Algorithm. Skin-color based face detection algorithm. Reading: Image Moments, Binary Image Analysis. Horn Ch. 3.

Thu Feb 3

Template Matching

Template matching, background differencing. Similarity Functions (SSD, NCC), Motion: Template-based Tracking. Image Pyramids.  Reading: template matching, normalized correlation. Skeleton

Fri

Feb 4

LAB2: Image Pyramid Template Matching

Tue Feb 8

Segmentation I

Thresholding techniques. Neighborhoods. Border following algorithm. Hausdorff Distance. Reading: Border following algorithm. Segmentation (most algorithms covered). Hausdorff distance. Thresholding.

ps1 due (5pm)

ps2 out

Thu Feb 10

Edges

Image gradients, derivative of gaussian filters, Canny edge detector. Reading: Sobel, Prewitt, Roberts, Mexican Hat, Difference of Gaussians, Canny Edge Detector. Williams and Shah, 1992: paper, figures. 

Fri

Feb 11

LAB3: Finding objects and their properties

Tue Feb 15

Keypoint detection

Corner detection, line fitting. Reading: Distinctive image features from scale-invariant keypoints

ps2 due (5pm)

ps3 out

Thu Feb 17

Alignment

Fitting of transformations, robust alignment. Reading: Automatic panoramic image stitching

Fri

Feb 18

LAB4: Morphology

Tue Feb 22

NO CLASS (Monday schedule)

ps3 due (5pm)

Thu Feb 24

Explainable AI (guest lecture by Vitali Petsiuk)

DROP DATE

Fri

Feb 25

LAB: Midterm review

Tue Mar 1

Midterm Review

Talk Writeup #1 due

Thu Mar 3

Midterm

(closed book, no electronics, you may bring a single 8“x11” sheet of paper with typed or handwritten notes on both sides)

covers everything up to and including Alignment; expect questions on material covered in lectures, problem sets, LABs and assigned reading

Fri

Mar 4

No Lab

Tue Mar 8

NO CLASS- Spring Recess

Thu Mar 10

NO CLASS- Spring Recess

Fri

Mar 11

NO LAB- Spring Recess

Tue Mar 15

Optical flow

Motion field, brightness constancy constraint, Lucas-Kanade. Reading: Optical Flow, Horn 81, SZ 9.3

Thu Mar 17

Camera Calibration

Intrinsic camera parameters, extrinsic camera parameters, calibration, triangulation. Reading: SZ 2.1, 11.1, 11.2

Fri

Mar 18

LAB: Midterm Review

Tue Mar 22

Single-view modeling

Calibration from vanishing points, single image measurements, single-view metrology. Reading: SZ 11.1, 11.2

ps4 out

Thu Mar 24

Epipolar Geometry

Epipolar constraints, essential matrix, fundamental matrix. Reading: SZ 12.1

Fri

Mar 25

LAB5: Optical Flow

Tue Mar 29

Structure from motion

Affine structure from motion, projective structure from motion, bundle adjustment. Reading: SZ 11.3

ps4 due (5pm)

ps5 out

Thu Mar 31

Two-view Stereo

Stereo setup, local matching, active stereo. Reading: SZ 12.3, 12.4

Fri

Apr 1

LAB6: Computing vanishing points

Tue

Apr 5

Multi-view Stereo

Plane sweep, depth map fusion, patch-based stereo. Reading: SZ 12.5, 12.7

Thu

Apr 7

Introduction to Recognition

Recognition pipelines, recognition tasks and supervision. Reading: SZ 5.1, 5.2

ps5 due (5pm)

Challenge Out

Fri

Apr 8

LAB7: Challenge discussion and introduction to Pytorch

Tue Apr 12

Neural Networks

Deep learning pipeline, perceptrons, gradient descent. Reading: SZ 5.3

Thu Apr 14

CNNs

Convolutional layers, pooling layers, common architectures.  Reading: SZ 5.4

Challenge Initial Submission Due

Fri

Apr 15

LAB8: Using CNNs

Tue

Apr 19

Detection

Sliding windows, region proposals, two stage detectors, one stage detectors. Reading: SZ 6.3

Thu

Apr 21

Segmentation II

Fully convolutional networks, transposed convolutions, networks for dense prediction and instance segmentation. Reading: SZ 6.4

Challenge Update Due

Fri

Apr 22

LAB9: Challenge discussion

Tue

Apr 26

NO CLASS

Talk Writeup #2 due

Thu

Apr 28

GANs

Formulation, class conditioned GANs, image-to-image translation. Reading: SZ 5.5.4

Challenge Deadline

Fri

Apr 29

LAB: Final Review

Tue May 3

Final Review

Fri

May 13 6-8pm

Final exam

(closed book, no electronics, you may bring a two 8“x11” sheets of paper with typed or handwritten notes on both sides)

expect questions on material covered in the entire course in lectures, problem sets, LABs, and assigned reading

*schedule is tentative and is subject to change.

SYLLABUS

This course is a rigorous introduction to modern computer vision concepts, techniques, and algorithms.

Course Prerequisites

This is a graduate course (open to upper-level undergraduates) and requires the following:

You should be very comfortable with programming in Python and be able to both write and understand code quickly. The above prerequisites can be replaced by Foundations of Data Science (CS 365).

Textbooks

I recommend some chapters in Robot Vision by BKP Horn, MIT Press. The book is not required. I will propose alternative reading material.

Recommended background reading on matrix calculus

Deliverables/Graded Work

The main graded work for the course is the midterm, final, problem sets and final challenge. There will be five problem sets (ps1-ps5), each consisting of written and programming problems, which are meant to prepare students for the two exams. Note: there are no team projects in this course. The course grade consists of the following:

Talk Reviews

You must attend at least two talks on subjects related to CS 585 and write a summary on each talk (at most 2 pages per summary). Your summary should give a problem definition, summarize the algorithms and results, discuss the work critically, and also briefly explain how the work relates to material discussed in class. You must submit your summary no later than the dates listed in the syllabus. Check your text for typographical and grammatical errors (e.g., use grammarly.com). You will lose points if you simply copy the speaker's abstract, your review is late, contains typographical and grammatical errors, does not contain a discussion, and does not contain a statement how the talk relates to class material.

Challenge

The challenge is an individual programming project that gives each student a chance to apply their knowledge and develop a computer vision solution to a real-world problem. The goal is to take what you learned in the weekly problem sets to the next level and spend a few weeks implementing computer vision algorithms from scratch and applying them to a challenging dataset. More information about the challenge will be released later on in the term.

Piazza

We will be using Piazza for class Q&A and posting problem sets. The system is highly catered to getting you help fast and efficiently from classmates and instructors. Rather than emailing questions to the teaching staff, we ask you to please post all your questions on Piazza. For private questions like grading please select just the “Instructors” in the “Post to:” field of the Piazza note and select the appropriate folder.

Software/Compute Resources

Programming assignments will be developed in the Python programming language, including iPython notebooks. No special computing resources are required besides a regular laptop.

Students may also have access to the SCC computing cluster, please request this from the instructors. See SCC Quick Start Guide for further information on getting started with the SCC.

Another resource you may want to use for some problem sets or the challenge is Google’s Colab, see https://colab.research.google.com/notebooks/welcome.ipynb

Help! My laptop broke!

If you find yourself in a situation where you need a back up machine, say, due to a broken laptop, the BU Common at the Mugar Library offers access to Microsoft Windows 10 virtual machines. The lab is co-located with the IT Help Center and the Print Center and hosts a broad selection of software including Python and Jupyter. Also note that the IT Help Center is an authorized repair center for most major laptop brands, and can often perform many common types of repairs within one day.  They also offer temporary loaners for certain models of laptops while they are working on your machine.

COURSE POLICIES

Late Policy

Late problem sets will incur the following penalties

The lowest of all the problem set grades will be dropped. Exceptions to these policies can only be made in cases of significant medical or family emergencies, and should be documented.

Regrading Policy

All regrade requests must be made through Gradescope within 1 week of when grades are released.  No regrade requests will be accepted after this date.

Copyright Notice

All course materials, including lecture slides/notes, videos/screenshots, homeworks, solutions, and solution code are copyrighted by Prof. Plummer or Prof. Margrit Betke. All rights are reserved. Other than for personal use by registered students, the course materials or any portion thereof may not be reproduced or used in any manner whatsoever (including posting on a public forum or website) without the express written permission of the authors. Acting otherwise would go against the ethical code of conduct expected of students at Boston University and from a legal standpoint it may even constitute a violation of copyright law.

Academic Honesty Policy

The instructors take academic honesty very seriously. Cheating, plagiarism and other misconduct will be subject to grading penalties up to failing the course.  Students enrolled in the course are responsible for familiarizing themselves with the detailed BU policy, available here. In particular, plagiarism is defined as follows, and applies to all written materials and software, including material found online. Collaboration on homework only in the form of discussion is allowed, but should be acknowledged and you should always come up with your own solution rather than copy others’ solutions (which is defined as plagiarism).

Plagiarism: Representing the work of another as one’s own. Plagiarism includes but is not limited to the following: copying the answers of another student on an examination, copying or restating the work or ideas of another person or persons in any oral or written work (printed or electronic) without citing the appropriate source, and collaborating with someone else in an academic endeavor without acknowledging his or her contribution. Plagiarism can consist of acts of commission-appropriating the words or ideas of another-or omission failing to acknowledge/document/credit the source or creator of words or ideas (see below for a detailed definition of plagiarism). It also includes colluding with someone else in an academic endeavor without acknowledging his or her contribution, using audio or video footage that comes from another source (including work done by another student) without permission and acknowledgement of that source.

Prohibited behaviors include:

Incidents of academic misconduct will be reported to the Academic Conduct Committee (ACC). The ACC may suspend/expel students found guilty of misconduct. At a minimum, students who engage in misconduct will have their final grade reduced by one letter grade (e.g., from a B to a C).

Religious Observance

Students are permitted to be absent from class, including classes involving examinations, labs, excursions, and other special events, for purposes of religious observance.  In-class, take-home and lab assignments, and other work shall be made up in consultation with the student’s instructors. More details on BU’s religious observance policy are available here.

COVID-19

There is no remote component of this class. If you miss class due to COVID-19 or any other health related absence, please check the website for lecture recordings and other posted materials. Masks are required and face coverings must be worn over the mouth and nose at all times when in public spaces on campus, including classrooms. Students should be prepared to show proof that they are compliant with health attestations and testing in order to attend class. All students are expected to follow all university guidelines with respect to daily symptom checks, testing, social distancing, and mask wearing when they leave their dorm or home. For a detailed description of official BU policies regarding COVID, please visit: https://www.bu.edu/back2bu/.