Image and Video Computing, Spring 2022 |
Time/Location: Tue/Thu 5:00pm-6:15pm in CAS 522
Course Number: CS 585
Instructors:
Prof. Bryan Plummer (bplum@bu.edu) office hours: Tue/Thu 11:00am-12:00pm in MCS-206
Teaching Fellows:
Hao Yu (haoyu@bu.edu) office hours: Fri 10:30am-11:30am in MCS 227
Praneeth Bogineni (pranchan@bu.edu) office hours: Mon 1:30pm-2:30pm in MCS 103
Graders: Zhongping Zhang, Yiwen Gu, Siqi Wang
Labs:
Fri 1:25pm (CAS 204A); 2:20pm (CAS 225); 3:35pm (CAS 225); 4:40pm (EMA 304)
Piazza: https://piazza.com/bu/spring2022/cs585 Please use Piazza, not email, for all questions, including grading or missing class etc. (use the private message to Instructors for such requests)
Gradescope: All assignments and regrade requests must be submitted through Gradescope.
Who should take this class? If you are a PhD, Master’s or upper-class undergrad interested in doing research in computer vision, this class is for you! It is a math- and programming-intensive class geared primarily towards graduate students. If you are not comfortable with the prerequisites, please DO NOT take this class before taking them. |
SCHEDULE*
Lectures will be recorded and posted later that day on piazza, slides will be available before class/lab at this link
Topic | Details | Assignments | |
Thu Jan 20 | Introduction | Why study IVC? Industry successes and current needs. Reading: Lecture 1 links, Wiki Intro or Horn Ch. 1. | |
Fri Jan 21 | NO LAB | ||
Tue Jan 25 | Cameras | Depth of field, field of view, lens aberrations, digital sensors. Reading: Depth of Field, Field of View | |
Thu Jan 27 | Light and Shading | In camera transformation of light, reflectance properties, diffuse and specular reflection, shape from shading. Reading: Lambertian Reflectance, Diffuse and Specular Reflection, Photometric Stereo | |
Fri Jan 28 | LAB1: Practicing image and video input/output in Python | ps1 out | |
Tue Feb 1 | Color | Image and video formats, color (RGB, HSV), Flood Fill Algorithm, Sequential Labeling Algorithm. Skin-color based face detection algorithm. Reading: Image Moments, Binary Image Analysis. Horn Ch. 3. | |
Thu Feb 3 | Template Matching | Template matching, background differencing. Similarity Functions (SSD, NCC), Motion: Template-based Tracking. Image Pyramids. Reading: template matching, normalized correlation. Skeleton | |
Fri Feb 4 | LAB2: Image Pyramid Template Matching | ||
Tue Feb 8 | Segmentation I | Thresholding techniques. Neighborhoods. Border following algorithm. Hausdorff Distance. Reading: Border following algorithm. Segmentation (most algorithms covered). Hausdorff distance. Thresholding. | ps1 due (5pm) ps2 out |
Thu Feb 10 | Edges | Image gradients, derivative of gaussian filters, Canny edge detector. Reading: Sobel, Prewitt, Roberts, Mexican Hat, Difference of Gaussians, Canny Edge Detector. Williams and Shah, 1992: paper, figures. | |
Fri Feb 11 | LAB3: Finding objects and their properties | ||
Tue Feb 15 | Keypoint detection | Corner detection, line fitting. Reading: Distinctive image features from scale-invariant keypoints | ps2 due (5pm) ps3 out |
Thu Feb 17 | Alignment | Fitting of transformations, robust alignment. Reading: Automatic panoramic image stitching | |
Fri Feb 18 | LAB4: Morphology | ||
Tue Feb 22 | NO CLASS (Monday schedule) | ps3 due (5pm) | |
Thu Feb 24 | Explainable AI (guest lecture by Vitali Petsiuk) DROP DATE | ||
Fri Feb 25 | LAB: Midterm review | ||
Tue Mar 1 | Midterm Review | Talk Writeup #1 due | |
Thu Mar 3 | Midterm (closed book, no electronics, you may bring a single 8“x11” sheet of paper with typed or handwritten notes on both sides) | covers everything up to and including Alignment; expect questions on material covered in lectures, problem sets, LABs and assigned reading | |
Fri Mar 4 | No Lab | ||
Tue Mar 8 | NO CLASS- Spring Recess | ||
Thu Mar 10 | NO CLASS- Spring Recess | ||
Fri Mar 11 | NO LAB- Spring Recess | ||
Tue Mar 15 | Optical flow | Motion field, brightness constancy constraint, Lucas-Kanade. Reading: Optical Flow, Horn 81, SZ 9.3 | |
Thu Mar 17 | Camera Calibration | Intrinsic camera parameters, extrinsic camera parameters, calibration, triangulation. Reading: SZ 2.1, 11.1, 11.2 | |
Fri Mar 18 | LAB: Midterm Review | ||
Tue Mar 22 | Single-view modeling | Calibration from vanishing points, single image measurements, single-view metrology. Reading: SZ 11.1, 11.2 | ps4 out |
Thu Mar 24 | Epipolar Geometry | Epipolar constraints, essential matrix, fundamental matrix. Reading: SZ 12.1 | |
Fri Mar 25 | LAB5: Optical Flow | ||
Tue Mar 29 | Structure from motion | Affine structure from motion, projective structure from motion, bundle adjustment. Reading: SZ 11.3 | ps4 due (5pm) ps5 out |
Thu Mar 31 | Two-view Stereo | Stereo setup, local matching, active stereo. Reading: SZ 12.3, 12.4 | |
Fri Apr 1 | LAB6: Computing vanishing points | ||
Tue Apr 5 | Multi-view Stereo | Plane sweep, depth map fusion, patch-based stereo. Reading: SZ 12.5, 12.7 | |
Thu Apr 7 | Introduction to Recognition | Recognition pipelines, recognition tasks and supervision. Reading: SZ 5.1, 5.2 | ps5 due (5pm) Challenge Out |
Fri Apr 8 | LAB7: Challenge discussion and introduction to Pytorch | ||
Tue Apr 12 | Neural Networks | Deep learning pipeline, perceptrons, gradient descent. Reading: SZ 5.3 | |
Thu Apr 14 | CNNs | Convolutional layers, pooling layers, common architectures. Reading: SZ 5.4 | Challenge Initial Submission Due |
Fri Apr 15 | LAB8: Using CNNs | ||
Tue Apr 19 | Detection | Sliding windows, region proposals, two stage detectors, one stage detectors. Reading: SZ 6.3 | |
Thu Apr 21 | Segmentation II | Fully convolutional networks, transposed convolutions, networks for dense prediction and instance segmentation. Reading: SZ 6.4 | Challenge Update Due |
Fri Apr 22 | LAB9: Challenge discussion | ||
Tue Apr 26 | NO CLASS | Talk Writeup #2 due | |
Thu Apr 28 | GANs | Formulation, class conditioned GANs, image-to-image translation. Reading: SZ 5.5.4 | Challenge Deadline |
Fri Apr 29 | LAB: Final Review | ||
Tue May 3 | Final Review | ||
Fri May 13 6-8pm | Final exam (closed book, no electronics, you may bring a two 8“x11” sheets of paper with typed or handwritten notes on both sides) | expect questions on material covered in the entire course in lectures, problem sets, LABs, and assigned reading |
*schedule is tentative and is subject to change.
SYLLABUS
This course is a rigorous introduction to modern computer vision concepts, techniques, and algorithms.
Course Prerequisites
This is a graduate course (open to upper-level undergraduates) and requires the following:
You should be very comfortable with programming in Python and be able to both write and understand code quickly. The above prerequisites can be replaced by Foundations of Data Science (CS 365).
Textbooks
I recommend some chapters in Robot Vision by BKP Horn, MIT Press. The book is not required. I will propose alternative reading material.
Recommended background reading on matrix calculus
Deliverables/Graded Work
The main graded work for the course is the midterm, final, problem sets and final challenge. There will be five problem sets (ps1-ps5), each consisting of written and programming problems, which are meant to prepare students for the two exams. Note: there are no team projects in this course. The course grade consists of the following:
Talk Reviews
You must attend at least two talks on subjects related to CS 585 and write a summary on each talk (at most 2 pages per summary). Your summary should give a problem definition, summarize the algorithms and results, discuss the work critically, and also briefly explain how the work relates to material discussed in class. You must submit your summary no later than the dates listed in the syllabus. Check your text for typographical and grammatical errors (e.g., use grammarly.com). You will lose points if you simply copy the speaker's abstract, your review is late, contains typographical and grammatical errors, does not contain a discussion, and does not contain a statement how the talk relates to class material.
Challenge
The challenge is an individual programming project that gives each student a chance to apply their knowledge and develop a computer vision solution to a real-world problem. The goal is to take what you learned in the weekly problem sets to the next level and spend a few weeks implementing computer vision algorithms from scratch and applying them to a challenging dataset. More information about the challenge will be released later on in the term.
Piazza
We will be using Piazza for class Q&A and posting problem sets. The system is highly catered to getting you help fast and efficiently from classmates and instructors. Rather than emailing questions to the teaching staff, we ask you to please post all your questions on Piazza. For private questions like grading please select just the “Instructors” in the “Post to:” field of the Piazza note and select the appropriate folder.
Software/Compute Resources
Programming assignments will be developed in the Python programming language, including iPython notebooks. No special computing resources are required besides a regular laptop.
Students may also have access to the SCC computing cluster, please request this from the instructors. See SCC Quick Start Guide for further information on getting started with the SCC.
Another resource you may want to use for some problem sets or the challenge is Google’s Colab, see https://colab.research.google.com/notebooks/welcome.ipynb
Help! My laptop broke!
If you find yourself in a situation where you need a back up machine, say, due to a broken laptop, the BU Common at the Mugar Library offers access to Microsoft Windows 10 virtual machines. The lab is co-located with the IT Help Center and the Print Center and hosts a broad selection of software including Python and Jupyter. Also note that the IT Help Center is an authorized repair center for most major laptop brands, and can often perform many common types of repairs within one day. They also offer temporary loaners for certain models of laptops while they are working on your machine.
COURSE POLICIES
Late Policy
Late problem sets will incur the following penalties
The lowest of all the problem set grades will be dropped. Exceptions to these policies can only be made in cases of significant medical or family emergencies, and should be documented.
Regrading Policy
All regrade requests must be made through Gradescope within 1 week of when grades are released. No regrade requests will be accepted after this date.
Copyright Notice
All course materials, including lecture slides/notes, videos/screenshots, homeworks, solutions, and solution code are copyrighted by Prof. Plummer or Prof. Margrit Betke. All rights are reserved. Other than for personal use by registered students, the course materials or any portion thereof may not be reproduced or used in any manner whatsoever (including posting on a public forum or website) without the express written permission of the authors. Acting otherwise would go against the ethical code of conduct expected of students at Boston University and from a legal standpoint it may even constitute a violation of copyright law.
Academic Honesty Policy
The instructors take academic honesty very seriously. Cheating, plagiarism and other misconduct will be subject to grading penalties up to failing the course. Students enrolled in the course are responsible for familiarizing themselves with the detailed BU policy, available here. In particular, plagiarism is defined as follows, and applies to all written materials and software, including material found online. Collaboration on homework only in the form of discussion is allowed, but should be acknowledged and you should always come up with your own solution rather than copy others’ solutions (which is defined as plagiarism).
Plagiarism: Representing the work of another as one’s own. Plagiarism includes but is not limited to the following: copying the answers of another student on an examination, copying or restating the work or ideas of another person or persons in any oral or written work (printed or electronic) without citing the appropriate source, and collaborating with someone else in an academic endeavor without acknowledging his or her contribution. Plagiarism can consist of acts of commission-appropriating the words or ideas of another-or omission failing to acknowledge/document/credit the source or creator of words or ideas (see below for a detailed definition of plagiarism). It also includes colluding with someone else in an academic endeavor without acknowledging his or her contribution, using audio or video footage that comes from another source (including work done by another student) without permission and acknowledgement of that source.
Prohibited behaviors include:
Incidents of academic misconduct will be reported to the Academic Conduct Committee (ACC). The ACC may suspend/expel students found guilty of misconduct. At a minimum, students who engage in misconduct will have their final grade reduced by one letter grade (e.g., from a B to a C).
Religious Observance
Students are permitted to be absent from class, including classes involving examinations, labs, excursions, and other special events, for purposes of religious observance. In-class, take-home and lab assignments, and other work shall be made up in consultation with the student’s instructors. More details on BU’s religious observance policy are available here.
COVID-19
There is no remote component of this class. If you miss class due to COVID-19 or any other health related absence, please check the website for lecture recordings and other posted materials. Masks are required and face coverings must be worn over the mouth and nose at all times when in public spaces on campus, including classrooms. Students should be prepared to show proof that they are compliant with health attestations and testing in order to attend class. All students are expected to follow all university guidelines with respect to daily symptom checks, testing, social distancing, and mask wearing when they leave their dorm or home. For a detailed description of official BU policies regarding COVID, please visit: https://www.bu.edu/back2bu/.