Image and Video Computing, Spring 2022
Time/Location: Tue/Thu 5:00pm-6:15pm in CAS 522
Course Number: CS 585
Prof. Bryan Plummer (email@example.com) office hours: Tue/Thu 11:00am-12:00pm in MCS-206
Hao Yu (firstname.lastname@example.org) office hours: Fri 10:30am-11:30am in MCS 227
Praneeth Bogineni (email@example.com) office hours: Mon 1:30pm-2:30pm in MCS 103
Graders: Zhongping Zhang, Yiwen Gu, Siqi Wang
Fri 1:25pm (CAS 204A); 2:20pm (CAS 225); 3:35pm (CAS 225); 4:40pm (EMA 304)
Piazza: https://piazza.com/bu/spring2022/cs585 Please use Piazza, not email, for all questions, including grading or missing class etc. (use the private message to Instructors for such requests)
Gradescope: All assignments and regrade requests must be submitted through Gradescope.
Who should take this class?
If you are a PhD, Master’s or upper-class undergrad interested in doing research in computer vision, this class is for you! It is a math- and programming-intensive class geared primarily towards graduate students. If you are not comfortable with the prerequisites, please DO NOT take this class before taking them.
Light and Shading
LAB1: Practicing image and video input/output in Python
Tue Feb 1
Thu Feb 3
LAB2: Image Pyramid Template Matching
Tue Feb 8
ps1 due (5pm)
Thu Feb 10
Image gradients, derivative of gaussian filters, Canny edge detector. Reading: Sobel, Prewitt, Roberts, Mexican Hat, Difference of Gaussians, Canny Edge Detector. Williams and Shah, 1992: paper, figures.
LAB3: Finding objects and their properties
Tue Feb 15
Corner detection, line fitting. Reading: Distinctive image features from scale-invariant keypoints
ps2 due (5pm)
Thu Feb 17
Fitting of transformations, robust alignment. Reading: Automatic panoramic image stitching
Tue Feb 22
NO CLASS (Monday schedule)
ps3 due (5pm)
Thu Feb 24
Explainable AI (guest lecture by Vitali Petsiuk)
LAB: Midterm review
Tue Mar 1
Talk Writeup #1 due
Thu Mar 3
(closed book, no electronics, you may bring a single 8“x11” sheet of paper with typed or handwritten notes on both sides)
covers everything up to and including Alignment; expect questions on material covered in lectures, problem sets, LABs and assigned reading
Tue Mar 8
NO CLASS- Spring Recess
Thu Mar 10
NO CLASS- Spring Recess
NO LAB- Spring Recess
Tue Mar 15
Thu Mar 17
Intrinsic camera parameters, extrinsic camera parameters, calibration, triangulation. Reading: SZ 2.1, 11.1, 11.2
LAB: Midterm Review
Tue Mar 22
Calibration from vanishing points, single image measurements, single-view metrology. Reading: SZ 11.1, 11.2
Thu Mar 24
Epipolar constraints, essential matrix, fundamental matrix. Reading: SZ 12.1
LAB5: Optical Flow
Tue Mar 29
Structure from motion
Affine structure from motion, projective structure from motion, bundle adjustment. Reading: SZ 11.3
ps4 due (5pm)
Thu Mar 31
Stereo setup, local matching, active stereo. Reading: SZ 12.3, 12.4
LAB6: Computing vanishing points
Plane sweep, depth map fusion, patch-based stereo. Reading: SZ 12.5, 12.7
Introduction to Recognition
Recognition pipelines, recognition tasks and supervision. Reading: SZ 5.1, 5.2
ps5 due (5pm)
LAB7: Challenge discussion and introduction to Pytorch
Tue Apr 12
Deep learning pipeline, perceptrons, gradient descent. Reading: SZ 5.3
Thu Apr 14
Convolutional layers, pooling layers, common architectures. Reading: SZ 5.4
Challenge Initial Submission Due
LAB8: Using CNNs
Sliding windows, region proposals, two stage detectors, one stage detectors. Reading: SZ 6.3
Fully convolutional networks, transposed convolutions, networks for dense prediction and instance segmentation. Reading: SZ 6.4
Challenge Update Due
LAB9: Challenge discussion
Talk Writeup #2 due
Formulation, class conditioned GANs, image-to-image translation. Reading: SZ 5.5.4
LAB: Final Review
Tue May 3
May 13 6-8pm
(closed book, no electronics, you may bring a two 8“x11” sheets of paper with typed or handwritten notes on both sides)
expect questions on material covered in the entire course in lectures, problem sets, LABs, and assigned reading
*schedule is tentative and is subject to change.
This course is a rigorous introduction to modern computer vision concepts, techniques, and algorithms.
This is a graduate course (open to upper-level undergraduates) and requires the following:
You should be very comfortable with programming in Python and be able to both write and understand code quickly. The above prerequisites can be replaced by Foundations of Data Science (CS 365).
I recommend some chapters in Robot Vision by BKP Horn, MIT Press. The book is not required. I will propose alternative reading material.
Recommended background reading on matrix calculus
The main graded work for the course is the midterm, final, problem sets and final challenge. There will be five problem sets (ps1-ps5), each consisting of written and programming problems, which are meant to prepare students for the two exams. Note: there are no team projects in this course. The course grade consists of the following:
You must attend at least two talks on subjects related to CS 585 and write a summary on each talk (at most 2 pages per summary). Your summary should give a problem definition, summarize the algorithms and results, discuss the work critically, and also briefly explain how the work relates to material discussed in class. You must submit your summary no later than the dates listed in the syllabus. Check your text for typographical and grammatical errors (e.g., use grammarly.com). You will lose points if you simply copy the speaker's abstract, your review is late, contains typographical and grammatical errors, does not contain a discussion, and does not contain a statement how the talk relates to class material.
The challenge is an individual programming project that gives each student a chance to apply their knowledge and develop a computer vision solution to a real-world problem. The goal is to take what you learned in the weekly problem sets to the next level and spend a few weeks implementing computer vision algorithms from scratch and applying them to a challenging dataset. More information about the challenge will be released later on in the term.
We will be using Piazza for class Q&A and posting problem sets. The system is highly catered to getting you help fast and efficiently from classmates and instructors. Rather than emailing questions to the teaching staff, we ask you to please post all your questions on Piazza. For private questions like grading please select just the “Instructors” in the “Post to:” field of the Piazza note and select the appropriate folder.
Programming assignments will be developed in the Python programming language, including iPython notebooks. No special computing resources are required besides a regular laptop.
Another resource you may want to use for some problem sets or the challenge is Google’s Colab, see https://colab.research.google.com/notebooks/welcome.ipynb
Help! My laptop broke!
If you find yourself in a situation where you need a back up machine, say, due to a broken laptop, the BU Common at the Mugar Library offers access to Microsoft Windows 10 virtual machines. The lab is co-located with the IT Help Center and the Print Center and hosts a broad selection of software including Python and Jupyter. Also note that the IT Help Center is an authorized repair center for most major laptop brands, and can often perform many common types of repairs within one day. They also offer temporary loaners for certain models of laptops while they are working on your machine.
Late problem sets will incur the following penalties
The lowest of all the problem set grades will be dropped. Exceptions to these policies can only be made in cases of significant medical or family emergencies, and should be documented.
All regrade requests must be made through Gradescope within 1 week of when grades are released. No regrade requests will be accepted after this date.
All course materials, including lecture slides/notes, videos/screenshots, homeworks, solutions, and solution code are copyrighted by Prof. Plummer or Prof. Margrit Betke. All rights are reserved. Other than for personal use by registered students, the course materials or any portion thereof may not be reproduced or used in any manner whatsoever (including posting on a public forum or website) without the express written permission of the authors. Acting otherwise would go against the ethical code of conduct expected of students at Boston University and from a legal standpoint it may even constitute a violation of copyright law.
Academic Honesty Policy
The instructors take academic honesty very seriously. Cheating, plagiarism and other misconduct will be subject to grading penalties up to failing the course. Students enrolled in the course are responsible for familiarizing themselves with the detailed BU policy, available here. In particular, plagiarism is defined as follows, and applies to all written materials and software, including material found online. Collaboration on homework only in the form of discussion is allowed, but should be acknowledged and you should always come up with your own solution rather than copy others’ solutions (which is defined as plagiarism).
Plagiarism: Representing the work of another as one’s own. Plagiarism includes but is not limited to the following: copying the answers of another student on an examination, copying or restating the work or ideas of another person or persons in any oral or written work (printed or electronic) without citing the appropriate source, and collaborating with someone else in an academic endeavor without acknowledging his or her contribution. Plagiarism can consist of acts of commission-appropriating the words or ideas of another-or omission failing to acknowledge/document/credit the source or creator of words or ideas (see below for a detailed definition of plagiarism). It also includes colluding with someone else in an academic endeavor without acknowledging his or her contribution, using audio or video footage that comes from another source (including work done by another student) without permission and acknowledgement of that source.
Prohibited behaviors include:
Incidents of academic misconduct will be reported to the Academic Conduct Committee (ACC). The ACC may suspend/expel students found guilty of misconduct. At a minimum, students who engage in misconduct will have their final grade reduced by one letter grade (e.g., from a B to a C).
Students are permitted to be absent from class, including classes involving examinations, labs, excursions, and other special events, for purposes of religious observance. In-class, take-home and lab assignments, and other work shall be made up in consultation with the student’s instructors. More details on BU’s religious observance policy are available here.
There is no remote component of this class. If you miss class due to COVID-19 or any other health related absence, please check the website for lecture recordings and other posted materials. Masks are required and face coverings must be worn over the mouth and nose at all times when in public spaces on campus, including classrooms. Students should be prepared to show proof that they are compliant with health attestations and testing in order to attend class. All students are expected to follow all university guidelines with respect to daily symptom checks, testing, social distancing, and mask wearing when they leave their dorm or home. For a detailed description of official BU policies regarding COVID, please visit: https://www.bu.edu/back2bu/.