Published using Google Docs
Applied Machine Learning, Fall 2023
Updated automatically every 5 minutes

Applied Machine Learning, Fall 2023

Schedule

Syllabus

Slides

Policies

Time/Location: Tue/Thu  3:30pm-4:45pm in STH B19

Course Number: CS 541

Instructors: 

Prof. Bryan Plummer (bplum@bu.edu) office hours: Tue  2-3pm, Wed 3-4pm in CDS844

Teaching Fellows: 

Arijit Ray (array@bu.edu) office hours: Mon 4-6pm in CDS 950

Graders: Sha (Stan) Lai, Nishant Nadkarni

Labs:

Wed 12:20pm and 1:25pm in CDS 364

Piazza:  piazza.com/bu/fall2023/cs541  Please use Piazza, not email, for all questions, including grading or missing class etc. (use the private message to Instructors for such requests)

Gradescope: All assignments and regrade requests must be submitted through Gradescope.

Who should take this class?

If you are a PhD, Master’s or upper-class undergrad interested in using machine learning, but not necessarily to conduct machine learning research, this class is for you! If you are not comfortable with the prerequisites, please DO NOT take this class before taking them.

SCHEDULE*

Lecture slides will be available before class at this link (note you must be logged into your BU account to access the slides)

Topic

Details

Assignments

Tue

Sep 5

Introduction

What is machine learning? types of learning, course information

Wed

Sep 6

Lab 1

Numpy Intro

Thu

Sep 7

Classification I

Classification intro, error rates, nearest neighbors; Reading: Forsyth Ch 1.1-1.2

ps1 out

Tue

Sep 12

Classification II

Overfitting, cross validation, naive bayes, support vector machines intro; Reading: Forsyth Ch 1.3, 2.1, 2.1.1

Wed

Sep 13

Lab 2

Sklearn Intro

Thu

Sep 14

Support Vector Machines

Regularization, stochastic gradient descent, multiclass SVM; Reading: Forsyth Ch 2.1.2-2.1.5, 2.1.7

Tue Sep 19

Random Forests

Decision tree, information gain, decision forest; Reading: Forsyth Ch 2.2

quiz 1 out

Wed

Sep 20

Lab 3

Multiclass classification

Thu Sep 21

High dimensional data

The curse of dimension, eigenvectors, affine transformations, PCA; Reading: Forsyth Ch 4.3-4.4, 5.1, 6.1

ps1 due

Tue Sep 26

CCA

Multidimensional scaling, Bag of Words, TF-IDF, Canonical Correlation Analysis; Reading: Forsyth Ch 6.2-6.3, 7.1

ps2 out

quiz 1 due

Wed

Sep 27

Lab 4

Data Transformations

Thu Sep 28

Clustering I

Agglomerative clustering, divisive clustering, K-means; Reading: Forsyth Ch 8.1-8.2

Tue

Oct 3

Clustering II

Gaussian mixture models, expectation maximization; Reading: Forsyth Ch 9.1-9.2

Wed

Oct 4

NO LAB

Thu

Oct 5

Regression I

Linear regression, interpreting your model, regularization; Reading: Forsyth Ch 10.2-10.3

quiz 2 out

Tue

Oct 10

NO CLASS (Monday schedule)

DROP DATE (without a “W”)

ps2 due

ps3 out

Wed

Oct 11

Lab 5

Clustering

Thu

Oct 12

Regression II

Model selection, bias and variance, robust regression; Reading: Forsyth Ch 11.1-11.2, 11.3.1

quiz 2 due

Tue

Oct 17

Regression III

Boosting, regression trees; Reading: Forsyth Ch 12.1-12.2

Wed

Oct 18

Lab 6

Regression

Thu

Oct 19

Neural Networks I

artificial neuron, MLP, sigmoid units; output vs hidden layers; linear vs nonlinear networks; feed-forward neural networks; Reading: Forsyth Ch 16.1, 16.3.1

Tue

Oct 24

Neural Networks II

learning via gradient descent; computation graphs, backpropagation algorithm; Reading: Forsyth Ch 16.3.2-16.3.4, 16.4, 18.1.7

ps3 due

ps4 out

Wed

Oct 25

Lab 7

Pytorch Intro

Thu

Oct 26

Neural Networks III

Convolutional neural networks, recurrent neural networks; Reading: Forsyth Ch 17.1, 18.1

quiz 3 out

Tue

Oct 31

Reinforcement Learning I

reinforcement learning intro; Markov Decision Process (MDP); policies, value functions, Q-learning

Wed

Nov 1

Lab 8

CNNs

Thu

Nov 2

Reinforcement Learning II

Q-learning cont’d; deep Q-learning (DQN)

quiz 3 due

Tue

Nov 7

Graphical Models I

Markov chains and Hidden Markov Models Reading: Forsyth Ch 13.1-13.3

ps4 due

ps5 out

Wed

Nov 8

Lab 9

Reinforcement Learning

Thu

Nov 9

Graphical Models II

Conditional Random Fields, inference and learning, label bias; Reading: Forsyth Ch 14.1-14.3

Tue

Nov 14

Understanding your model

Visualizing features, denoising autoencoders, saliency-based network explanations; Reading: Forsyth Ch 19.1-19.2

quiz 4 out

Wed

Nov 15

Lab 10

XAI

Thu

Nov 16

GANs

Variational autoencoders, adversarial methods, Generative Adversarial Nets (GANs); Diffusion models; Reading: Forsyth Ch 19.3

challenge out

Tue

Nov 21

Guest Lecture

Content Creation and Understanding with Structured Representations

ps5 due

quiz 4 due

Wed

Nov 22

NO LAB- Fall Recess

Thu

Nov 23

NO CLASS- Fall Recess

Tue

Nov 28

Label Scarcity I

Domain adaptation/generalization, self-supervision

Wed

Nov 29

Lab 11

Self-Supervised Learning

Thu

Nov 30

Label Scarcity II

Semi-supervised learning, learning with noisy labels, active learning

Tue

Dec 5

Distributed/Efficient Neural Networks

Mixed precision, model/data parallelism, network/feature quantization, parameter pruning; knowledge distillation

quiz 5 out

Wed

Dec 6

Lab 12

Semi-supervised learning

Thu

Dec 7

Auto-ML

neural architecture search, parameter allocation search, automated data augmentation

Tue

Dec 12

NO CLASS

challenge due

quiz 5 due

No final exam

*schedule is tentative and is subject to change.

SYLLABUS

This course is a rigorous introduction to machine learning concepts, techniques, and algorithms.

Course Prerequisites

This is a graduate course (open to upper-level undergraduates) and requires the following:

You should be very comfortable with programming in Python and be able to both write and understand code quickly. The above prerequisites can be replaced by Foundations of Data Science (CS 365).

Textbooks

The required textbook for the course is:

Students can obtain this book for free from the BU Library by getting the Online Access version

Other recommended supplemental textbooks on general machine learning:

Recommended background reading on matrix calculus

Deliverables/Graded Work

The main graded work for the course is the problem sets, class participation, quizzes, and final challenge. There will be five problem sets (ps1-ps5), each consisting of written and programming problems, and five multiple choice quizzes for each major topic covered. Note: there are no team projects in this course. The course grade consists of the following:

Final Grade Distribution

The final grade distribution will be no worse than the standard grade cutoffs, i.e.,

The final grade cutoffs will be posted on Piazza at the end of the term.

Participation

Students are expected to actively participate in course discussions.  These can include in-class participation, where lectures will include audience participation components, as well as responding to student questions on Piazza.  Full credit for participation will be given to students who make contributions to course discussions every week. Top student contributor(s) will have extra credit added to their final grade.

Challenge

The challenge is an individual programming project that gives each student a chance to apply their knowledge and develop a computer vision solution to a real-world problem. The goal is to take what you learned in the weekly problem sets to the next level and spend a few weeks implementing computer vision algorithms from scratch and applying them to a challenging dataset. More information about the challenge will be released later on in the term.

Labs

Labs consist of hands-on exercises meant to provide students with experience on how to implement the concepts covered in lectures.  Students are expected to attend and participate in a lab to receive credit.  Students can make-up a missed lab by performing the activity and turning it into gradescope by 11:59pm no later than the Friday directly following the lab to obtain half credit.  Only students wishing to make-up a lab section are required to submit their completed lab activity on gradescope.  Labs are posted by the time of the first lab section.  In addition, only the top 10 labs are counted towards the final grade.   For example, a student missing 4 labs (of 12) would have a lab grade of 8 out of a max of 10 points (resulting in a contribution of 16/20 to your final grade), but the student’s grade could be raised to 18/20 by make-up two of the labs.  Note: the labs are hands-on exercises, so having your laptop is recommended so you can participate in lab activities.

Piazza

We will be using Piazza for class Q&A and posting problem sets.  Students responding to questions posed by other students on Piazza will count towards their participation grade.  Course staff may take up to two business days to respond to questions on Piazza, so students are encouraged to begin their assignments early to ask any clarifying questions to course staff well before the due date. Immediate answers to questions from course staff students can be obtained during office hours.  In addition, rather than emailing questions to the teaching staff, we ask you to please post all your questions on Piazza. For private questions like grading please select just the “Instructors” in the “Post to:” field of the Piazza note and select the appropriate folder.

Software/Compute Resources

Programming assignments will be developed in the Python programming language, including iPython notebooks. No special computing resources are required besides a regular laptop.

Another resource you may want to use for some problem sets or the challenge is Google’s Colab, see https://colab.research.google.com/notebooks/welcome.ipynb

Help! My laptop broke!

If you find yourself in a situation where you need a back up machine, say, due to a broken laptop, the BU Common at the Mugar Library offers access to Microsoft Windows 10 virtual machines. The lab is co-located with the IT Help Center and the Print Center and hosts a broad selection of software including Python and Jupyter. Also note that the IT Help Center is an authorized repair center for most major laptop brands, and can often perform many common types of repairs within one day.  They also offer temporary loaners for certain models of laptops while they are working on your machine.

COURSE POLICIES

Late Policy

Late submissions are only accepted for problem sets for up to two days after the due date.  Students are given up to two days worth of penalty-free late days for the semester.  Note that this counted on a per-day basis.  In other words, if you are 1 hour late, then it would count as one of your two late days.  The penalty-free days are applied chronologically to your late submissions.  After the two penalty-free days are allocated, the following policy applies for future late submissions:

Exceptions to these policies can only be made in cases of significant medical or family emergencies, and should be documented.

Regrading Policy

All regrade requests must be made through Gradescope when possible within 1 week of when grades are released (1 week after assignments are due).  Thus, no regrade requests will be approved if it is submitted more than 2 weeks after the assignment’s due date.  Note that regrade requests for programming portions of assignments (listed as .py for the problem sets) must be submitted via Piazza as a private note to “Instructors” as Gradescope does not support regrading for these types of submissions.

Copyright Notice

All course materials, including lecture slides/notes, videos/screenshots, homeworks, solutions, and solution code are copyrighted by Prof. Plummer. All rights are reserved. Other than for personal use by registered students, the course materials or any portion thereof may not be reproduced or used in any manner whatsoever (including posting on a public forum or website) without the express written permission of the authors. Acting otherwise would go against the ethical code of conduct expected of students at Boston University and from a legal standpoint it may even constitute a violation of copyright law.

Academic Honesty Policy

The instructors take academic honesty very seriously. Cheating, plagiarism and other misconduct will be subject to grading penalties up to failing the course.  Students enrolled in the course are responsible for familiarizing themselves with the detailed BU policy, available here. In particular, plagiarism is defined as follows, and applies to all written materials and software, including material found online. Collaboration on homework only in the form of discussion is allowed, but should be acknowledged and you should always come up with your own solution rather than copy others’ solutions (which is defined as plagiarism).

Plagiarism: Representing the work of another as one’s own. Plagiarism includes but is not limited to the following: copying the answers of another student on an examination, copying or restating the work or ideas of another person or persons in any oral or written work (printed or electronic) without citing the appropriate source, and collaborating with someone else in an academic endeavor without acknowledging his or her contribution. Plagiarism can consist of acts of commission-appropriating the words or ideas of another-or omission failing to acknowledge/document/credit the source or creator of words or ideas (see below for a detailed definition of plagiarism). It also includes colluding with someone else in an academic endeavor without acknowledging his or her contribution, using audio or video footage that comes from another source (including work done by another student) without permission and acknowledgement of that source.

Prohibited behaviors include:

Incidents of academic misconduct will be reported to the Academic Conduct Committee (ACC). The ACC may suspend/expel students found guilty of misconduct. At a minimum, students who engage in misconduct will have their final grade reduced by one letter grade (e.g., from a B to a C).

Religious Observance

Students are permitted to be absent from class, including classes involving examinations, labs, excursions, and other special events, for purposes of religious observance.  In-class, take-home and lab assignments, and other work shall be made up in consultation with the student’s instructors. More details on BU’s religious observance policy are available here.