Published using Google Docs
Applied Machine Learning, Fall 2024
Updated automatically every 5 minutes

Applied Machine Learning, Fall 2024

Schedule

Syllabus

Slides

Policies

Time/Location: Tue/Thu  5:00pm-6:15pm in CAS 211

Course Number: CS 541

Instructors: 

Prof. Bryan Plummer (bplum@bu.edu) office hours: Thursday 10am-12pm in CDS844

Teaching Fellows/Assistants: 

Keanu Nichols (kmn5409@bu.edu) office hours: Tuesday 12-2pm in CDS801

Yitong (Elly) Wang (ellywang@bu.edu) office hours: Monday 5-7pm in CDS362C

Labs:

Wed 12:20pm in MCS B37, 1:25pm in CAS 116, 2:30pm in CDS 950

Piazza:  piazza.com/bu/fall2024/cs541  Please use Piazza, not email, for all questions, including grading or missing class etc. (use the private message to Instructors for such requests)

Gradescope: All assignments and regrade requests must be submitted through Gradescope.

Who should take this class?

If you are a PhD, Master’s or upper-class undergrad interested in using machine learning, but not necessarily to conduct machine learning research, this class is for you! If you are not comfortable with the prerequisites, please DO NOT take this class before taking them.

SCHEDULE*

Lecture slides will be available before class at this link (note you must be logged into your BU account to access the slides)

Topic

Details

Assignments

Tue

Sep 3

Introduction

What is machine learning? types of learning, course information

Wed

Sep 4

Lab 1

Numpy Intro

Thu

Sep 5

Classification I

Classification intro, error rates, nearest neighbors; Reading: Forsyth Ch 1.1-1.2

Tue

Sep 10

Classification II

Overfitting, cross validation, naive bayes, support vector machines intro; Reading: Forsyth Ch 1.3, 2.1, 2.1.1

ps1 out

Wed

Sep 11

Lab 2

Sklearn Intro

Thu

Sep 12

Classification III

Regularization, stochastic gradient descent, multiclass SVM; Reading: Forsyth Ch 2.1.2-2.1.5, 2.1.7

Tue Sep 17

Random Forests

Decision tree, information gain, decision forest; Reading: Forsyth Ch 2.2

quiz 1 out

Wed

Sep 18

Lab 3

Multiclass classification

Thu Sep 19

High dimensional data

The curse of dimension, eigenvectors, affine transformations, PCA; Reading: Forsyth Ch 4.3-4.4, 5.1, 6.1

Tue Sep 24

CCA

Multidimensional scaling, Bag of Words, TF-IDF, Canonical Correlation Analysis; Reading: Forsyth Ch 6.2-6.3, 7.1

ps1 due

ps2 out

quiz 1 due

Wed

Sep 25

Lab 4

Data Transformations

Thu Sep 26

Clustering I

Agglomerative clustering, divisive clustering, K-means; Reading: Forsyth Ch 8.1-8.2

Tue

Oct 1

Clustering II

Gaussian mixture models, expectation maximization; Reading: Forsyth Ch 9.1-9.2

Wed

Oct 2

Lab 5

Clustering

Thu

Oct 3

Regression I

Linear regression, interpreting your model, regularization; Reading: Forsyth Ch 10.2-10.3

quiz 2 out

Tue

Oct 8

Regression II

DROP DATE (without a “W”)

Model selection, bias and variance, robust regression; Reading: Forsyth Ch 11.1-11.2, 11.3.1

ps2 due

ps3 out

Wed

Oct 9

Lab 6

Regression

Thu

Oct 10

Regression III

Boosting, regression trees; Reading: Forsyth Ch 12.1-12.2

quiz 2 due

Tue

Oct 15

NO CLASS (Monday schedule)

Wed

Oct 16

Lab 7

Regression in Pytorch

Thu

Oct 17

Graphical Models I

Markov chains and Hidden Markov Models Reading: Forsyth Ch 13.1-13.3

Tue

Oct 22

Graphical Models II

Conditional Random Fields, inference and learning, label bias; Reading: Forsyth Ch 14.1-14.3

ps3 due

ps4 out

Wed

Oct 23

Lab 8

HMMs

Thu

Oct 24

Neural Networks I

Artificial neuron, MLP, activation functions, learning with gradient descent; Reading: Forsyth Ch 16.1, 16.3-16.4

quiz 3 out

Tue

Oct 29

Neural Networks II

Data augmentation, normalization layers, convolutional networks; Reading: Forsyth Ch 18.1.3-18.1.4, 18.1.6, 18.1.9

Wed

Oct 30

Lab 9

CNNs

Thu

Oct 31

Neural Networks III

Recurrent networks, attention, transformers; Reading: Forsyth Ch 17.1, 18.1

quiz 3 due

Tue

Nov 5

Reinforcement Learning I

reinforcement learning intro; Markov Decision Process (MDP); policies, value functions, Q-learning

ps4 due

ps5 out

Wed

Nov 6

Lab 10

Reinforcement Learning

Thu

Nov 7

Reinforcement Learning II

Q-learning cont’d; deep Q-learning (DQN)

Tue

Nov 12

Understanding your model

DROP DATE (with a “W”)

Visualizing features, denoising autoencoders, saliency-based network explanations; Reading: Forsyth Ch 19.1-19.2

quiz 4 out

Wed

Nov 13

Lab 11

XAI

Thu

Nov 14

Generative Models

Variational autoencoders, adversarial methods, Generative Adversarial Nets (GANs); Diffusion models; Reading: Forsyth Ch 19.3

challenge out

Tue

Nov 19

Label Scarcity I

Domain adaptation/generalization, self-supervision

ps5 due

quiz 4 due

Wed

Nov 20

Lab 12

Self-Supervised Learning

Thu

Nov 21

Label Scarcity II

Semi-supervised learning, learning with noisy labels, active learning

Tue

Nov 26

Distributed Learning

Model/data parallelism, mixed precision, federated learning

challenge initial submission due

Wed

Nov 27

NO LAB- Fall Recess

Thu

Nov 28

NO CLASS- Fall Recess

Tue

Dec 3

Efficient Neural Networks

Network/feature quantization, parameter pruning; knowledge distillation, low-rank adaptation

quiz 5 out

Wed

Dec 4

Lab 13

Semi-supervised learning

Thu

Dec 5

Auto-ML

neural architecture search, parameter allocation search, automated data augmentation

Tue

Dec 10

NO CLASS

challenge due

quiz 5 due

No final exam

*schedule is tentative and is subject to change.

SYLLABUS

This course is a rigorous introduction to machine learning concepts, techniques, and algorithms.

Course Prerequisites

This is a graduate course (open to upper-level undergraduates) and requires the following:

You should be very comfortable with programming in Python and be able to both write and understand code quickly. The above prerequisites can be replaced by Foundations of Data Science (CS 365).

Textbooks

The required textbook for the course is:

Students can obtain this book for free from the BU Library by getting the Online Access version

Other recommended supplemental textbooks on general machine learning:

Recommended background reading on matrix calculus

Deliverables/Graded Work

The main graded work for the course is the problem sets, class participation, quizzes, and final challenge. There will be five problem sets (ps1-ps5), each consisting of written and programming problems, and five multiple choice quizzes for each major topic covered. Note: there are no team projects in this course. The course grade consists of the following:

Final Grade Distribution

The final grade distribution will be no worse than the standard grade cutoffs, i.e.,

The final grade cutoffs will be posted on Piazza at the end of the term.

Participation

Students are expected to actively participate in course discussions.  These can include in-class participation, where lectures will include audience participation components, as well as responding to student questions on Piazza.  Full credit for participation will be given to students who make contributions to course discussions every week. Top student contributor(s) will have extra credit added to their final grade.

Challenge

The challenge is an individual programming project that gives each student a chance to apply their knowledge and develop a computer vision solution to a real-world problem. The goal is to take what you learned in the weekly problem sets to the next level and spend a few weeks implementing computer vision algorithms from scratch and applying them to a challenging dataset. More information about the challenge will be released later on in the term.

Labs

Labs consist of hands-on exercises meant to provide students with experience on how to implement the concepts covered in lectures.  Students are expected to attend and participate in a lab to receive credit.  Students can make-up a missed lab by performing the activity and turning it into gradescope by 11:59pm no later than the Friday directly following the lab to obtain half credit.  Only students wishing to make-up a lab section are required to submit their completed lab activity on gradescope.  Labs are posted by the time of the first lab section.  In addition, only the top 11 labs are counted towards the final grade.   For example, a student missing 4 labs (of 13) would have a lab grade of 9 out of a max of 11 points (resulting in a contribution of (9/11)*20 = 16.4% to your final grade), but the student’s grade could be raised to 18/20 by make-up two of the labs.  Note: the labs are hands-on exercises, so having your laptop is recommended so you can participate in lab activities.

Piazza

We will be using Piazza for class Q&A and posting problem sets.  Students responding to questions posed by other students on Piazza will count towards their participation grade.  Course staff may take up to two business days to respond to questions on Piazza, so students are encouraged to begin their assignments early to ask any clarifying questions to course staff well before the due date. Immediate answers to questions from course staff students can be obtained during office hours.  In addition, rather than emailing questions to the teaching staff, we ask you to please post all your questions on Piazza. For private questions like grading please select just the “Instructors” in the “Post to:” field of the Piazza note and select the appropriate folder.

Software/Compute Resources

Programming assignments will be developed in the Python programming language, including iPython notebooks. No special computing resources are required besides a regular laptop.

Another resource you may want to use for some problem sets or the challenge is Google’s Colab, see https://colab.research.google.com/notebooks/welcome.ipynb

Help! My laptop broke!

If you find yourself in a situation where you need a back up machine, say, due to a broken laptop, the BU Common at the Mugar Library offers access to Microsoft Windows 10 virtual machines. The lab is co-located with the IT Help Center and the Print Center and hosts a broad selection of software including Python and Jupyter. Also note that the IT Help Center is an authorized repair center for most major laptop brands, and can often perform many common types of repairs within one day.  They also offer temporary loaners for certain models of laptops while they are working on your machine.

COURSE POLICIES

Late Policy

Late submissions are only accepted for problem sets for up to two days after the due date.  Students are given up to two days worth of penalty-free late days for the semester.  Note that this counted on a per-day basis.  In other words, if you are 1 hour late, then it would count as one of your two late days.  The penalty-free days are applied chronologically to your late submissions.  After the two penalty-free days are allocated, the following policy applies for future late submissions:

Exceptions to these policies can only be made in cases of significant medical or family emergencies, and should be documented.

Regrading Policy

All regrade requests must be made through Gradescope when possible within 1 week of when grades are released (1 week after assignments are due).  Thus, no regrade requests will be approved if it is submitted more than 2 weeks after the assignment’s due date.  Note that regrade requests for programming portions of assignments (listed as .py for the problem sets) must be submitted via Piazza as a private note to “Instructors” as Gradescope does not support regrading for these types of submissions.

Copyright Notice

All course materials, including lecture slides/notes, videos/screenshots, homeworks, solutions, and solution code are copyrighted by Prof. Plummer. All rights are reserved. Other than for personal use by registered students, the course materials or any portion thereof may not be reproduced or used in any manner whatsoever (including posting on a public forum or website) without the express written permission of the authors. Acting otherwise would go against the ethical code of conduct expected of students at Boston University and from a legal standpoint it may even constitute a violation of copyright law.

Academic Honesty Policy

The instructors take academic honesty very seriously. Cheating, plagiarism and other misconduct will be subject to grading penalties up to failing the course.  Students enrolled in the course are responsible for familiarizing themselves with the detailed BU policy, available here. In particular, plagiarism is defined as follows, and applies to all written materials and software, including material found online. Collaboration on homework only in the form of discussion is allowed, but should be acknowledged and you should always come up with your own solution rather than copy others’ solutions (which is defined as plagiarism).

Plagiarism: Representing the work of another as one’s own. Plagiarism includes but is not limited to the following: copying the answers of another student on an examination, copying or restating the work or ideas of another person, persons, or AI generator in any oral or written work (printed or electronic) without citing the appropriate source (and prompt), and collaborating with someone else in an academic endeavor without acknowledging his or her contribution. Plagiarism can consist of acts of commission-appropriating the words or ideas of another-or omission failing to acknowledge/document/credit the source or creator of words or ideas (see below for a detailed definition of plagiarism). It also includes colluding with someone else in an academic endeavor without acknowledging his or her contribution, using audio or video footage that comes from another source (including work done by another student) without permission and acknowledgement of that source.

Prohibited behaviors include:

Incidents of academic misconduct will be reported to the Academic Conduct Committee (ACC). The ACC may suspend/expel students found guilty of misconduct. At a minimum, students who engage in misconduct will have their final grade reduced by one letter grade (e.g., from a B to a C).

Religious Observance

Students are permitted to be absent from class, including classes involving examinations, labs, excursions, and other special events, for purposes of religious observance.  In-class, take-home and lab assignments, and other work shall be made up in consultation with the student’s instructors. More details on BU’s religious observance policy are available here.