The course is a programming-focused introduction to Machine Learning. Increasingly, extracting value from data is an important contributor to the global economy across a range of industries. The field of Machine Learning provides the theoretical underpinnings for data-analysis as well as more broadly for modern artificial intelligence approaches to building artificial agents that interact with data; it has had a major impact on many real-world applications.

The course will emphasize understanding the foundational algorithms and “tricks of the trade” through implementation and basic-theoretical analysis. On the implementation side, the emphasis will be on practical applications of machine learning to computer vision, data mining, speech recognition, text processing, bioinformatics, and robot perception and control. Real data sets will be used whenever feasible to encourage understanding of practical issues. The course will provide an opportunity for an open-ended research project. On the theoretical side, the course will give a undergraduate-level introduction to the foundations of machine learning topics including regression, classification, kernel methods, regularization, neural networks, graphical models, and unsupervised learning.

Recommended Textbooks

(recommended) Chris Bishop, "Pattern Recognition and Machine Learning," Springer, October 1, 2007.
(recommended) Kevin Murphy, “Machine Learning: A Probabilistic Perspective”, MIT Press, 2012

Other relevant (and advanced) books

Hastie, Tibshirani, Friedman, "Elements of Statistical Learning," Springer, 2010. (available online)
David Barber, “Bayesian Reasoning and Machine Learning”, Cambridge University Press, 2012 (available online)
Sutton and Barto, "Reinforcement Learning: An Introduction," MIT Press, 1998 (available online)
Boyd and Vandenberghe, "Convex Optimization," Cambridge University Press, 2004. (available online)
Mackay, "Information Theory, Inference, and Learning Algorithms," Cambridge University Press, 2003. (available online)

Prerequisites

Linear algebra (equivalent to MATH 217 or MATH 417)
Multivariate calculus
Probability (equivalent to EECS 401)
EECS 281

* NOTE: Please see the instructor if you do not satisfy the above math requirements. In particular, if you haven't taken linear algebra or if you haven’t taken at least two of linear algebra, multivariate calculus, and probability courses, it is recommended that you finish them first before taking this course. We will have some review lectures and discussion sessions to refresh your knowledge on these topics. However, they are not meant to be replacement of the regular prerequisite courses.

Discussion Sections

Discussion sections are designed to facilitate understanding of specific topics through problem solving, practical hands-on tutorials on implementation of learning algorithms, and interactive QnAs.

Linear Algebra and Matrix Calculus Notes: http://cs229.stanford.edu/section/cs229-linalg.pdf

Homework

There will be four or five problem sets to implement the algorithms we learned about class and also strengthen the understanding of the fundamental concepts, mathematical formulations, algorithms. The substantial portion of problem sets will include programming assignments to implement algorithms covered in the class.

You are given 7 late days during this semester. You can use at most 3 days per homework. In other words, All assignments beyond 3 days will be graded zero. If you use all your 7 late days, you can use more days for 20% grade deduction per additional late days (beyond the 7 late days). Nonetheless, no assignments will be accepted 3 days after the deadline as we intend to make solutions available 3 days after submission.

Project

This course offers an opportunity for getting involved in open-ended research in machine learning. Students are encouraged to develop new algorithms in machine learning, or apply existing algorithms to new problems. Please talk to the instructor before deciding about the project topic. Students will be required to complete their project proposals, progress reports, poster presentations and the final report.

The project will be a group assignment. Students are required to form into groups of 4 or 5 people.

Check resource page in ctools for more detailed information.

Policy about the study group size

1. Regarding study groups for homeworks, you are allowed to form a group of maximum 4 people, as stated in the syllabus. This means that you are welcome to form a group of 3 or less at your convenience. In any case, you should be writing your own solution independently. For example, submitting the same solution with other group members is not allowed. No code sharing (or copying) among the group members will be allowed.

2. For mini projects (that will be handed later in this class), the same policy applies as 1. You are expected to do your own mini-project independently. No code sharing (or copying) among the group members will be allowed.

3. For final projects, you are allowed to form a group of 4 or 5 people. In grading, we will not penalize the score because the team size is bigger, so we implicitly encourage the group size of 5. To make sure that you do approximately an equal share, you will be asked to write a statement in your report on how each member contributed to the project (details of work done, percentage of contribution to the project, etc.).

Grading

Homework: 30%

Mini-project: 15%

Midterm: 30%

Project: 25% (progress report 10%; final project 15%)

Extra credit: 2 % for active class participations.

Important dates

Project proposal due: 10/6 11:55pm
Project progress report due: 11/6 11:55pm
Midterm exam: TBD (mid/late November)
Final project poster presentation: TBD (final period)
Final project report due (tentative): 12/23 11:55pm

Frequently Asked Questions

Q. What’s difference between EECS 445 and EECS 545?

EECS 445 is more implementation and application oriented, and it will cover fewer topics, but more implementation details. It's more geared towards undergraduate or graduate students outside EECS.

In contrast, EECS 545 will cover more theoretical aspects and advanced techniques.

NOTE:

If you are an undergraduate student, we recommend taking EECS 445 instead of EECS 545. If you want to take EECS 545, please consult the corresponding instructor.
If you are a CSE graduate student, then EECS 445 will not satisfy your breadth/depth requirements. In other words, this course does NOT count as a technical elective for CSE grad students

Q. What’s difference between EECS 445 and EECS 492?

EECS 445 will focus on details of machine learning algorithms (such as logistic regression, support vector machines, neural networks, etc.), so you will learn more about detailed techniques and implementation of these algorithms. There will be also an opportunity to get involved in projects that will deal with real-world problems. If you are interested in data mining, data analysis/prediction, and pattern recognition, this course will be a better fit.

EECS 492 will cover more broad topics in artificial intelligence, such as search, logic, knowledge representation and reasoning, planning. There will be some overlap since EECS 492 will also cover a few lectures of machine learning.

Q. How much will the workload be?

In terms of workload, it will be 4 unit course with heavier load than other courses. This means that you would spend on average 18~20 (or more) hours per week for this course (i.e., all activities such as, attending lectures, discussion sections, homeworks, projects, studying for exams, etc.). In particular, proficiency in math (linear algebra, probability, etc.) are strongly recommended. If you don't have reasonable math skills (such as linear algebra), you may feel higher workload. Therefore, we strongly recommend taking 445 after you take linear algebra courses first.

For your reference, EECS Undergraduate Workload Survey Results are available below

https://www.eecs.umich.edu/eecs/undergraduate/survey/

Topics to be covered (tentative)

* indicates advanced topics

Introduction
Regression

Linear regression
Gradient descent and stochastic gradient
Newton method
Probabilistic interpretation of linear regression: Maximum likelihood

Classification

k-nearest neighbors (kNN)
Naive Bayes
Linear discriminant analysis/ Gaussian discriminant analysis
Logistic regression
Generalized linear models, softmax regression

Kernel methods

Kernel density estimation, kernel regression
Support vector machines
Convex optimization*
Gaussian processes*

Regularization

L2 regularization
Bias-Variance tradeoff
Overfitting
Cross validation, model selection
L1 regularization, sparsity and feature selection*

Advice for developing machine learning algorithms
Neural networks

Perceptron
MLP and back-propagation

Deep learning*
Unsupervised learning

Clustering: K-means
Gaussian mixtures: Expectation maximization
Dimensionality reduction: Principal Component Analysis (PCA)
Independent Component Analysis (ICA) and sparse coding

Ensemble learning

Bagging
Boosting

Graphical models*
Hidden Markov Models (HMM)*
Reinforcement Learning*
Learning Theory*

Schedule (tentative)

The schedule is tentative and subject to change.