Math for Machine Learning

Discussion Mini Lecture 1

Review of Pre-Req Material and Connection to ML

CS 189/289A, Fall 2025 @ UC Berkeley

Sara Pohland

Notes about Discussion Mini Lectures

• Discussions are offset from lectures
• E.g., Discussion 1 takes place in Week 2 and covers concepts presented in Lecture 1 during Week 1
• You can utilize course resources in whatever way works best for you, but we recommend that you:
1. Attend/watch lectures
2. Watch the corresponding discussion mini lecture
3. Attend a discussion section
4. Review uncovered discussion problems on your own*
5. Optionally, look at additional resources at end of slides

* We will likely not cover all of the discussion material during the 50 min discussion session! Additional problems are provided for you as a way to review concepts and get extra practice.

1. ML as Function Approximation
2. Linear Algebra
3. Multivariate Calculus
4. Probability Theory

Concepts Covered

ML as Function Approximation

1. ML as Function Approximation
2. Linear Algebra
3. Multivariate Calculus
4. Probability Theory

Supervised Learning

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

spam

$1,200,000

ML Model

ML Model

ML Model

dog

ML as Function Approximation

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

spam

$1,200,000

dog

How do we Represent our Data?

flatten

vector of pixels

Practice with data manipulation: Discussion 1 notebook

vector of words

vector of features

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

ML as Function Approximation

vector of pixels

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

vector of words

vector of features

Linear Algebra Review

1. ML as Function Approximation
2. Linear Algebra
3. Multivariate Calculus
4. Probability Theory

ML as Function Approximation

vector of pixels

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

vector of words

vector of features

Linear Function Approximation

vector of pixels

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

vector of words

vector of features

Properties of Functions

injective – every input in our domain maps to one* output in our range

* If an input maps to more than one output, we do not have a valid function.

surjective – every output in our range maps to at least one input in domain

bijective – there exists exactly one output in range for every input in domain

Every image has a label within our set of labels.

Every label is associated with at least one image.

We have a perfect set of image-label pairs.

Properties of Linear Functions

vector of pixels

Determining Matrix Rank

Option 1: Python

Option 2: Inspection

How many linearly independent rows/columns?

There are two; notice that:

row3 = 2 * row1

col3 = 2 * col2 – col1

Option 3: Singular Values

How many non-zero singular values?

Practice calculating singular values: Discussion 1 problem 2b

Multivariate Calculus Review

1. ML as Function Approximation
2. Linear Algebra
3. Multivariate Calculus
4. Probability Theory

ML as Function Approximation

vector of pixels

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

vector of words

vector of features

Sensitivity of Function

How sensitive is my output to my input variables?

How will my output change if I change my input by a small amount?

partial derivative!

Practice calculating partial derivatives: Discussion 1 problem 1

Learning New Features

Image Features

Text Features

Tabular Features

spam

$1,200,000

dog

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

Chain Rule

How sensitive is my output to my input variables?

How will my output change if I change my input by a small amount?

Practice with chain rule: Discussion 1 problem 1a

Probability Theory Review

1. ML as Function Approximation
2. Linear Algebra
3. Multivariate Calculus
4. Probability Theory

ML as Function Approximation

vector of pixels

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

vector of words

vector of features

Probabilistic Interpretation of ML

vector of pixels

HI, It’s your boss. Im stuck in Nigeria with none money. Please wire to TRWIGB2LXXX SOON.

vector of words

vector of features

Bayes’ Theorem

Practice with Bayes’ Theorem: Discussion 1 problem 3b

Law of total probability:

Math for Machine Learning

Discussion Mini Lecture 1

Contributors: Sara Pohland

Additional Resources

1. ML as Function Approximation
• Lecture 1 and 3
2. Linear Algebra
• Deep Learning Foundations and Concepts – Appendix A
• Mathematics for Machine Learning – Section 3
3. Multivariate Calculus
• Calculus Early Transcendentals – Chapter 14
• Mathematics for Machine Learning – Section 4
4. Probability Theory
• Lecture 4
• Deep Learning Foundations and Concepts – Chapter 2
• Mathematics for Machine Learning – Section 5