Math Refresher

Prob Stats and Linear Algebra

Probability and Statistics Basics

There is always an uncertainty in learning

Probability and Random Variables

​

• Intuitively, the uncertainty in the occurrence of an event
• Related terms:
• Event
• Sample Space
• Random Variable: A function from an event space to a probability space.
• Two types based on support:
• Continuous
• Discrete
• Examples

​

Probability and Random Variables II

• Discrete RVs are represented with a Probability Mass Function
• Continuous RVs are represented with a Probability Density Function
• Examples: PMF of an (un)fair die, PDF of a uniform RV between 0 and 1
• Cumulative Distribution Functions (CDF): ℙ(X <= t)

Common Types of RVs

• Discrete
• Bernoulli Random Variable (coin toss)
• Poisson Random Variable (count values)
• Multinoulli RV (multiple coins) - joint distribution of multiple RVs
• Binomial / Multinomial RV (number of successes)
• Continuous
• Uniform RV (uniformly between two values)
• Normal / Gaussian RV (a real number)
• Beta RV (number between 0 and 1)
• Gamma RV (a positive real number)
• Dirichlet (a vector that sums to 1)

Statistics of a Distribution (RV / Sample)

• Expectation: Average / Mean value of a distribution. Denoted by 𝔼[X].
• Expected number of heads if a coin is tossed 100 times. (Binomial)
• Expected value of a number drawn from a Normal Distribution. (like grades data)

​

• Variance: A measure of the average deviation of the values in a distribution from their expected value.
• The more the variance, the less can be our reliance on the expectation.
• Moment matching: matching moments from the model to the sample for inference.
• σ² = 𝔼[X²] - 𝔼[X]²

Linear Algebra Basics

This is where it all begins

Basic Concepts

• Notation
• Vector Spaces
• Basis / Span
• Matrix Inverse
• Norms
• Eigenvalues / eigenvectors

​