2 of 31

Thomas Bayes

1701-1761

English statistician and philosopher

It is doubtful whether this portrait is actually of him.

Credits with Bayes’ Rule; never published his work (published after his death).

An Essay towards solving a Problem in the Doctrine of Chances. By the

late Rev. Mr. Bayes, FRS communicated by Mr. Price, in a letter to John

Canton, AMFRS." Philosophical Transactions (1683-1775) (1763): 370-418.

3 of 31

Frequentist vs Bayesian

Frequentist:

Experiments are infinitely repeatable under the same conditions
The parameter of interest is fixed and unknown

Bayesian:

Each experiment is unique (i.e., not repeatable)
The parameter of interest has an unknown distribution

4 of 31

Philosophy

All models are wrong, but some are useful. (George Box)

5 of 31

Example: Flipping a coin

7 of 31

Bayes Rule

proportion to because denominator is constant

8 of 31

Conjugate Prior

Data

Prior

9 of 31

Nonconjugate Prior

10 of 31

Metropolis Hastings Algorithm

11 of 31

Baseball statistics

12 of 31

Empirical Bayes

13 of 31

Shrinkage towards average

15 of 31

Application in Differential Expression

16 of 31

Empirical Bayes estimator of gene specific variance

17 of 31

Moderated t test

19 of 31

Linear Regression

20 of 31

Ridge Regression

21 of 31

Training, Validation and Testing

Training, validation and Testing should be independent

Split the dataset into three disjoint sets:

Training set: used to fit the model parameters
Validation set: used to tune hyperparameters (e.g., λ in ridge regression)
Test set: used only for final evaluation

Common mistake: Training and testing model on the same dataset (overfitting)

22 of 31

Cross Validation

Goal: choose the regularization parameter (e.g., λ in ridge regression)
Split data into k folds (e.g., 5 or 10)
For each fold i=1,…,k:

Train the model on the other k-1 folds
Predict on the held-out fold
Compute testing metrics (e.g.

correlation/Area under curve )

Average the error across folds

23 of 31

History of Ridge Regression

Invented independently in many different contexts.
Tikhonov regularization 1943 (mathematics)
Arthur Hoerl 1962 Manus Foster 1961 (engineering)
In animal breeding, this is called best linear unbiased prediction (BLUP)/Linear mixed model (LMM) Charles Henderson 1950

24 of 31

Taken from https://cnsgenomics.com/data/teaching/GNGWS23/module5/Lecture3_BLUP.pdf

Train

Test

25 of 31

Extension of Ridge Regression

26 of 31

Transcriptome-wide association study (TWAS)

28 of 31

Closing Thoughts

This course introduces essential concepts in biostatistics: application of statistics in biology

The three themes focused in this course:

Clean data and visualizing data in R (Week 1-3)
Basics of probability theory and statistical inference (Week 4-7)
Statistical model (Week 9-16)

Our discussion is mainly motivated by applications, followed by mathematical concepts when it is necessary.

29 of 31

Connection to other courses in statistics

STAT 512 Applied Regression Analysis (Linear regression ANOVA)
STAT 516 Basic Probability and Application (Central limit theorem)
STAT 517 Statistical Inference (hypothesis Testing, confidence intervals, estimation)
STAT 524 Applied Multivariate Analysis (PCA)
STAT 526 Advanced Statistical Methodology (Generalized linear model)
STAT 527 Introduction to Computing for Statistics (R programming, Newton’s methods)

30 of 31

Course Wrap Up Discussion

What is the one concept you remember from this course?

What works (topics you like) and what didn’t work (e.g. topics you wish not to cover)?

How would you evaluate your overall engagement with AI, positive or negative? Should this course be redesigned to limit AI tools for assignments?

Any other feedback

1 of 31