JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

1 of 37

Regression: Estimating Equations and GLMs

Johns Hopkins Bloomberg School of Public Health

Instructor: Jeff Leek

Youtube: http://www.youtube.com/JHSPHAppliedStat

2 of 37

A bit cliche but...

Essentially, all models are wrong, but some are useful.

-George Box

3 of 37

What is regression?

4 of 37

What is regression?

Estimating differences in Y between subjects whose X values differ in a specified manner.

Usually based on expectations�

E[Y|X=1] - E[Y|X=0]

log(E[Y|X=1]/E[Y|X=0])

exp(E[log(Y)|X=1] - E[log(Y)|X=0])

5 of 37

Why not just linear regression?

Momentous sprint at the 2156 Olympics?

6 of 37

Why not just linear regression?

An epigenetic signature in blood predicts active ovarian cancer

7 of 37

Why not just linear regression?

From: Modern Applied Statistics with S

8 of 37

Basic Ideas

With reasonable sample size, you can learn about parameters without making strong assumptions about the distribution of Y and how it varies with X.

If you have good reasons to believe parametric assumptions a priori they may be ok and can help.
For small sample size this may be the only alternative (other than quitting)
Checking assumptions after you've used them doesn't actually work very well

9 of 37

General Approach

10 of 37

Defining your parameter

11 of 37

Mathematical definitions

12 of 37

Defining parameters with equations

13 of 37

Link functions

14 of 37

Estimating parameters

15 of 37

Examples of estimating equations

16 of 37

Newton's Method

http://en.wikipedia.org/wiki/Newton's_method

17 of 37

Estimating parameters with Newton

18 of 37

Using the Central Limit Theorem

19 of 37

The sandwich estimator

20 of 37

An example

21 of 37

Standard errors

22 of 37

Multiple ways to get confidence intervals

23 of 37

Fitting lines (least squares)

24 of 37

Least squares for curves

25 of 37

Weighted least squares

26 of 37

Fitting lines (popular weights)

http://en.wikipedia.org/wiki/Iteratively_reweighted_least_squares

27 of 37

Line fitting general case

28 of 37

Line fitting: weights

29 of 37

Generalized Linear Models

30 of 37

Likelihood

http://www.youtube.com/watch?v=wLaRXYai19A

31 of 37

Exponential Family

32 of 37

GLMs: Three components

33 of 37

GLMs: Important properties

34 of 37

GLMs: Maximum likelihood

35 of 37

GLMs: ML Estimation

36 of 37

GLMs: Asymptotic standard errors

37 of 37

Fisher Information

http://simplystatistics.tumblr.com/post/18903448428/r-a-fisher-is-the-most-influential-scientist-ever