LO 4.2.2.E

Learning Objective: Describe the ridge regression.

Review:

• With reference to the figure shown below:

• For values of λ up to about 10, the variance decreases rapidly (green curve), with very little increase in bias (black curve).
• Consequently, the test MSE (purple curve) drops considerably as λ increases from 0 to 10.
• Beyond this point, the decrease in variance due to increasing λ slows, and the shrinkage on the coefficients causes them to be significantly underestimated, resulting in a large increase in the bias (black curve).
• The minimum test MSE is achieved at approximately λ = 30 (purple cross).
• Note that, the test MSE for λ = 0  (regular least-squares regression) is higher than the MSE  of the null model M0 (base case with no predictors) obtainable when λ→+∞ ( 𝛽j→ 0).

Source: Assigned reading

        Legenda: Squared-bias (black), variance (green), and test MSE (purple)