What Are Degrees of Freedom?
Floyd Bullard
Teaching Contemporary Mathematics Conference 2025
NCSSM Durham NC
0 degrees of freedom
1 degree of freedom
2 degrees of freedom
3 degrees of freedom
4 degrees of freedom
Q: What are degrees of freedom?
A: “Degrees” means dimensions. And “freedom” means the freedom to move around in an error space.
Q: What’s an “error space”?
A: It’s the component of your data that isn’t explained by your model.
Here is how degrees of freedom are lost:
More parameters = less freedom
Model | Dimensionality of explained component of data | Dimensionality of unexplained component of data |
| 0 | 5 |
| 1 | 4 |
| 2 | 3 |
| 3 | 2 |
| 4 | 1 |
| 5 | 0 |
Aside:
The following are both linear models, because they’re linear in the parameters.
A closer look at the first dimension lost
This looks like it’s free to be anything in a 5D space.
�But it’s not—the 5D space that the raw data were free to move around in has been flattened in one direction.
The errors are only free to move around in a 4D space.
Incidentally, the particular model that minimizes the length of the error vector would necessarily also minimize the sum of the squared errors. Here’s the breakdown for our data: [part predicted by model] + [error]
Here’s the same idea, but for just two data points. With just two data points we can represent the vectors in our model graphically, not just algebraically.
not:
Here’s what a Least Squares Regression Line looks like from this perspective.
3 degrees of freedom in the data
2 degrees of freedom in the model space
1 degree of freedom in the error space
A: Because you didn’t simply choose to not to travel in the up-down direction. You can’t travel in that direction! The fact that there’s an extra dimension there doesn’t mean you have to count it in your average! You only count dimensions in which you are free to move.
Thank you for coming!
I hope you enjoy the rest of the TCM Conference!