1 of 33

NFL Ghosts:

Evaluating pass defense with high-dimensional CDEs

1

Ron Yurko

​

Sept 23, 2023

​

Assistant Teaching Professor

Department of Statistics & Data Science, Carnegie Mellon University

​

2 of 33

Traditional view of sports data for analysis

2

Period of time

Starting value

Ending value

Observe difference in some measure over a period of time

​

Example: Team’s score before and after a play occurs

3 of 33

NESSIS 2017: How do we value plays?

3

Expected points (EP): how many points have teams scored in similar situations?

  • Predict probability of different scoring events
  • Multinomial logistic regression

​

Yurko et al. (2019)

4 of 33

This is an oversimplification of sports!

4

Period of time

Starting value

Ending value

? ? ?

5 of 33

There are many intermediate events and actions taking place in continuous-time

5

Period of time

Starting value

Ending value

6 of 33

NFL tracking data courtesy of Big Data Bowl

On-field (x, y), speed, direction, orientation, for each player (+ ball) at 10Hz

NFL provides event annotations within plays (e.g. handoff, first contact, etc)

Example play: Cordarrelle Patterson’s 47 yard jet sweep TD run

​

​

​

​

6

7 of 33

Patterson’s 47 yard jet sweep TD run

​

7

8 of 33

NESSIS 2019:

How do we compute continuous-time within-play value?

Given our prediction for the end-of-play yard line we proceed to update:

  • , , etc. as input for EP and WP models

8

​

Generate point estimate for using multinomial logit model

​

9 of 33

We need a distribution estimate for the ending yardline!

Play value V (such as EP) is a complex function g() of the ending yardline Y

​

​

​

​

We’ll estimate f(Y|X) using Random Forests for Conditional Density Estimation (RFCDE), Pospisil and Lee (2018)

​

9

10 of 33

10

Yurko et al. (2020)

11 of 33

Tool of the day: RFCDE

Weighted kernel density estimate (KDE) based on “nearby” observations:

​

​

Average weights over T trees, , plug-in KDE:

​

​

​

Key difference: tree splits are chosen to minimize CDE loss

​

​

11

Pospisil and Lee (2018)

12 of 33

Generate continuous-time estimates…

12

13 of 33

… to estimate within-play values

13

14 of 33

How can we assess player performance with this?

14

15 of 33

Evaluating defender ability to limit yards after catch (YAC)

Big Data Bowl 2021:

  • Evaluate pass defense
  • All passing plays during 2018 season
  • All linemen are removed from data

Two yardage components for passing:

  1. Air yards
  2. YAC

Goal: quantify defender impact after the catch in terms of value of interest V

15

16 of 33

RFCDE for YAC distribution at the moment of catch

​

​

Estimate conditional density of receiver’s ending yard line w/ RFCDE

​

16

17 of 33

Feature engineering with for YAC RFCDE

17

Features describing locations and velocity vectors for receiver, teammates, QB (at release), and defenders

​

Ordered teammates and defenders by distance to receiver

​

Use leave-one-week-out cross-validation to evaluate model

​

Performed best with:

receiver, QB, and closest defender

18 of 33

Example play: Green Bay CB Jaire Alexander stops WR

18

19 of 33

19

20 of 33

20

? ? ?

21 of 33

21

22 of 33

We need a distribution estimate for defender location!

22

Estimate 2D conditional density of defender’s location at-catch with RFCDE

Use LOWO CV to determine best estimates based only on receiver data

​

23 of 33

Framework for estimating expected ghost value

Reminder:

​

​

​

​

​

​

​

​

​

​

23

Requires two CDE estimates: (1) YAC and (2) 2D player location

​

For each 2D location , create new new ghost features

​

Generate YAC CDE, integrate with value function to compute

​

Then integrate over 2D location CDE to compute

24 of 33

But we need to account for velocity vectors as well!

24

For each 2D location , sample B velocity vectors (speed, direction, orientation), creating

​

Generate YAC CDE, integrate with value function to compute

​

Average over B samples to compute

​

Then integrate over 2D location CDE to compute

25 of 33

View distribution of ghost values

25

26 of 33

Compute

26

27 of 33

Player rankings based on accumulated change in EPV

27

28 of 33

Recap and limitations

  • Introduced framework for evaluating defenders impact on YAC in terms of EPV via modeling conditional density estimates for ghost defenders

​

  • Cannot attribute EPV change entirely to player without recognizing the team’s scheme may have played a large role
  • Only considering the closest defender, conditional on this distance and information about the receiver… do we pursue modeling team defense?
  • Focusing on YAC… which means the pass was complete! Could extend our approach for EPV prior to completion but requires additional event models

28

29 of 33

What about continuous-time?...

29

30 of 33

Continuous-time EPV of YAC for reception

30

31 of 33

Next steps: compare to ghosts across continuous-time

31

32 of 33

Acknowledgements and CMSAC

  • Joint work with Kostas Pelechrinis
  • Thanks to Sam Ventura and Nic Dalmasso

​

​

Thank you for your attention, questions?

32

33 of 33

References

Yurko, R., Ventura, S. & Horowitz, M. (2019). nflWAR: a reproducible method for offensive player evaluation in football. Journal of Quantitative Analysis in Sports, 15(3), pp. 163-183.

Yurko, R., Matano, F., Richardson, L.F., Granered, N., Pospisil, T., Pelechrinis, K. & Ventura, S. (2020). Going deep: models for continuous-time within-play valuation of game outcomes in American football with tracking data. Journal of Quantitative Analysis in Sports, 16(2), pp.163-182.

Pospisil, T. and Lee, A. (2018). RFCDE: Random forests for conditional density estimation. URL: https://arxiv.org/abs/1804.05753.

​

33