Using Air Yards Per Target and Machine Learning to Dominate Your 2016 Drafts

Last week Josh Hermsmeyer introduced to us air yards per target (aYPT), which aside from charting differences, is identical to Pro Football Focus’ aDOT. However, he was able to further break down receiving into its constituent components: air yards, whether the ball was caught or not, and if caught, the yards created after the catch.

Using these constituent components, plus some other important variables from the RotoViz Screener,1 I created a machine learning model that predicts year N+1 PPR points with a cross-validated R-squared of 0.93 for the wide receiver position2 on in-sample data, and 0.60 on out-of-sample data. In other words, this new information by Hermsmeyer allows us to make a massive improvement over previous wide receiver models.

I’ll break down my methodology, the logic behind it, and then of course list the 2016 PPR projections for the WR position.

The Data

The data set I built the model with consists of data from 2009 to present. To be eligible for the model, the receiver had to have either 30 or more targets, or play in 8+ games in year N, and then play in at least 8 games in year N+1. The goal is to take the model inputs and predict the following season’s PPR points per game for each WR that meets the criteria.

Building, Training, and Testing the Model

To identify the optimal model inputs, I first used a couple of screening methods to whittle the variables down into a subset of variables with the most predictive potential. This gave me a subset of 17 potential inputs. From there, I used some techniques to narrow the list down further, mainly by eliminating highly correlated inputs. As a results, on first pass I ended up with the following list:

  1. Weight (WT)
  2. Age (AGE)
  3. NFL Draft Position (Draft)
  4. PPR points per game the prior year3 (PPR.PPG.N)
  5. Receiving TD Rate (reTDRT)
  6. Receiving average margin4 (reAVGMGN)
  7. Rushing attempt market share (ruATTMS)
  8. Air yards per game (aYPG)
  9. Completed air yards per target (CaYPT)
  10. Incomplete air yards per target (IaYPT)
  11. Yards after catch per target (YACPT)

The nice thing about this is that the only two variables that were even remotely close to being highly correlated were PPR points per game and air yards per game.5 Even then, air yards completely describe PPR points. Everything else was mostly independent from each other, meaning essentially every variable was adding information to the model.

From there I clustered the wide receivers using all of the variables except AGE and reAVGMGN. The reason I didn’t choose AGE is because I took an average of the stats where each receiver played at least 8+ games, so I didn’t want to average over AGE.6 I also chose not to include reAVGMGN, because the quality of team a player plays on can vary greatly throughout a career.

Using a hierarchical clustering method7 I created seven clusters of WRs with similar attributes. This is similar to how our positional Similarity Score Apps are powered.

Using the 11 variables listed plus the clusters as inputs, I then used an ensemble of machine learning models to come up with a predictive model for PPR points per game. The cross validation R-squared on in-sample data was 0.835.

However, when I looked at the 2016 predicted results, a few things struck me as odd. Odell Beckham, Jr. was projected as the 6th WR and Dez Bryant was way down in the mid 30s. I figured this would get people up in arms about why a few players were so far off from ADP. Then it hit me — I also needed to incorporate ADP into the model! That’s because ADP incorporates a wisdom of the crowds (WOTC) element that can’t be determined through a player’s stats alone. Bryant is the perfect example, we know he is better than a WR in the 30s, because last year his stats suffered due to injury both to himself and Tony Romo. How do you model that statistically? With WOTC!

Once I added in positional ADP the projections made more sense, and the predictions got better. This time a cross-validated R-squared of 0.93 on in-sample data. For a random subset of data8 that I withheld, the R-squared was a solid 0.60 on this out-of-sample set. The RMSE on the out-of-sample data was 3.67 meaning about 95 percent of players will end up with a PPR within +/- 6.6 points.9 It’s also interesting to note that these 13 variables let us backfit, or explain, 93 percent of all past performances in PPR points per game. These models are a big performance improvement compared to most regression models, which give a back-fit R-squared of around 0.5-0.6 for WRs, but only explain out-of-sample data in the 0.3-0.4 range.

WRPred

We can also use one of the algorithms to look at the importance of each variable in making these predictions.

WRModelVarSummary

Notice the clusters really do help the model prediction (this is validated by the R-squared and RMSE values of the model without the cluster term, both of which were far worse). The reason the clusters are the most important is because this best describes the type of player we are looking at, and has all these stats rolled into one. The model then figures out how each of the individual stats interacts with each other, and with each of the the player types. I’ll get more into the clusters in a future article. Notice also Pos.ADP significantly helped the model, so that WOTC effect was very important. Finally, it’s cool to see the different air yards stats (air yards per game, complete air yards per target, and incomplete air yards per target, and yards after catch per target) were more important in making predictions than things like reTDRT. This is a tip of the cap to Hermsmeyer for pulling those individual stats together and really breaking down the components of receiving.

2016 PPR Projections

Here are the projections for 2016 as given by the model:

PLAYERPPR.PPG.N1reTDRTreAVGMGNWTAGEDRAFTPPR.PPG.NaYPGruATTMSCaYPTIaYPTYACPTPos.ADPCluster
Julio Jones19.840.039-3.222026623.2128.706.14.13.231.D
Antonio Brown19.660.0520.41862719523.9132.30.016.44.53.111.M
Allen Robinson18.910.093-5.7220226119140.306.98.02.471.D
Odell Beckham Jr.18.530.082-1.2198231221.3121.905.56.23.821.D
DeAndre Hopkins18.050.057-5.4214232720.6166.806.77.21.241.D
A.J. Green16.950.0761.921127417.2105.407.35.52.651.D
Sammy Watkins16.360.094-1.421122416.8123.509.07.71.9161.D
Mike Evans16.070.02-523122714.2143.806.68.01.6121.D
Brandon Marshall15.950.081-2.72303111921.4131.106.16.02.681.D
Alshon Jeffery15.720.043-4.6216254517.6152.606.87.91.9131.D
Julian Edelman15.630.086.31982923219.474.40.014.33.33.6211.S
Keenan Allen15.550.045-7.4206237620.484.605.52.12.7111.M
Brandin Cooks15.180.07-2.4189222015.993.50.025.36.23.4141.M
Jarvis Landry15.080.024-8.1205236316.876.80.053.73.63.2181.S
Amari Cooper15.000.046-2.321121413.583.80.015.45.03.0101.M
Randall Cobb14.890.0470.2192256412.760.90.033.04.53.4191.S
Demaryius Thomas14.240.034-0.6229282217116.204.75.72.6171.M
Jeremy Maclin14.070.0651.6198271916.385.10.015.94.42.8221.M
Ty Hilton14.050.037-4.3183269213.2107.805.57.32.8151.M
Dez Bryant14.040.042-2.222027249.9103.303.69.32.061.M
Jordan Matthews13.390.062-3.7212234214.566.304.63.73.2291.M
Larry Fitzgerald13.070.0623.621832317.877.505.23.33.2281.M
Golden Tate13.050.047-4.1202276013.247.90.022.23.74.1241.S
Eric Decker13.020.091-1.2215288717101.306.25.41.6261.M
Stefon Diggs12.840.048-3.51952214611.568.50.015.15.53.5411.M
Steve Smith12.560.041-6.1185367418.7104.605.94.23.3531.M
Tavon Austin12.410.057-2.417424812.339.80.121.26.24.3445
Doug Baldwin12.190.1361.61892732916.861.806.13.54.3232
Emmanuel Sanders12.180.044-1.2180288215.2124.50.015.58.22.8302
Michael Crabtree12.040.062-2.3214281014.489.704.55.31.8381.M
John Brown11.990.0691.317925911498.80.016.88.03.2341.D
Vincent Jackson11.920.048-7.7230326110.583.607.26.31.6511.D
Torrey Smith11.760.065-6.920526587.754.307.26.73.4501.D
DeVante Parker11.730.06-10.6209221411.776.907.96.12.1352
Willie Snead11.490.029-1.81952332912.567.506.33.63.4431.M
DeSean Jackson11.340.082-1.3175294911.980.307.67.33.2371.D
Tyler Lockett11.070.0875.218223699.756.90.016.07.23.6312
Travis Benjamin10.940.04-6.41752610012.289.40.014.96.62.8472
Michael Floyd10.870.0675.2220261311.593.907.38.72.3272
Kendall Wright10.830.05-3.119626209.770.10.024.47.32.4771.S
Donte Moncrief10.680.057-5.8221229010.865.404.45.52.5252
Ted Ginn10.060.1033.518030913.1104.60.015.211.12.5632
Anquan Boldin10.020.036-7.8220355412.363.104.53.52.6721.M
Mike Wallace9.970.028-119929846.651.404.26.52.3641.D
Jermaine Kearse9.810.0742.7209253299.250.706.85.33.4852
Devin Funchess9.740.0797.523221416.850.605.87.21.8452
Marvin Jones9.700.0391.72002516610.983.50.015.27.82.8362
Pierre Garcon9.460.054-1.32102920511.674.305.55.21.5661.M
Markus Wheaton9.110.063-2.118924799.370.807.56.92.0482
Kamar Aiken9.070.039-4.42122632912.597.705.86.61.7522
Davante Adams8.990.011-4.421223538.173.203.76.51.6712
Kenny Stills8.620.048-11.4194231445.970.005.511.21.5742
Jaelen Strong8.620.1253.721721704.825.304.36.32.4792
Rishard Matthews8.600.066-7.32102622712.163.307.14.43.8572
Terrance Williams8.490.032-5.520826749.674.706.16.83.0732
Dorial Green-Beckham8.470.06-7.923722407.964.406.07.42.1602
Robert Woods8.230.038-3.420123418.659.505.05.41.9822
Brandon LaFell8.2103.521129788.191.90.013.99.83.0902
Mohamed Sanu8.0400.821126835.721.90.023.83.24.1554
Chris Hogan7.960.034-5.8215273297.258.205.97.01.7652
Danny Amendola7.880.0342.41863032910.842.40.014.42.43.0784
Cole Beasley7.620.067-5.2177263298.522.402.91.84.2894
Jamison Crowder7.410.026-3.3185221058.227.803.42.34.3834
Kenny Britt7.300.042-521527307.671.607.78.31.8862
Phillip Dorsett7.130.026-5.618522294.446.00.014.38.91.6583
Bruce Ellington6.880-5.6197241062.912.802.36.55.8704
Nelson Agholor5.640.023-3.519822204.436.904.36.62.1754
Chris Conley5.500.032-121323764.837.704.26.82.3874

A few things to note:

  • Visually the projections look pretty solid. The big three hold three of the top four spots, for example, and Dez Bryant is now at WR20 instead of buried in the mid 30s. I suspect this is still too low, but hey, we also didn’t expect him to have 9.9 PPR points per game last year.
  • Players like Jordy NelsonKelvin BenjaminKevin Whiteand Breshad Perriman aren’t listed due to injury. Rookies also aren’t listed, but you can find rookie WR projections here.
  • The model likes receivers who consistently get targets but suffered in efficiency to rebound, such as Randall Cobb and Jarvis Landry. That makes sense, since volume is more predictive of future success than efficiency is.
  • I believe ensembling these results even further with our staff projections from the Projection Machine, as well as with composite projections from fantasyfootballanalytics.net may yield even stronger R-squared and RMSE values.
  • Stefon Diggs is a RotoViz favorite, and the model loves him too.
  • Anquan Boldin might be the Detroit receiver to target.

I’ll continue to work to improve this model and make it even more predictive – but the performance of this model should be quite strong. Let me know in the comments what else you notice from the projections, or ideas to improve the model.

  1. Which I’ll get to in a minute.  (back)
  2. For WRs with 8+ games or 30+ targets the prior year  (back)
  3. From here the inputs will be from the prior year.  (back)
  4. The amount a team was ahead or behind by when the player was targeted.  (back)
  5. Volume, baby  (back)
  6. This is mostly to see how career arcs play out for the clusters, but that’s for a later article.  (back)
  7. I chose this instead of k-means for a number of reasons — the least of being which it gave the best results.  (back)
  8. From the same time frame.  (back)
  9. To calculate, just take 1.96 times the RMSE  (back)