Chase Stuart was nice enough to email me following my original (perhaps pig titled) piece Chubby Chasers and point out that while I had treated draft position linearly, it likely doesn’t have a linear relationship with production. He’s absolutely right.
Here’s a graph which shows Adjusted Yards as on the y-axis and and Draft Position on the x-axis.
Note that the variable sumADJAY is just the season adjusted AY number that I created to control for differences in passing volume by year.
Does that change the results of the analysis that I performed? It certainly does. I used the smoothed line shown in the chart to estimate the expected AY for each draft position and then re-ran both the draft only model, and the draft+weight model.
Here’s a summary of the draft only model:
The adjusted r-squared ended up increasing from .3 to .38 once the non-linear value for draft position is used. Note that the coefficient of .97 is really close to 1, which is roughly what you would expect considering that we’ve estimated the value before we actually added it to the regression.
Then if I add weight to the model the summary looks like this:
You can see that the p-value for WT is .168, which would not pass commonly accepted tests of significance. Also, recall that previously the coefficient for WT was 3.88. In this run, the (non-significant) WT variable’s coefficient is about 25 percent lower at 2.87.
Interestingly, I threw height back into the model to see what would happen and I was actually surprised by the result. Here’s the summary from the Draft Position + HT + WT model:
You can see that the p-value for HT is .02 and the value for WT is .01. Except the really interesting thing is that the coefficients (the Estimate column) are going in opposite directions. The kind of player that benefits from this model is a thick player. Think Quincy Enunwa at 6’2” and 225 pounds. But it also gives credit for shorter/stockier players like Bruce Ellington who is 5’9” and 197 pounds.
There are probably a few things worth addressing here. First, I think it’s always worth remembering that while this model does show the utility of using draft position in your evals, it’s also worth remembering that in the NFL draft picks only have about a coin flip’s chance of outperforming the next player picked. This is true of receivers as well.
The average number of picks that separate receivers drafted over the period in this study was 9.4 picks. However, that difference in nine picks doesn’t translate to the higher pick outperforming the lower pick at a rate better than 50 percent.
But the odds of a draft pick outperforming the next receiver picked, conditional on the first receiver being heavier than the next receiver, was 52 percent. Then if you reverse that and filter for just receivers that were lighter than the next player picked the number drops to about 49.6 percent.
If you wanted some sense as to the Adjusted Yards difference in that number we can break it down that way as well. Lighter players only outperformed the next draft pick (who is heavier) by about 31 Adjusted Yards on average over their first three years. Heavier players outperformed the next drafted player (who was lighter) by about 89 yards over their first three years.
The difference in those numbers, both the Adjusted Yards difference and the percentage that outperformed the next pick, is small enough that random chance is certainly on the table as an explanation.
I was going to post a projection table that would show predicted AY for this year’s class but I don’t have the market share information in the model yet. Unfortunately that requires a lot more data munging than I’ve had time to do this week. On an initial join that I performed it did look like both Market Share of Yards and Games Dominated would be significant variables, but there are enough missing players that I need to be sure I get that right before publishing it.
I think there are a few important implication of this exercise to keep in mind.
- Stuart is absolutely right that using a non-linear estimate for the effect of draft pick is right and ends up reducing the impact1 that weight has in the model.
- Some part of the value of draft position is likely accounted for by playing time advantages. Players like Tavon Austin and Jon Baldwin get lots of chances. Players like Aldrick Robinson and Junior Hemingway will get fewer chances.
- To the extent that any equivalencies exist between draft position and other variables, NFL teams should seek out those equivalencies because teams are in control of the playing time side of the equation. This is a defense of using models that show significant variables beyond draft position, but don’t boost the r-squared very much. With apologies to Mr. Occam, if you knew the equivalent value of increased stockiness in Adjusted Yards, and you knew that you could influence playing time, it’s very likely that you could increase your draft focus on that type of player (or trade back for them) and get something for nothing when compared to using scarce draft resources on a player whose evaluation only contains the regular emphasis on size.
- It’s also worth remembering that the specific methodology employed will get you to different places and it might be better to think about these things broadly rather than specifically. For instance, if you change the value of a touchdown to 60 yards as Max Mulitz argues we could, WT on its own would have a p-value of .10. If we leave the value of a TD at 20, but just add the 2013 draft class back into the sample, then the p-value for WT is in the .07 range.
- I think it’s still worth thinking about the idea that when big receivers do hit, the benefits can be outsized when compared to when small receivers do hit. For instance, DeSean Jackson and Brandon Marshall just had seasons that were roughly similar. But in the offseason DJax was cut from his team while Marshall was extended. That probably undermines my claim that big receivers are undervalued broadly by NFL teams, but it does support the idea that hitting on a small receiver and hitting on a big receiver are not equivalent outcomes.
That’s all. Thanks especially to Chase Stuart for his constructive comment to my earlier post.
- to the point that it would not be called significant (back)