One of the most exciting trends in fantasy football is the continued advancement of predictive analytics.
Building on the data revolution in other sports as well as businesses and industries around the world, advanced data analysis and machine learning models have continued to infiltrate a world that was once hyperdependent on film study. This is something of an upset due to the sport’s nature of small sample sizes and multidimensionality, and how these things impact any and all data collected.
I’ve had the luxury of working closely with many fantasy football data scientists over the last couple seasons as an editor and contributor here at RotoViz. Josh Hermsmeyer’s work on wide receivers using Air Yards data, Kevin Cole’s running back model, RotoDoc on quarterbacks,1 and Phil Watkins’ tight end model are some of the latest examples of pushing the predictive analytics envelope. From this experience, it’s become clear to me the perception model-builders trust the models as gospel is misrepresented. On the contrary, those who build models are generally the first to tell you where their models fall short, a subtle irony of conceptual alignment with what seems to be the biggest charge from skeptics.
Personally, I don’t build models more advanced than weighted averaging in Excel. These guys are elbow-deep in code, and every time I read one of their posts I see a new algorithm I know nothing about.2
So in some ways it makes sense that I’m writing this post, rather than one of them.3
Applying Models to Fantasy Football
First of all, one of the great things the names I mentioned above do is try to account for many of the things I’ll discuss. For example, Hermsmeyer goes through the projections in his Air Yards Countdown and comes to a conclusion on whether to accept or modify the output of the model.
It’s an important step, because as we know, football models are notoriously less predictive than models in other sports. A season is inherently a small sample size, there are multitudes of variables to account for on every play and in every game, and things like injuries or poor play impact the effectiveness of other players on the team in a much more direct way than in sports like baseball or even basketball.
But if you go back through the links I posted above, you’ll find reports of promising results. So what gives?
First of all, the models these guys and others build are extremely advanced, but they aren’t immune to human error. At RotoViz, we’re lucky to have some incredibly diligent analysts and then a Mathematics Ph.D. for an editor (RotoDoc) to review the process and findings.
I can’t personally review the process these guys take, because I don’t understand it. In a similar way, I can’t personally review the process film analysts take. I don’t know enough about the details of either. One advantage models do have is they can be tested.
Testing, Testing, 1, 2, 3
One of the biggest problems with model building is what’s called overfitting – essentially, the model’s goal is to identify trends in the data that are used to train it, but that data isn’t going to be a perfect representation of what might come next. Overfitting is when the model mimics the data too much, instead of identifying the trend.
It’s relevant here because of the way testing models occurs. In the posts here at RotoViz, you’ll often see writers reference a process where they take a small, random sample (think 20 percent) out of the set of data they have, and after they have trained their model only with the rest of the data (i.e. the other 80 percent), they can then see how good the model is at predicting that randomly pulled test group (the 20 percent again). It’s a pretty simple but ingenious process to ensure the model isn’t overfit to the sample it was trained on, and instead is able to predict data that wasn’t used to build it.
This is what’s really important to understand. While the test gives us great information about how accurate that model is, it’s still testing historic data. The data we hope to predict hasn’t happened yet. Duh.
Why am I emphasizing that data we’re trying to predict hasn’t happened yet? Because while in most instances the conditions don’t change — and thus the above described testing method doesn’t need to be tinkered with — football is a game that evolves. RotoDoc recently discussed similar in one section of his comprehensive post Win the Flex: The 2017 Edition. It bears repeating.
Model predictions imply success in similar circumstances to what they were tested in. If the league changes, it’s time to reconsider. We happen to be in a period of pretty severe change across the league, in my opinion influenced by the increased emphasis of illegal contact, defensive holding, and offensive pass interference that was announced by Dean Blandino in August 2014.
In March 2015, Jim Kloet wrote an excellent piece testing the bigger is better hypothesis for WRs. That was just seven months after Blandino’s announcement. Kloet concluded:
To be clear, a lot more work needs to be done before we can make any hard conclusions about the bigger is better hypothesis. These results suggest that the importance of WR size depends on what you’re evaluating. Savvy football fans and forecasters know that size is just one component of a player evaluation. But If I had to choose between a heavier than average WR and a taller than average WR, all other things being equal, I would choose the heavier guy.
Weight in particular was notable in his research, but we have two more seasons of data since then. I didn’t ask him to re-run the test because one example doesn’t make or break a point, but I do wonder if those rules changes have impacted his results. Anecdotally, there have been some strong seasons from smaller WRs over the past two years, like 183-pound T.Y. Hilton leading the league in receiving yards in 2016.
Beyond just the macro-level changes, team changes need to be accounted for. Situation is key for success. Volume drives points. And while volume can be fairly accurately predicted, there is still a large amount of variance from year to year that goes unexplained by even the best models.
It’s fair to question whether trying to fill that gap by forecasting is overdoing it. I still see a game where you’re competing with a bunch of other owners for one championship, where a slightly better draft can mean very little when injuries and the reality of busts start to pile up. The entire reason people fell in love with Zero RB is it can open the door to a team so dominant that some of the variance in this fickle game is mitigated.
If you’re not a mathematician, but still looking for an edge in fantasy football for 2017, the place you’ll find it is on the macro scale. Piggyback off the great work of the fantasy football data scientists you can find on these pages, and put your forecasting hat on.