revolutionary tools.  groundbreaking articles.  proven results.

The Scouting Model vs. The Algorithm

This piece was originally published on in spring of 2012 and addresses fantasy scoring only – although fantasy scoring is actually a pretty good proxy for our collective impression of what makes a good wide receiver.  To put it another way, try to think of a successful real world receiver who isn’t a successful fantasy receiver.  This piece pits a model that includes simple box score stats and also Market Share of college team yards vs. Draft Position in a game of “Who can pick the better WR?”  

Also, while Percy Harvin’s 2012 season will now make some of the examples in this piece seem off, Harvin played the majority of his rookie contract at a level that would still make these conclusions valid.  Basically, teams should not be drafting WRs hoping that in their 4th pro season they finally start outproducing WRs picked after them. 

I sort of assume that some NFL teams use various forms of analytics in their draft process.  I would be really shocked to find out that the Packers don’t use some form of analytics.  The way that they won a Super Bowl in a year that they were decimated by injury was the first red flag for me that they are doing something a little different.  Both Greg Jennings and Jordy Nelson are receivers that a statistical model would have liked very much coming out of college, and yet the Packers got both receivers in the 2nd round.

But I also assume that most teams are sort of blissfully unaware that analytics might provide any value.  I’ve discussed in the past that the league’s phrase “Stats are for Losers” puts it in sort of a weird anti-science place.  It’s also common to hear people say “Well, I don’t think you could use stats for that.  It’s too complex.” (every time I hear someone say that, I think “Oh, because you’ve probably tried right?”).

One thing I’ve talked about is how NFL teams don’t optimize the way that they select receivers.  While NFL draft position does have some explanatory power over receiver production, you also have to consider that the system is rigged in favor of the draft evaluators.  The same coaching staff/front office that selects a player is also the same group of people that assign playing time.  If there are mistakes in the draft evaluation, they are going to carry forward.  So really the explanatory power that draft position has on receiver production isn’t very impressive.

One thing I did recently to try to account for the inherent bias in the system that favors draft evaluators was to look at the draft in terms of just wide receivers that had been picked right after each other.

To think about what I did in simple terms, imagine that I created a game that was going to pit an NFL draft evaluator (except we’ll use draft order as a proxy for the opinion of an evaluator) against a formula.  The evaluator and the formula will be presented with two wide receivers and will have to try to guess which will have more production in the early part of their career.  Each set of two wide receivers will only be guys who had been picked after each other.

For example, in the 2009 draft, the game would go like this:

Heyward-Bey vs. Crabtree

(evaluator picks DHB, algorithm picks Crabtree, algorithm wins)

Crabtree vs. Maclin

(both the evaluator and the algorithm pick Maclin)

Maclin vs. Harvin

(both pick Maclin)

Harvin vs. Nicks

(evaluator picks Harvin, algorithm picks Nicks, algorithm wins)

The evaluator would make its guess and the algorithm would rely on its formula for its guess.  Then when the evaluator and the formula disagreed, we would tally up which was right and which was wrong.

But again, for our purposes we’ll just use draft order as a stand in for the evaluator.  So can draft order beat a simple algorithm at a game of prediction when choosing between two wide receivers?  It turns out it can’t.  The algorithm would win.

Out of the 120 receiver pairs that I looked at over the last decade, my simple algorithm disagreed with the draft order in 49 instances.  Out of those 49 disagreements over which player should be taken, the algorithm was right almost 60% of the time, compared with 40% for the draft order.


But the interesting thing is that when the algorithm disagreed with draft order and the algorithm was right, it was right by a larger margin than the draft order was right by when it won.  The following graph shows this difference.  When the algorithm was right, the receiver that the algorithm picked outperformed the receiver that the draft order picked by 2.84 fantasy points per game.  But when draft order was right, the receiver that it picked outperformed the algorithm’s receiver by only 2.48 fantasy points/game.


The results are even more impressive for the algorithm if you consider that the algorithm is always taking the lower picked receiver.  Even though we are trying to isolate for the effect of draft position, the algorithm still does only get to be right on lower picked receivers, who are also going to have marginally fewer opportunities.  The graph below shows that on the algorithms wins, it is winning with a receiver that went off the draft board about 11 spots behind the competing receiver.  In the draft board’s 20 wins, it’s winning receiver went off the draft board 12 spots in front of the algorithm’s receiver.  Even though we are trying to level the playing field for the algorithm, it is still at a disadvantage when it does win, and an even greater disadvantage when it loses.


Even when we try to level the field in a wide receiver picking contest, an algorithm is still at a disadvantage against the NFL’s current evaluators, and yet the algorithm can still “outpick” the evaluators.  This is another in the growing list of reasons why NFL teams should increase their use of analytics.

recent and related...

in case you missed it...

Week 10 Primetime Slate DFS Breakdown

Who are the best plays for the Week 10 DFS primetime slate on DraftKings and FanDuel? Tyler Loechner runs through notes on the primetime short slate featuring SNF/MNF games.1 DSF Week 10 Primetime Slate DFS Week 10 Primetime Slate theory thoughts: The games on this slate might not have “shootout”

Read More

Week 11 Waiver Wire Advice: Top Targets At Each Position

Looking for Week 10 waiver wire advice for fantasy football? You’ve come to the right place. We’ll give you some of the top targets at each position so that when you submit a waiver claim, you do it with confidence. This article will run through the top players available in

Read More

High Staked: What Went Wrong

Veteran high stakes fantasy football player Monty Phan chronicles his season. I can officially say things didn’t work out this season like we’d hoped. For the first time since 2014, none of the FFPC high-stakes teams I run with three friends are likely to advance to the league playoffs or

Read More

Nick Chubb Almost Destroys Week 10 and Christian McCaffrey Actually Destroys Everyone Else’s Fantasy Teams: What Expected Points Are Telling Us After Week 10

If you spend any time reading RotoViz, eventually you’ll hear us talking about expected points. Expected points (EP) are the number of fantasy points that a target or carry should score based on game situation — down, distance, and field position. In other words, expected points allow us to transform

Read More

Sign-up today for our free Premium Email subscription!

© 2019 RotoViz. All rights Reserved.