It’s All Ball Bearings Nowadays, or It’s All Small Sample Sizes


The post below is a re-post of an article originally written on in 2012.

I’m starting to feel about sample sizes the way that Fletch felt about ball bearings.  If you haven’t seen the movie I don’t think I’m spoiling the plot at all by saying that he thought they were pretty important.

The cacophony (yeah, that just happened) of opinions surrounding the NFL’s spring draft is a target rich environment for the application of small sample sizes.  Let’s think about all of the way small samples end up with outsized importance being assigned to them.

The Combine.

The combine is a few days of drills that aren’t even football related except that they also involve speed and strength.  But the draftniks assign all manner of importance to the combine.

This guy needs to have a good combine, they might say.  This guy needs to run at least a X.XX.

But the combine is a small sample.  It’s a player’s performance on a single day.  Yet outsized importance is attached to that day and it can counteract a collegiate record that might be four years long.  Isn’t that screwed up?  Every Saturday in the fall the player goes out after a week of coaching and plays in a football game, and after 4 years you might have 50 or so games as a record, and that record can be affected by a single day of workouts?

Speed is pretty important to football, but the best wide receivers are rarely the fastest, and as far as running backs go, if you show me a running back who averaged over six yards per carry for his college career, I’ll show you a running back who will have a good Speed Score (and I’ll even bet you can pick one up in the fourth round).

Moving on.

Player comparisons.

Player X is like Larry Fitzgerald.  Wait, no he’s not.  Maybe he’s like Anquan Boldin.  Wait, no, he’s really like a bigger Santonio Holmes.

Saying someone could be like Larry Fitzgerald is applying a comparison that is supposed to have meaning, but is essentially just a small sample size.  Larry Fitzgerald is just one guy with a particular skillset who happened to be a good pro.  There’s no guarantee that another guy who came along with a similar skillset would have Larry Fitzgerald’s success.

But the other problem with player comparisons is that when you try to shoehorn them into a type, you ignore the possibility that they might not fit any type, yet might be good in spite of that.  They might be a new kind of good.  To ignore this possibility is to say basically that only a few types of successful player exist and every player coming out of college needs to fit into one of the pre-made molds.  This is preposterous.

Here’s another problem with player comparisons.  Even the EXACT SAME PLAYER can have different results.  Forget about trying to predict the future by coming up with a reasonable approximation (Player A will be successful because he has the same skillset as Larry Fitzgerald).  Players, not just similar players – exact players, have different results. Randy Moss had about as bad of a season as you can have in Oakland in 2006, then he had a record setter the next year.  If the exact same player can have a range of results based on situation, single player comparisons between pros and college players become ridiculous.  The college player is almost assured of walking into a different situation than the one that allowed the pro to acquire whatever reputation we assign to him.  Similarity is not destiny.

(Side note: This might seem like I’m making an argument that would counter my use of similarity scores.  However, my similarity scores compare 20 player seasons typically for the very reason that similarity does not equal destiny and because single player comparisons create a ridiculously small sample.)

If he can do it, he can do it.

This is a phrase I’ve been using more of lately.  It basically means that a player’s accomplishments on the field can generally speak for themselves.  If a guy catches 1500 yards in a season, he figured out a way to get open.  Don’t rob him of his skills because he doesn’t seem good in a way we’ve ever seen before.  He figured out a way to do it.  He went out every Saturday and figured out a way to beat double teams, but we’re going to downgrade him because he didn’t run a sub 4.4 forty and we don’t understand how he did it?

What we do

Sign-up today for our free Premium Email subscription!

© 2019 RotoViz. All rights Reserved.