The Box Score Scout App is one of the most powerful tools in the RotoViz arsenal, so let’s take a few minutes to explore not only what it does, but why it does it so well.
Why Have an App for Generating Player Comparisons?
Naming an app that generates player comps the “Box Score Scout” is a case of co-opting a pejorative that film watchers often hurl against metrics-based analysts. That’s right. No box-score-shaming us. We own that ish.
Like anyone that loves football, we like to study college players, and project them to the pro game. A great way to do that is to find similar players, and see how their careers progressed. The old fashioned way of doing that is to grind tape, look for stylistic similarities, and voila! comps are born. But that methodology has a few shortcomings: There’s the sommelier’s risk of sequence errors, not to mention the Tyranny of the Outlier, that can lead to some really insane comparisons. But most simply, it’s very hard to find the time to watch enough film to really understand current prospects – let alone all the historical players to whom they might compare. That’s where box score scouting actually helps get you there faster. And because numbers more easily capture all the relevant players, you’re less likely to miss a prospect. In other words, box score scouting isn’t everything – it’s the only thing.
So What Does the Box Score Scout Do?
The Box Score Scout supports two different types of queries: Similarity searches and Heatmaps.
First, for any specific player, based on a selected set of metrics, the app will use a similarity algorithm to generate a list of 11 comparable players. That helps calibrate your expectations of said player. Instead of a 1:1 comparison between the prospect you’re studying and some random Hall of Famer, you get a range of comps that will give you a better sense of not only the player’s upside, but their downside as well.
Second, you can directly compare any two (or three, or four, etc.) prospects based on the metrics that matter most to you. The color-coded heatmap helps you quickly visualize where players stand in comparison to each other.
How Does it Work?
Glad you asked. Let’s step through the inputs and outputs.
First let’s choose a player. I’m going with Sammy Watkins, since he’s been in the league awhile and we’ve got a sense of how he turned out. Note the player name is followed by three bits of information: School, draft year, and data source. “C” indicates the data will be taken from the player’s combine testing; PD indicates it will be taken from the player’s pro day.
Next we’ll choose a set of variables. I’ll define each of the choices later, but most of them are straightforward. There’s a balance between too few variables (getting comps that don’t mean much) and too many (overfitting the comps). I recommend trying a few combinations of variables and comparing / contrasting how the comps change.
There’s also an Arbitrage option. Unchecked, the App will search the historical database (which goes back to 1998) for comps. But if you check the arbitrage filter, the results will be limited to only players from the same draft class. That’s particularly useful during pre-draft evaluation time.
Finally, there’s a Draft Assumption filter option. If “Draft” is one of your variables, but the player hasn’t yet been drafted, you can plug in an assumed draft position.
Okay, let’s check the results.
The subject player is listed first, and then the 11 closest comparable players, in descending order of similarity. So in this example, the player most similar to Sammy Watkins is Michael Clayton, and the 11th-most similar is Nelson Agholor.
It’s important to consider the totality of the comps. What I mean is, don’t look at the list and say “OMG he’s comparable to Julio Jones!” You need to also recognize that Michael Clayton, Reggie Williams, and Dwayne Jarrett were very similar players based on these measures (and that might change depending on which measures you look at).
Obviously with hindsight we know that Watkins has turned into a successful pro player. But looking at these comps before he was drafted, the takeaway would have been that he had some very high-ceiling comps, and a few busts, but overall a very strong profile. Compare that to Watkins’ teammate, Robert Woods.
These comps paint a very different picture. There are a few nice matches, but mostly farther down the list, meaning they’re less similar. The point isn’t that anyone expected Robert Woods to be better than Sammy Watkins. The point is that the comps, when considered as a group, do a good job of framing the player’s range of outcomes. I think “somewhere between Justin Hunter and Jeremy Maclin” is a good description of Robert Woods.
Now let’s look at heatmaps.
Direct Heatmap Comparisons
Sometimes you don’t want an algorithmically projected set of comps; you want to directly compare two or more players to each other. Let’s hop over to the second tab on the Box Score Scout App, and build a Heatmap.
This time I’ll compare the top receiving prospect from the last three years.
Notice that the heatmap puts Amari Cooper (2014) above Watkins (2013) and Laquon Treadwell (2015). That’s because Cooper has the best combination of the variables I selected. I could come up with a few takeaways from this. First, Cooper was the best “top WR” prospect in the past three drafts. Second, Watkins is very close. It’s easy to lose sight of how good Watkins was as a prospect, given how many fantastic WRs came out of the 2014 draft class. This helps remind us how strong his profile was. Finally, Treadwell doesn’t appear to be of similar quality – at least not based on the metrics I chose to include.
But that doesn’t mean he’s a bad prospect. If we toggle back over to the similarity tab, and plug in an expected draft value of 16 (e.g. mid-way through the first round), we get these comps for Treadwell.
There’s some very solid names there, but probably a few more busts or marginal hits than show up when we look at Watkins or Cooper. That helps you put a value to each of them. Instead of thinking “Treadwell is crap compared to other recent prospects,” I realize that Treadwell has solid upside, somewhat less than Watkins or Cooper, as well as more downside. That’s a more realistic way to appraise him.
Hopefully this exercise helps you understand how to use the Box Score Scout. There’s some additional instructions on the App page. Finally, here’s the list of all the available input variables. As you’ll see, you can evaluate any of the offensive skill positions on either an athletic or production basis, using raw or percentage variables, and by career or per-game numbers.
- Agility – Sum of 3-cone and short shuttle times. Useful for evaluating running backs
- Bench – Bench press reps
- BMI – Body Mass Index
- Broad – Broad Jump, in inches
- Career Games – Total collegiate games played
- Career Market Share Receiving Yards – Market share of receiving yards based on all career games
- Career Pass Pct – Completion percentage for all career games
- Career Pass Yards – Total passing yards in player’s collegiate career
- Career Receiving Yards – Total receiving yards in player’s collegiate career
- Career Receiving Yards / Game – Average yards receiving per game, over entire collegiate career
- Career Rushing Yards – Total rushing yards in player’s collegiate career
- Career Rushing Yards / Game – Average yards rushing per game, over entire collegiate career
- Cone – 3-cone time
- Draft – Player’s overall draft number
- Explosion – Sum of vertical and broad jumps. Useful for evaluating running backs and wide receivers.
- Forty – 40-yard dash time
- Freak – A score that attempts to assess a player’s NFL TD-scoring potential.
- Games – Games played in final college season
- Height – Height in inches
- Market Share Receiving TDs – In the specific collegiate season identified in the player input field (e.g. “2013” or “2014”)
- Market Share Receiving Yards – In the specific collegiate season
- Market Share Rushing Yards – In the specific collegiate season
- Pass Attempts / Game – In the specific collegiate season
- Passing AYA – Adjusted yards per attempt in the specific collegiate season
- Passing Yards – Total passing yards in the specific collegiate season
- Passing Yards / Game – Average passing yards per game, in the specific collegiate season
- Receptions / Game – Average receptions per game, in the specific collegiate season
- Receiving TDs / Game – Average receiving TDs per game, in the specific collegiate season
- Receiving Yards / Game – Average receiving yards, in the specific collegiate season
- Receiving Yards / Reception – Average yards per reception, in the specific collegiate season
- Rushing Attempts / Game – Average attempts per game, in the specific collegiate season
- Rushing TDs / Game – Average rushing TDs per game, in the specific collegiate season
- Rushing Yards / Carry – In the specific collegiate season
- Rushing Yards / Game – In the specific collegiate season
- Shuttle – Short shuttle time
- Strength of Schedule – Scaled to 100. Higher is more difficult
- Speed Score – Weight adjusted 40-yard dash time
- Vertical – Vertical jump, in inches
- Weight – Weight, in pounds