RotoViz recently began posting positional fantasy football rankings. Editors RotoDoc and Shawn Siegele, Head of Analytics Josh Hermsmeyer, Senior Fantasy Analyst Ben Gretch, veteran writer Brian Malone, and myself gathered for a roundtable discussion. We talked about the difference between projections and rankings, and our respective philosophies for constructing — and utilizing — both.
Charlie: Okay guys, welcome to the first roundtable. You guys know we’re working on posting rankings for the first time. What’s your philosophical approach to constructing rankings?
Shawn: I was going to start with the idea that most rankings are overly conservative. And there are a lot of ideas rolled up in that.1
Obviously high quality projections are valuable, but projections and rankings are only value in the context of how it helps you draft. You could do a lot worse than simply starting with the ADP range in which you’re selecting, and then choose the player with the greatest potential to outperform what is implied by that ADP.2 Not every player will recognize that upside, but you need the upside built in when you’re working with an entire lineup.
Nick: I’m not sure we don’t need to take a step back here. What’s the goal in mind? Is it to be as accurate as possible at the end of the year? If so, we probably don’t just rank for things like upside. I know we’ve had some different rankers change their rankings specifically because they ranked a player well ahead of his ADP. My philosophy is different. When I’m ranking, I’m trying to get as close as I can to the correct order at the end of the season. One of the hardest parts about that is, of course, injuries. I’d be curious to hear how you guys deal with that in your ranking process.
Shawn: The goal of projections should be to accurately reflect the range of outcomes for each individual player. I think the goal of rankings is entirely about winning.
Brian: I’m with Shawn here. My rankings goal isn’t end-of-season accuracy; it’s winning. So there’s a difference between my rankings and my projections. I may have Chris Thompson projected to outscore Derrick Henry, for example, but I’m ranking Henry higher because he’s more likely to provide meaningful value in my lineup.
Charlie: Ben, what’s your philosophy?
Ben: It’s the same. I think the Thompson/Henry example is perfect. What about you Josh?
Josh H: I don’t know if my approach is different in any meaningful way, but I think rankings need to start with something objective. For me that’s projections, and the goal of projections is accuracy. At least that’s how I measure if they are any good. Then you use those projections to create rankings that introduce subjectivity and things the model doesn’t know about. Finally, I kind of gloss back over that faux precision by crafting tiers. It helps remind me that all the work, while important, is likely rife with errors.
Charlie: For myself, I do a little of each. My rankings at the top are very close to projected output. Further down, they’re more about upside or contingent upside. I’d describe my rankings as “who do I want on my team?” That’s a mixture of projection, ADP, and roster building strategy.
Charlie: Shawn, any other philosophical thoughts about rankings?
Shawn: We talked in the preparation for this discussion about the difference between rankings and projections, and how to do projections. Even if you’re not drafting directly from your projections, they’re incredibly valuable. Going through the work of doing the projections provides the information and player/team familiarity that the rest of your work is based on. And I would start by trying to understand the teams from a football perspective, for example, taking in the excellent work of Evan Silva, and then moving your football knowledge into a tool like the Projection Machine to develop a more rigorous understanding of the scenarios, the volumes, the market shares, and how that will translate into fantasy points. If you do start with more accurate projections, as Josh says, then you have a much better idea of how much a player can move if a scenario breaks the right way.3
Ben: Absolutely. To Brian’s point though, when you do projections, it all must add up, and that creates different projections for different types of players. Whatever you’re projecting for Derrick Henry in terms of volume has to take into account what you’re projecting for DeMarco Murray. Chris Thompson’s projection will not be similarly capped. His projection likely represents his true, healthy role, or something close to it, even if an injury befell a teammate like Rob Kelley. Henry’s does not capture the upside if Murray were hit by injury.
Nick: There’s another thing we need to talk about, and that’s rankings vs. projections vs. drafting. I can rank Kenny Britt WR28, but I’m not going to draft him as the WR28. That’s where a tool like our Draft Lab is so important, because it has the cheat sheet component to help you weigh your projections/rankings appropriately vs ADP.
Brian: Right. I’ll be aggressive with rankings, but I don’t reach much. I’d rather have a 75 percent chance of getting a player a little before his ADP than a 95 percent chance of getting him 1.5 rounds early.
Charlie: Have you guys changed your approach over time?
Nick: I’ve probably just changed it over the course of this conversation.
Ben: Nailed it. I’ve used tier-based rankings for drafting for a very long time, for most of the reasons Josh introduced. My preparation changes all the time, but at a certain point you must make a decision. I like to whittle all the information down to something simplified, so I’m not scrambling for news or information while drafting and can pay attention to things like draft strategy.
Charlie: How has your strategy changed over time, Brian?
Brian: I used to rely heavily on projections and use value based drafting (VBD) for my rankings. I still use the concept of value over replacement player, especially for inter-positional rankings. But now I’m much more focused on weekly production in my lineup, not season-long results.
Nick: I’m curious, Brian, does it change based on position? For example, at quarterback we’re not worried about if someone in front of them gets hurt. So, are your rankings ever different from projections at QB?
Brian: Rarely. Maybe in 2QB leagues — where I’m more worried about having warm bodies in my lineup — I’ll discount someone like Tyrod Taylor. But otherwise it’s mostly a points-per-game projection translated into rankings.
Nick: Good point. Format matters. And that’s why we’re working on rankings for a few different formats at RotoViz.
Charlie: What’s the biggest pitfall with rankings?
Josh: I’d say the biggest pitfall with rankings is the same one Shawn identified so long ago with value based drafting. If you do not use them in concert with an overall strategy and instead view each pick as tactical within the context of the draft, you can end up getting “value” but fielding a lousy team.
Charlie: Excellent point. One problem I’ve had is getting too attached to the rankings. It’s easy to get hung up on the difference between the No. 8 and No. 9 WR for example, but it’s better to think in terms of tiers.
Ben: One of the biggest pitfalls for me is that my rankings will adjust as I build out my roster. With regards to the Henry/Thompson (or Gordon/Gore) discussion, there are numerous different distributions for specific players’ ranges of outcomes. I’m willing to forgo some upside for more predictable production in some cases, which I think is a bit different than Shawn’s strategy. From my experience of drafting with him, I think he’s essentially always focusing on upside. (This means I’m probably wrong.)
Nick: I think Charlie makes a great point. If I were solely going to go by my rankings, I’d probably draft a WR with every pick because as I’ve shown in my Win the Flex article, WRs always seem to be the better “value” relative to a RB across the whole board until very late in drafts, even accounting for last year’s insane RB season relative to WRs.
Charlie: Ben, when you do rankings, are you starting with projections and adjusting, or just updating previous rankings? Or something else?
Ben: Projections. Adding in some bias toward ADP to account for the wisdom of crowds. I also like Nick’s point about different approaches at different positions. I’m more likely to rely on the deviations of my projections from the crowd at WR than RB.
Brian: Projections are just part of my background knowledge. I’m starting with ADP and adjusting from there. If I’m confident about my preference for a player with a worse ADP, I’ll move him up. Otherwise, I default to the crowd.
Charlie: What’s been the biggest surprise in your rankings (or projections) so far this year? A different way to ask that would be – which player have you had the toughest time slotting? For me it’s Elliott and Doug Martin. Figuring out how to balance the per-game output vs seasonal points.
Shawn: Seeing those names, it reminds me of some of the top-ranked seasonal players and their drafts, which tend to include some of those players, but are otherwise very conservative so that they have the built-in points to get to that time where they can deploy suspended stars.
Josh: Keenan Allen. If you start with an evidence-based approach, forecasting a player like Allen is exceptionally difficult. You need to rely on types of analysis that can be shown to not be great in the aggregate, like extrapolation. I mainly lean on volume with players like him, and he certainly will get his share of targets, but I’m not confident in any of my numbers.
Charlie: What would you like to tell the random reader that stumbles on our rankings? How should they be used?
Brian: Use them against us if you find yourself in the same league.
Ben: As if they were written in pencil, not ink.
Nick: I’d tell them to not only use our rankings, but also check out our Draft Lab and some of our other most popular articles of the offseason to put together the complete picture.
Charlie: They’re about the forest, not the trees, or something poetic like that.
Brian: You should treat them as our opinions about which players will be most “valuable” — loaded term, but let’s define it as most likely to help you win a fantasy title — in a generic fantasy football league. Adjust accordingly based on your league settings, your own player evaluations, and the flow of your draft.
Nick: If they just used our rankings, they’d probably draft Kenny Britt too early!
Charlie: The other piece of advice I’d offer is that draft strategy and roster construction are as — or more — important than the rankings.
Ben: One of the reason I love tiers is that my adjustments for players I have strong opinions about often essentially amount to moving them down or up one tier. And that feels right. I won’t reach too high, and I won’t refuse to take someone if they fall a tier below where ADP and the wisdom of the crowd values them.
Charlie: What’s your best advice for someone who wants to start doing projections or rankings. Either a book to read, a template to follow, some other source of inspiration or direction, or maybe just something from your experience you wish you’d known earlier.
Nick: I think the Rotoworld book that just came out will be an interesting read.
Josh: Yes, if you want to learn how some of the sharpest guys like Chris Raybon create their projections, I’d highly recommend reading this book.
Ben: For me it’s be diligent. Projections take a long time. Understand that. It’s incredibly rewarding, but if you aren’t careful with things like how much volume you’re projecting for the whole offense, or whether your rates are reasonable, you’ll wind up farther off that you think. If you have a player way too high or low based on other projections you see, it’s often not just that you like him that much. It’s often that one of your underlying assumptions needs to be questioned, and you should be willing to do that. This all sounds ominous, but it’s as important as it gets with projections in my mind. Sometimes those differences are very notable though. It’s not always a mistake of some sort. But those are the players I really dig into – the ones I’m way off consensus on. I need to understand if I truly believe the assumptions I’ve made.
Nick: To that point, the best way I’ve improved in DFS, outside of just game theory, is doing weekly projections. Also, talking with Shawn on the phone got me close to winning a million.
Shawn: I was rooting very hard for that game to get into overtime.
Nick: Fucking Catanzaro.
Charlie: On that note, I really appreciate and value all your contributions. Great job on our first roundtable.
Brian: Wait, when do we get to Blake Bortles and Chad Henne?
- For example, I was noting one of Brian’s responses on twitter a couple of days ago, that if you have a relatively deep starting lineup, consistency is overrated. That’s something I’ve talked about with Matthew Freedman, Josh, and others. Although different player profiles are going to lend themselves to different types of performances, we tend to both overestimate our ability to project consistency and overrate its importance. (back)
- Which is different than taking the trendiest name or the “breakout player” since a lot of the breakout is usually already priced in. (back)
- And you have a better understanding of which players are being valued within realistic ranges before the chaos starts. For example, Golden Tate appears to be valued based on his volume when all his teammates were injured and Kenny Golladay wasn’t on the team. We can find obvious errors – and players to avoid – through strong projections. (back)