Analyzing a Fantasy

Some of the folks here at Infinity Interactive are avid players of fantasy sports, and this year, they convinced me to join their Fantasy Football league. Two months into the season, what started as a casual game has turned into a trip through a data analytics wonderland as well as what will hopefully be a recurring series of posts here looking at various aspects of the data analysis that I’ve been doing.

I Feel Your Pain, Charlie Brown

There are lots of varieties of fantasy football (or “FF”). The core of the game, for those who aren’t familiar with it, is that each player (in FF parlance, an “owner”) compiles a virtual team consisting of actual gridiron football players from the National Football League (NFL); each week of the NFL season, each owner selects a subset of their players to be “active”, and then scores points depending on the real-world performance of those players in actual NFL games.

What this means is that one of the more important parts of the game happens before the season really even starts: the “draft”, when the owners take turns selecting players from the NFL to be on their respective rosters. I had never played any fantasy sports before this year, but as a statistics and game theory geek, I’ve spent some time studying things like optimal draft strategies, so I wasn’t too worried about our league’s draft, and I felt even better after some Googling that morning led me to one of the many websites out there that provide “draft value” scores for each player before the start of the season. Armed with a way to quantify the value of each player, plus a game-theoretical model of how to best select them in competition with the other owners, I came out of the draft fairly happy (especially since I grew up as a huge fan of gridiron football, and through sheer coincidence, coldly following the numbers had led me to a roster that was 20% from my beloved Oakland Raiders).

And then I discovered which variety of FF we were playing.

Oops.

That variety is, I’ve since discovered, a very common one — in FF jargon, it’s a “14-week, 8-team, ESPN PPR head-to-head league w/ a 4-team playoff”. The problem was that I, having never played FF before, had made a lot of assumptions about how the scoring worked in our league, and almost all of those assumptions turned out to be bad ones.

For example, take those terms “head-to-head” and “playoff”. I had assumed that the winner of the league was the owner who, at the end of the season, had accumulated the most points. In many FF variants, I discovered, what actually happens is that each week, every owner is paired up against another owner in a “head-to-head” matchup, with the owner who scores the most points between the two winning that particular week; at the end of the season, the owners with the most wins enter a playoff — another few rounds of head-to-head matchups, with the losing owners each week being eliminated, until only one owner is left as champion. The total number of points scored over the course of the season doesn’t matter; for the most part, in fact, points don’t matter at all beyond determining the winner in a given head-to-head matchup. Since I had assumed that total points did matter, I had drafted players based on their odds of scoring a reasonable number of points over the course of the entire season; if I suspected that a player would score very highly in a few games and then poorly for the rest of the season, that was fine with me because their average would still be relatively high. In a head-to-head league, though, what that meant was that I might have a few weeks where I would win the head-to-head matchup overwhelmingly, but then also a lot more weeks where I would lose, and not make the playoffs at the end of the season despite having a high point total for the season.

I had also been unaware that in ESPN leagues, you can change your roster of players, either by trading them with other owners, or replacing your players with ones who are not currently on any other owner’s roster (“adds/drops”). Since I had thought I was going to have the same roster all season long, I had drafted to try to ensure a stable performance over the entire season, so that if Player A, who might give me an average of 10 points per week, were injured in a given week, I could replace them with Player B might give me an average of 9 points. In a league with trades and adds/drops, though, it is usually more effective to have a wider spread between your primary and secondary players — Player A, your “starter”, might average a consistent 15 points, while Player B, your “backup”, might average 6 points where some times he would score 3 and others 10; then, over the course of the season, the goal would be to try to modify your roster to minimize how often you have to actually play those less-reliable backups, and when you do play them, hope that you catch the “boom” side of their boom-or-bust variance.

And finally, I had also missed that it was a “PPR”, or point-per-reception, league. That means that every player scores one additional point every time they catch a pass, and in turn means that, all other things being equal, players whose role on a real-world team is to catch passes more than it is to run with the football are more valuable; not being aware of that, I had used draft values from a site that didn’t use PPR scoring, and as a result I had drafted a lot of run-centric players who were thus less valuable than I had thought they would be.

The end result? The roster I had after that initial draft was weaker than I thought because of too many running backs, had too many starters who were likely to be high performers in only a few games, and had too many backups who were fine but who I had drafted too early at the expense of getting stronger and more reliable starters. Weeks later, I found a website that would analyze your draft choices based on pre-season projections of each players’ performance (and taking into account the specific scoring rules of your league), and this was the verdict it gave:

Let us say this as nicely as we can. This team is brutal. It is below average and/or too thin at all three core positions (quarterback, running back and receiver). To make this team into a serious contender, you are going to need to be extremely active in trades and on the waiver wire. You esentially need to turn over significant parts of this roster.

Data To The Rescue

Within a day or two of the draft, I had come to the same conclusion: I needed to overhaul my team. Since all of the obviously good players had already been taken in the draft, though, I feared I was going to have to use a Moneyball approach to that overhaul: using analytics to try to find players who might perform especially well in a given situation but who had slipped under the radar of the other owners. Moreover, I was probably going to have to keep overhauling my roster each week, since players who were likely to be consistent top performers were unlikely to drop into my lap. Fantasy Football is big business these days, so there are paid services out there that will give you all of that analysis, but I was intrigued by the idea of trying to do it on my own.

So I did it: I’ve created a relatively simple model for tweaking my roster, and ceded control of my team to it almost entirely. Along the way, I’ve written some Perl scripts, started to learn the data analysis language R, brushed off some old data analysis skills like doing exponential smoothing and linear regressions, and generally had a lot of fun. And the best part is that it may have actually worked! To avoid falling down the rabbit-hole of constant re-optimization, I’ve decided to hold off on doing a rigorous analysis of how good my model is until after the season is over, but as I write this (after week 8 of the season), I’m currently tied for second place in my league, despite that “brutal” initial roster that footballguys.com rated as worst in my league.

In some upcoming posts here, I hope to dig into some of the data analysis techniques and tools I’m already using in my model (e.g. exponential smoothing, range estimates, and Perl’s Text::CSV module). I’m also hoping to continue revising my model and the tools I’ve written around it, so I should be making at least a few posts on my efforts there (e.g. automated web scraping to gather my source data, dumping the data into a SQL database to make it easier to do ad-hoc queries, and hopefully even some forays into pattern recognition and machine learning). And once the season is done, I hope to do some analysis of how well my model actually fared, both in absolute terms as well as pitting it against professionals.

To wrap up this initial post, though, I thought I would include some sample output from the main Perl script that I’m using to help me decide what to do each week. My team’s abbreviation is YRMM, and this week I’m up against team MSSX; in this command, I’m asking my script to show me both of our rosters for this week, plus what it considers the two best available replacement players at each position:

My model thinks I will probably win (my 136 expected points for my current active roster in the summary at the bottom, vs MSSX’s 116), but it also telling me that a large part of that margin comes from having Alex Smith at quarterback, and that he’s pretty risky — it predicts that he’ll probably score anywhere from 14-25 points, with a high likelihood that his actual score will be outside of that already-wide range. If I’m not comfortable with him at quarterback, it suggests that I could replace him with LA Ram Case Keenum (which would give me a range of 11-21 points but with much less risk of going outside of that range) or Arizona Cardinal Carson Palmer (though it pegs him as even riskier than Smith).

Meanwhile, after a flurry of activity so far this season (86 roster changes, 36 adds/drops, and 4 trades), I’ve been able to rework my run-centric team into one that is more evenly balanced between running backs and wide receivers, all of whom look to do pretty well this week, and there aren’t any available players at either position who look better than the ones that I already have.

On the other hand, I could potentially improve my expected score a tiny bit, while also reducing my risk a bit, if I swap out tight end Jason Witten for Zach Ertz. It’s only an improvement of less than a point of expected value, though, so I think I’ll use my time tonight to go work on trying to automate my web scraping so that preparing my lineup for next week will be easier…

Tags: technology culture passions perl