Monday, March 9, 2009

Are college sports polls more accurate later in the season?

The question I'm interested in here is whether the AP Poll rankings in college football and basketball are more accurate later in the season when pollsters have seen more games and have more information about teams' abilities. To measure this I look at whether upsets--a lower ranked team beating a higher ranked team--occur more often early in the season. This is what we'd expect to see if the rankings get more accurate as the season progresses. In the graphs below I show the percentage of games between ranked teams that end in upsets, graphed against the week of the season. These numbers are averages from college football seasons 1936-2007 and college basketball seasons 1956-2008. Since the number of teams that are ranked has changed over time and since some people believe that the top 10 rankings are much more accurate, I've also shown graphs that only include games between teams ranked in the top 10.

The main point is that all four graphs show no decrease over the course of the season. The overall average is close to 40% in all four graphs. Although the numbers have random variation, the overall trend is flat for each. On average, upsets take place just as frequently in week 15, for example, as in week one.
This is interesting because it means that the AP Poll journalists are just as good at ranking teams when they haven't yet seen them play as when they've seen them play for a whole season. Of course, they've seen the teams or the members of the team play in previous seasons, which probably accounts for this. Before putting these numbers together my guess was that these graphs would show a decrease in upsets over the season.

My data sources are, Prof. John Trono's NCAA Basketball Archive, James Howell's College Football Scores, and
Only games where both teams are ranked are included. When teams play on the day a poll is released, their rankings are based on the new poll.
I include preseason polls, Bowl games and NCAA tournament games in the results. For football, all Bowl games are assigned a week number one greater than the final week of the regular season. For basketball, all NCAA Tournament games are assigned a week number one greater than the final week of the regular season. In both cases this is because there are no new polls after the end of the regular season and before any of the Bowls or Tournament games.
The basketball results above are missing seasons 1963-1967 , 1978 and 1979.
In years when there is no preseason poll, I exclude all games that take place before the first poll, but I start counting the number of weeks when the games start.

1 comment:

  1. Since you've got the data, how about P(upset) vs Delta(rank)? I guess I'd expect something that starts at .5 and monotonically decreases with larger separation.