Re-Working 1999 Preseason, New DIII Members
It's a long offseason, especially when I didn't have recruiting or spring ball to keep me occupied. What did I do over this long offseason then? Tweak, tweak, and tweak some more. These tweaks can produce some pretty major changes in the rankings for some teams, but overall they have improved the system's win-loss accuracy by about 1% overall (which means it probably will get one more game correct per week), and point-spread standard error has improved from about 15.78 points to about 15.44.
Here's what I changed:
- Method for establishing 1999, 2000, 2001, & 2002 preseason ratings
- Method for establishing preseason ratings for teams new to Division III
1999 - 2002 Preseason Ratings
In the earliest version of my model, I started every team in 1999 with a perfectly average rating of 0.500 - AdjO of 24 & AdjD of 24 - and let the system do all the sorting. This led to some crazy results. The WIAC only played a handful of non-conference games that year against DIII competition (most were vs. DII competition, and the WIAC won most of those), so their teams were vastly underrated. The IIAC's only non-conference games that year were in the playoffs, so their ratings were all over the place also. In all, I wasn't very satisfied with this method.
My next attempt at 1999 preseason ratings involved an iteration of sorts. I took the end-of-season results from 2000-2003 (with every team starting at 0.500 in 1999), and used a weighted average to develop preseason ratings for 1999.
For every other season, I use a weighted average of the previous four seasons ratings to develop a preseason rating. For 2000-2002, I didn't have four years worth of ratings, so I just regressed every team to the national average. This was how my model has been sitting the entirety of the 2015 season, and I was moderately happy with how everything looked.
But then I discovered that the Massey Ratings have archived ratings all the way back to the founding of Division III, in 1973. His games are in a text format that would require a lot of work to incorporate into my system, but copy-pasting each team's end-of-year offensive and defensive ratings would be easy.
So what I have done is include each team's ratings (adjusted to match the scale of my system) from 1995-1998 as a method for determining every team's preseason ratings from 1999-2002. I use the same weights for Year n-1, Year n-2, etc. as I do for every other season, but to avoid over-fitting to a different system, I regress each year of Massey Ratings toward the mean by 20%.
The effect of this method is that the relative strength of conferences evens out quite a bit. If you've looked at my All-Time Conference Rankings, you've probably noticed that every conference is pretty jumbled the first few years and the system tries to figure things out. With this update, the early seasons are a lot less chaotic.
Another visible change is the relative strength of teams based on their geographic position. Teams on the West Coast have improved, and teams on the East Coast have not. I good example of this is Hobart's 2016 preseason ranking. Before including Massey's 1995-1998 ratings, Hobart was sitting at around #30 in my system, and since the change the have dropped below #40.
This might seem like folly, allowing seasons from 17-20 years ago to impact a team's current rating. I would argue to the contrary though; in such a large division, and with so few non-conference games, using relative historical performance increases overall accuracy.
Teams New to DIII
Being an alum of Wartburg in the IIAC, and with the IIAC adding Nebraska Wesleyan to the conference this year (the first non-Iowa school in the conference ever, in over 100 years), I wanted to make sure I was getting a good preseason rating for NWU.
Last season, I lumped every team playing their first year of Division III into the same "pot," whether they were a startup program or making the transition from DII. Historically every team in this "pot" would finish their first season with a rating of around 0.050, so that's what I used for their first-year preseason ratings. I liked this method because it was easy, but it was pretty inaccurate at times. There's a huge difference between 2015 Finlandia and 2000 Westminster (Pa.).
My first attempt to fix this method was to separate each team into four different pots:
- Pot 1: Startup Programs
- Pot 2: Former JUCOs (there have been 3 since 1999, do you know who they are?)
- Pot 3: Former NAIA
- Pot 4: Former DII
Using this method, I found that each pot had predictable different results for their first season. Former DII teams were right around average, NAIA teams would finish around 0.150, JUCOs near 0.075, and startups around 0.025. Still though, I wasn't satisfied in the margin of error for DII and NAIA teams.
Right around this time I was also shoring up my 1999 ratings, so I decided to use the same method for DII and NAIA teams - Massey Ratings regressed 20% to the mean.
This method definitely had a drastic change in the model's "opinion" of the American Southwest Conference, who has seen the most influx of DII and NAIA teams. Before making the change, I was projecting the ASC as the 16th best conference in the country, and since the change they're projected as the 7th best (the large change is because they have the two most recent cross-over teams - McMurry & Belhaven).
In summary, I feel I must apologize to the East, because my model hates you now, and the ASC, because I may have been a little harsh in my All-Time Conference Breakdown.