Archive for the ‘Articles’ Category

Playoff predictions 2006, after week 16

Tuesday, December 26th, 2006

Here they are.

Once the playoff teams are finalized, I’ll do a year in reveiw post and a preview of improvements on slate for next season.

 

Percent probability of team finishing in each place within the division

AFC EAST

Team 1st 2nd 3rd 4th
New England 100.0 0.0 0.0 0.0
NY Jets 0.0 100.0 0.0 0.0
Buffalo 0.0 0.0 100.0 0.0
Miami 0.0 0.0 0.0 100.0

AFC NORTH

Team 1st 2nd 3rd 4th
Baltimore 100.0 0.0 0.0 0.0
Cincinnati 0.0 100.0 0.0 0.0
Pittsburgh 0.0 0.0 100.0 0.0
Cleveland 0.0 0.0 0.0 100.0

AFC SOUTH

Team 1st 2nd 3rd 4th
Indianapolis 100.0 0.0 0.0 0.0
Tennessee 0.0 69.68 30.32 0.0
Jacksonville 0.0 30.32 69.68 0.0
Houston 0.0 0.0 0.0 100.0

AFC WEST

Team 1st 2nd 3rd 4th
San Diego 100.0 0.0 0.0 0.0
Denver 0.0 85.48 14.52 0.0
Kansas City 0.0 14.52 85.48 0.0
Oakland 0.0 0.0 0.0 100.0

NFC EAST

Team 1st 2nd 3rd 4th
Philadelphia 71.68 28.32 0.0 0.0
Dallas 28.32 71.68 0.0 0.0
NY Giants 0.0 0.0 100.0 0.0
Washington 0.0 0.0 0.0 100.0

NFC NORTH

Team 1st 2nd 3rd 4th
Chicago 100.0 0.0 0.0 0.0
Green Bay 0.0 100.0 0.0 0.0
Minnesota 0.0 0.0 100.0 0.0
Detroit 0.0 0.0 0.0 100.0

NFC SOUTH

Team 1st 2nd 3rd 4th
New Orleans 100.0 0.0 0.0 0.0
Carolina 0.0 75.08 24.92 0.0
Atlanta 0.0 24.92 75.08 0.0
Tampa Bay 0.0 0.0 0.0 100.0

NFC WEST

Team 1st 2nd 3rd 4th
Seattle 100.0 0.0 0.0 0.0
St Louis 0.0 83.9 16.1 0.0
San Francisco 0.0 16.1 75.4 8.5
Arizona 0.0 0.0 8.5 91.5

Generated: Tue Dec 26 00:38:01 EST 2006

AFC Percent Probability Playoff Seeding

Team 1st 2nd 3rd 4th 5th 6th Total
San Diego 91.28 8.72 0.0 0.0 0.0 0.0 100.0
Baltimore 8.72 71.08 20.2 0.0 0.0 0.0 100.0
Indianapolis 0.0 20.2 64.86 14.94 0.0 0.0 100.0
New England 0.0 0.0 14.94 85.06 0.0 0.0 100.0
Denver 0.0 0.0 0.0 0.0 74.84 10.64 85.48
NY Jets 0.0 0.0 0.0 0.0 22.84 61.22 84.06
Cincinnati 0.0 0.0 0.0 0.0 2.32 21.02 23.34
Kansas City 0.0 0.0 0.0 0.0 0.0 2.86 2.86
Jacksonville 0.0 0.0 0.0 0.0 0.0 2.3 2.3
Tennessee 0.0 0.0 0.0 0.0 0.0 1.96 1.96
Buffalo 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Miami 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Cleveland 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Pittsburgh 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Houston 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Oakland 0.0 0.0 0.0 0.0 0.0 0.0 0.0

NFC Percent Probability Playoff Seeding

Team 1st 2nd 3rd 4th 5th 6th Total
Chicago 100.0 0.0 0.0 0.0 0.0 0.0 100.0
New Orleans 0.0 100.0 0.0 0.0 0.0 0.0 100.0
Philadelphia 0.0 0.0 71.68 0.0 28.32 0.0 100.0
Dallas 0.0 0.0 28.32 0.0 71.68 0.0 100.0
Seattle 0.0 0.0 0.0 100.0 0.0 0.0 100.0
NY Giants 0.0 0.0 0.0 0.0 0.0 68.64 68.64
Carolina 0.0 0.0 0.0 0.0 0.0 10.58 10.58
Atlanta 0.0 0.0 0.0 0.0 0.0 8.3 8.3
St Louis 0.0 0.0 0.0 0.0 0.0 7.64 7.64
Green Bay 0.0 0.0 0.0 0.0 0.0 4.84 4.84
Washington 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Detroit 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Minnesota 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Tampa Bay 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Arizona 0.0 0.0 0.0 0.0 0.0 0.0 0.0
San Francisco 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Generated: Tue Dec 26 00:38:01 EST 2006

 

Graphical Comparison of DVOA and NFL-Forecast after Week 13, 2006

Tuesday, December 5th, 2006

I’ve prepared the comparison between playoff predictions based on DVOA power ratings and NFL-Forecast playoff predictions in graphical time-series form , from week 9 through week 13. The filled circles represent DVOA predictions while the open circles represent my predictions. Hopefully the team-based color coding that I used is self explanatory. I’ll include the Week 13 DVOA ratings as soon as Mike Harris posts them.

The first set of graphs shows the division races (I’ve shown only teams that have had a significant chance since week 9). All 8 division races appear to be converging to selecting a champion, using both the DVOA and NFL-Forecast projections. DVOA ratings did significantly better in predicting the AFC West race, recognizing the superiority of the Chargers compared to the Broncos. In the NFC East race, my software had greater misplaced confidence in the Giants than DVOA, but transferred this misplaced confidence to the Eagles. Neither system really picked out the Cowboys as the division winner significantly better that the other. In the rest of the races, the predictions were comparable, with DVOA converging to apparent division winners slightly faster than NFL-forecast, with the exception of the NFC West race. My system still holds a bit of hope for the Jets, to eek out the AFC East title, while the DVOA has declared the race all but over. In contrast, the Wild Card races are not converging, with only 4 weeks remaining in the season.

The Wild Card race this year appears to be a white noise process, with team’s fortunes rising and falling on a weekly basis. The early front runners as predicted by both systems, the Broncos and Giants, have lost much of their lead. It will be interesting to see if either system outperforms the other in the WC race, but the lack of convergence would seem to indicate that any better performance of one system over the other would be attributable to luck rather than reproducible predictive ability. Nevertheless, forecasts like these during dynamic non-converging playoff races do help quantify the race if it is understood to be a snapshot in time.

Only preliminary conclusions can be made, since the playoff races are not over yet, and even when they are, a single season is still a pretty small data set. But this playoff forecasting exercise appears to be pointing to a small but significant advantage of DVOA compared to win-only systems in predicting future results.

Playoff predictions, 2006 after week 13

Tuesday, December 5th, 2006

Here are this week’s playoff predictions. If you’re a fan of one of the five 7-5 AFC teams or one of the four 6-6 NFC teams, you’ll want to try out the software this week if you haven’t already done so. There are lots and lots of scenarios to analyze and my software automatically analyzes all the tiebreakers for you. It’s playoff madness!

Tomorrow, I’ll post a graphical week-by-week comparison of my forecasts and the DVOA forecasts, starting from week 9 until now.

Percent probability of team finishing in each place within the division

AFC EAST

Team 1st 2nd 3rd 4th
New England 85.82 14.18 0.0 0.0
NY Jets 14.18 79.94 5.72 0.16
Buffalo 0.0 4.2 52.86 42.94
Miami 0.0 1.68 41.42 56.9

AFC NORTH

Team 1st 2nd 3rd 4th
Baltimore 90.06 9.8 0.14 0.0
Cincinnati 9.82 86.7 3.46 0.02
Pittsburgh 0.12 3.38 73.04 23.46
Cleveland 0.0 0.12 23.36 76.52

AFC SOUTH

Team 1st 2nd 3rd 4th
Indianapolis 99.38 0.62 0.0 0.0
Jacksonville 0.62 80.26 18.08 1.04
Tennessee 0.0 17.48 59.12 23.4
Houston 0.0 1.64 22.8 75.56

AFC WEST

Team 1st 2nd 3rd 4th
San Diego 94.58 4.54 0.88 0.0
Denver 3.16 51.08 45.76 0.0
Kansas City 2.26 44.38 53.36 0.0
Oakland 0.0 0.0 0.0 100.0

NFC EAST

Team 1st 2nd 3rd 4th
Dallas 87.18 10.88 1.9 0.04
NY Giants 5.82 58.76 31.64 3.78
Philadelphia 6.98 27.56 48.72 16.74
Washington 0.02 2.8 17.74 79.44

NFC NORTH

Team 1st 2nd 3rd 4th
Chicago 100.0 0.0 0.0 0.0
Minnesota 0.0 68.6 30.24 1.16
Green Bay 0.0 31.14 65.96 2.9
Detroit 0.0 0.26 3.8 95.94

NFC SOUTH

Team 1st 2nd 3rd 4th
New Orleans 86.5 11.12 2.38 0.0
Atlanta 3.74 53.52 42.54 0.2
Carolina 9.76 35.36 54.82 0.06
Tampa Bay 0.0 0.0 0.26 99.74

NFC WEST

Team 1st 2nd 3rd 4th
Seattle 97.98 1.9 0.12 0.0
St Louis 0.22 38.86 57.48 3.44
San Francisco 1.8 57.32 35.14 5.74
Arizona 0.0 1.92 7.26 90.82

Generated: Tue Dec 05 00:20:05 EST 2006

AFC Percent Probability Playoff Seeding

Team 1st 2nd 3rd 4th 5th 6th Total
Indianapolis 47.86 34.56 11.76 5.2 0.3 0.22 99.9
San Diego 41.5 30.56 17.6 4.92 4.22 0.98 99.78
Baltimore 7.3 21.14 37.78 23.84 3.8 2.74 96.6
New England 3.2 11.9 26.52 44.2 6.36 3.76 95.94
Denver 0.08 0.82 1.4 0.86 29.22 18.36 50.74
Cincinnati 0.04 0.56 1.72 7.5 22.3 20.68 52.8
Kansas City 0.0 0.32 0.94 1.0 8.92 14.42 25.6
NY Jets 0.0 0.04 2.0 12.14 14.38 22.02 50.58
Jacksonville 0.02 0.1 0.28 0.22 10.36 15.6 26.58
Pittsburgh 0.0 0.0 0.0 0.12 0.0 0.28 0.4
Buffalo 0.0 0.0 0.0 0.0 0.06 0.52 0.58
Tennessee 0.0 0.0 0.0 0.0 0.06 0.24 0.3
Miami 0.0 0.0 0.0 0.0 0.02 0.12 0.14
Houston 0.0 0.0 0.0 0.0 0.0 0.04 0.04
Cleveland 0.0 0.0 0.0 0.0 0.0 0.02 0.02
Oakland 0.0 0.0 0.0 0.0 0.0 0.0 0.0

NFC Percent Probability Playoff Seeding

Team 1st 2nd 3rd 4th 5th 6th Total
Chicago 96.22 3.46 0.3 0.02 0.0 0.0 100.0
New Orleans 2.8 35.54 25.2 22.96 6.9 3.88 97.28
Dallas 0.46 30.94 29.46 26.32 6.92 3.62 97.72
Seattle 0.52 28.6 38.08 30.78 0.58 0.48 99.04
NY Giants 0.0 0.78 1.46 3.58 25.82 18.54 50.18
Philadelphia 0.0 0.26 2.32 4.4 15.22 18.2 40.4
Carolina 0.0 0.18 1.68 7.9 12.48 15.04 37.28
Atlanta 0.0 0.24 1.32 2.18 23.36 17.06 44.16
San Francisco 0.0 0.0 0.16 1.64 1.16 4.42 7.38
Minnesota 0.0 0.0 0.0 0.0 6.12 12.22 18.34
St Louis 0.0 0.0 0.02 0.2 1.08 4.48 5.78
Washington 0.0 0.0 0.0 0.02 0.34 1.52 1.88
Green Bay 0.0 0.0 0.0 0.0 0.02 0.54 0.56
Detroit 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Tampa Bay 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Arizona 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Generated: Tue Dec 05 00:20:05 EST 2006

Comparison of DVOA and NFL-Forecast Playoff Predictions, After Week 12

Tuesday, November 28th, 2006

Here is this week’s comparison Mike Harris’ playoff predictions  based on DVOA power ratings and my playoff forecasts. The two forecasts are really starting to converge.

 Edited to correct data entry error.

                      Division Championship   Playoff Odds
Team              DVOA       NFL-F           DVOA       NFL-F
NewEngland    98.70%     88.64        99.30%     94.46
NYJets            1.00%       11.04       12.60%     35.8
Miami              0.10%       0.14          0.90%       2.74
Buffalo            0.00%       0.18          0.60%       4.82
                                                                         
                      Division Championship   Playoff Odds
Team              DVOA       NFL-F           DVOA       NFL-F
Indianapolis     99.80%     99.94        99.9%     99.98
Jacksonville     0.10%       0.06         26.60%    14.84
Houston           0.00%       0             0.00%     0.02
Tennessee       0.00%       0              0.00%      0.1
                                                                         
                      Division Championship   Playoff Odds
Team              DVOA       NFL-F           DVOA       NFL-F
Baltimore         99.30%     95.08       99.90%     98.1
Cincinnati        0.60%       4.9          27.20%     33.12
Pittsburgh        0.00%       0.02        0.10%       0.22
Cleveland        0.00%       0             0.00%       0
                                                                         
                      Division Championship   Playoff Odds
Team              DVOA       NFL-F           DVOA       NFL-F
SanDiego        92.90%     80.76        99.70%     98.32
Denver            4.40%       11.44       74.20%     70.56
KansasCity      2.60%       7.8           55.70%     46.92
Oakland           0.00%       0               0.00%       0
                                                                         
                      Division Championship   Playoff Odds
Team              DVOA       NFL-F           DVOA       NFL-F
NYGiants         19.80%     24.22       75.60%     64.38
Philadelphia     4.50%       3.24         40.70%     17.26
Dallas              75.60%     72.06       97.80%     91.4
Washington     0.00%       0.48           1.70%       5.48
                                                                         
                      Division Championship   Playoff Odds
Team              DVOA       NFL-F           DVOA       NFL-F
Chicago          100.00%   99.94        100.00%   99.98
GreenBay        0.00%       0                2.80%     1.18
Minnesota        0.00%       0.06          11.60%   18.04
Detroit             0.00%       0               0.00%       0
                                                                         
                      Division Championship               Playoff Odds    
Team              DVOA       NFL-F           DVOA       NFL-F
NewOrleans     75.10%     67.62       92.00%     89.06
Carolina          24.50%     30.66       57.90%     61.6
Atlanta             30.00%     1.72        2.30%     10.48 
TampaBay       0.00%       0              0.00%       0.04
                                                                         
                      Division Championship               Playoff Odds    
Team              DVOA       NFL-F           DVOA       NFL-F
Seattle             83.70%     79.98      88.00%     89.34.96
SanFrancisco  4.10%       14.72          6.00%     25.44
StLouis            12.10%     5.3          22.90%     26.32
Arizona           0.00%       0               0.00%       0.08

NFL playoff predictions 2006, after week 12

Tuesday, November 28th, 2006

Things crystalized quite a bit in week 12. We now have a very good idea of 5 of the playoff teams in each conference, although Denver and the Giants are still a little squishy, both in their computer generated playoff odds and in their on the field play. So in each conference we have one, maybe two, playoff spots up from grabs. Those last slots probably won’t be firmed up until the last week of the season. Plus the teams that are likely playoff qualifiers are still slugging it out for seeding positions that yield home field advantage and first round byes.

Edited to correct data entry error 

Percent probability of team finishing in each place within the division

AFC EAST

Team 1st 2nd 3rd 4th
New England 88.64 11.1 0.26 0.0
NY Jets 11.04 67.84 17.52 3.6
Buffalo 0.18 13.28 51.44 35.1
Miami 0.14 7.78 30.78 61.3

AFC NORTH

Team 1st 2nd 3rd 4th
Baltimore 95.08 4.92 0.0 0.0
Cincinnati 4.9 90.74 4.32 0.04
Pittsburgh 0.02 4.12 74.04 21.82
Cleveland 0.0 0.22 21.64 78.14

AFC SOUTH

Team 1st 2nd 3rd 4th
Indianapolis 99.94 0.06 0.0 0.0
Jacksonville 0.06 81.1 16.38 2.46
Tennessee 0.0 14.76 51.32 33.92
Houston 0.0 4.08 32.3 63.62

AFC WEST

Team 1st 2nd 3rd 4th
San Diego 80.76 14.82 4.42 0.0
Denver 11.44 45.82 42.74 0.0
Kansas City 7.8 39.36 52.84 0.0
Oakland 0.0 0.0 0.0 100.0

NFC EAST

Team 1st 2nd 3rd 4th
Dallas 72.06 22.9 4.4 0.64
NY Giants 24.22 58.34 14.4 3.04
Philadelphia 3.24 13.14 48.2 35.42
Washington 0.48 5.62 33.0 60.9

NFC NORTH

Team 1st 2nd 3rd 4th
Chicago 99.94 0.06 0.0 0.0
Minnesota 0.06 61.8 36.24 1.9
Green Bay 0.0 37.62 58.94 3.44
Detroit 0.0 0.52 4.82 94.66

NFC SOUTH

Team 1st 2nd 3rd 4th
New Orleans 67.62 30.42 1.96 0.0
Carolina 30.66 48.5 20.68 0.16
Atlanta 1.72 20.98 72.46 4.84
Tampa Bay 0.0 0.1 4.9 95.0

NFC WEST

Team 1st 2nd 3rd 4th
Seattle 79.98 16.28 3.74 0.0
St Louis 5.3 47.44 46.54 0.72
San Francisco 14.72 36.1 48.12 1.06
Arizona 0.0 0.18 1.6 98.22

Generated: Mon Dec 04 21:24:30 EST 2006

AFC Percent Probability Playoff Seeding

Team 1st 2nd 3rd 4th 5th 6th Total
Indianapolis 67.96 20.16 9.08 2.74 0.02 0.02 99.98
San Diego 16.48 28.86 27.78 7.64 13.58 3.98 98.32
Baltimore 13.04 34.74 31.04 16.26 1.5 1.52 98.1
New England 1.98 9.5 23.0 54.16 2.6 3.22 94.46
Denver 0.4 4.54 4.5 2.0 35.0 24.12 70.56
Kansas City 0.1 1.84 2.4 3.46 19.92 19.2 46.92
Cincinnati 0.04 0.3 0.94 3.62 12.3 15.92 33.12
NY Jets 0.0 0.06 1.2 9.78 8.1 16.66 35.8
Jacksonville 0.0 0.0 0.06 0.0 5.06 9.72 14.84
Buffalo 0.0 0.0 0.0 0.18 1.38 3.26 4.82
Miami 0.0 0.0 0.0 0.14 0.54 2.06 2.74
Pittsburgh 0.0 0.0 0.0 0.02 0.0 0.2 0.22
Tennessee 0.0 0.0 0.0 0.0 0.0 0.1 0.1
Houston 0.0 0.0 0.0 0.0 0.0 0.02 0.02
Cleveland 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Oakland 0.0 0.0 0.0 0.0 0.0 0.0 0.0

NFC Percent Probability Playoff Seeding

Team 1st 2nd 3rd 4th 5th 6th Total
Chicago 93.44 5.32 0.98 0.2 0.04 0.0 99.98
New Orleans 4.28 24.36 23.76 15.22 13.62 7.82 89.06
Dallas 1.26 36.38 19.68 14.74 12.74 6.6 91.4
Seattle 0.66 19.74 29.46 30.12 4.2 5.16 89.34
NY Giants 0.26 7.8 7.36 8.8 22.72 17.44 64.38
Carolina 0.04 4.64 11.04 14.94 18.04 12.9 61.6
San Francisco 0.0 0.64 4.9 9.18 3.62 7.1 25.44
Philadelphia 0.02 0.52 1.36 1.34 4.78 9.24 17.26
St Louis 0.02 0.42 1.02 3.84 7.28 13.74 26.32
Minnesota 0.02 0.0 0.02 0.02 7.16 10.82 18.04
Atlanta 0.0 0.14 0.32 1.26 3.94 4.82 10.48
Washington 0.0 0.04 0.1 0.34 1.68 3.32 5.48
Green Bay 0.0 0.0 0.0 0.0 0.18 1.0 1.18
Tampa Bay 0.0 0.0 0.0 0.0 0.0 0.04 0.04
Detroit 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Arizona 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Generated: Mon Dec 04 21:24:29 EST 2006

Comparison of DVOA and NFL-Forecast Playoff Predictions, after week 11

Tuesday, November 21st, 2006

I’m going to do these comparisons between Mike Harris’ playoff predictions  based on DVOA power ratings and my playoff forecasts for the remainder of the season. I think they will provide a basis for determining the predictive value of the DVOA based power ratings in comparison to power ratings based on wins and losses only.

 I’ve decided to use the playoff predictions based on the standard DVOA, rather than the Donovan McNabb adjusted version. Every team suffers injuries, and I don’t think you can consider an injury to one player in the whole league on an ad hoc basis. I also discovered that in week 10 I had mistakenly entered a Viking win over Green Bay, that was making my projections for the Vikings overly optimistic and to a lesser extent, penalizing the chances of the Packers. I fixed that in this weeks projections, but I haven’t gone back and fixed week 10 projections.

                         Division Championship   Playoff Odds
Team                DVOA       NFL-F          DVOA       NFL-F
NewEngland      97.80%      89.76           98.40%      94.18
NYJets              1.90%       10                8.40%       28.48
Miami               0.10%       0.14             0.40%       1.32
Buffalo              0.00%       0.1               0.30%       2.36
                                                                             
                        Division Championship   Playoff Odds
Team                DVOA       NFL-F          DVOA       NFL-F
Indianapolis       95.70%      99.34           99.50%      99.94
Jacksonville       4.20%       0.66             59.20%      32.16
Houston            0.00%       0                  0.00%       0.14
Tennessee        0.00%       0                  0.00%       0.02
                                                                             
                        Division Championship   Playoff Odds
Team                DVOA       NFL-F          DVOA       NFL-F
Baltimore          98.50%      96.64           99.10%      98
Cincinnati         1.10%       2.62             10.20%      18.94
Pittsburgh         0.30%       0.74             1.80%       1.76
Cleveland          0.00%       0                  0.00%       0.24
                                                                             
                        Division Championship   Playoff Odds
Team                DVOA       NFL-F          DVOA       NFL-F
SanDiego          87.00%      62.6             99.60%      96.82
Denver              11.80%      31.96           88.60%      90.72
KansasCity       1.10%       5.44             33.70%      34.92
Oakland            0.00%       0                  0.00%       0
                                                                             
                        Division Championship   Playoff Odds
Team                DVOA       NFL-F          DVOA       NFL-F
NYGiants          43.20%      62.2             89.40%      82.2
Philadelphia      8.00%       29.46           54.90%      27.92
Dallas               48.60%      8.34             93.60%      61.42
Washington       0.00%       0                  0.50%       0.54
                                                                             
                        Division Championship   Playoff Odds
Team                DVOA       NFL-F          DVOA       NFL-F
Chicago            100.00%    99.94           100.00%    99.98
GreenBay         0.00%       0.04             7.50%       2.72
Minnesota         0.00%       0.02             4.60%       14.9
Detroit              0.00%       0                  0.00%       0.02
                                                                             
                        Division Championship   Playoff Odds
Team                DVOA       NFL-F          DVOA       NFL-F
NewOrleans       51.50%      43.04           70.20%      75.84
Carolina            44.60%      43.84           62.50%      70.46
Atlanta              3.80%       13.12           7.60%       29.66
TampaBay        0.00%       0                  0.00%       0.42
                                                                             
                        Division Championship   Playoff Odds
Team                DVOA       NFL-F          DVOA       NFL-F
Seattle              69.60%      72.32           72.50%      82.16
SanFrancisco    11.00%      21.64           12.60%      33.42
StLouis             19.20%      6.04             23.40%      18.34
Arizona             0.00%       0                  0.00%       0

Comparison of Football Outsiders DVOA and NFL-Forecast Playoff Predictions after week 10, 2006

Wednesday, November 15th, 2006

Football Outsiders (or more precisely, FO reader Mike Harris) has started doing playoff prediction simulations similar to those I have been doing here, except based on the DVOA power ratings.  Football Outsiders uses extensive play-by-play data for their power ratings. I use just the opposite approach, using only the out come of each game (won-loss, not score) and where the game was played as the basis of my power ratings. I am a frequent reader of Football Outsiders and have a lot of respect for their work. I’ve considered adding DVOA power ratings as an alternate power ranking scheme to my software, and building a win prediction model from the DVOA ratings, but the job thing has kept me pretty busy this fall. So I’m glad Mr. Harris has taken that off my to do list.

I thought it would be worthwhile to publish a side by side comparion. It is very interesting how closely the predictions based on the two power ranking methods compare, although there are some very significant differences. The difference in prediction for the Denver-SD division race is particularly interesting. SD is clearly better based on DVOA; watching the two teams play it is easy to see why. Denver seldom gets any style points for the games they have won this season. Yet my power rating system is more impressed by Denver because they have beaten three team with winning records (New England, Baltimore, Kansas City) while San Diego hasn’t beaten any. And that is the thing about Denver, they consistently find a way to win. San Diego on the other hand found a way to lose to the Ravens, a game they probably should have won. To some extent, I think that wins and losses captures some of the intangibles that DVOA or other statistical measures aren’t able to quantify. I’m not arguing that W-L is a better predictor of future wins than DVOA; rather, I think the two approaches are complementary. Furthermore, I feel a little more comfortable with some of the DVOA-based NFC predictions. In particular, Minnesota is looking very shaky after their fast start and San Francisco’s high power rating this week is an anomaly due to the large variances in my power ratings. The SF bug is something that I know how to fix, but don’t have time to code this season. On the other hand, it will be interesting to see if Dallas and Philadelphia do as well in the playoff race as projected by DVOA.

Very cool, Mike Harris and Football Outsiders, and welcome to the playoff forecasting business.

More on tiebreakers and the schedule format

Wednesday, October 4th, 2006

A few months ago, Doug Drinen wrote a series of fascinating articles on his blog based on 10,000 simulations of the NFL season. One of the surprising findings of his study was that the strongest team in the NFL is likely to win the Superbowl only about 24% of the time, and even the worst team in the NFL could win the Superbowl by random chance.

His study inspired me to conduct a simulation study to determine the effectiveness of the NFL tiebreaking procedures for identifying the “better” of the two tied teams. Isn’t that what a tiebreaker should do? What I mean by “better” is the team with the higher power ranking, as used in my software. So I basically simulated the 2005 season 100,000 times, using the end-of-season power rankings. For the purposes of this study only, I am treating the power ranking as a true indicator (as opposed to approximate indicator) of each team’s relative strength. For each simulation, I noted whether each tiebreaker decision favored the team with the better or worse power rating. For simplicity, I only considered two team tiebreakers. My software breaks all ties (even between 3rd and 4th place teams in a division) so I only considered tiebreakers between teams with win-loss-tie percentages of 0.600 and better to emphasize the most important tiebreaking decisions. Listed below are the fraction of time each tiebreaker favored the better team:

Head to head: 0.48

Division: 0.505

Common Opponent: 0.485

Conference: 0.51

Strength of Victory: 0.45

Strength of Schedule: 0.47

These numbers are accurate to the number of significant figures shown, as they remain consistent with repeated simulations with different random seeds. Two points jump out: 1) For the most part tiebreakers are no better than a coin flip for determining which of two teams is better. 2) Most of the tiebreakers seem to favor the weaker teams by small, but statistically significant amounts. This could be an artifact of my 2005 power ratings and the 2004 finishes (which determines the 2005 schedule). I haven’t yet found an obvious explanation. I’ll do this analysis again at the end of 2006 and see if the trends are real.

If the NFL tiebreakers are not effective in identifying which team is intrinsically stronger, then what do they do? Obviously, they emphasize certain games as being more important that other games. Each team’s NFL schedule breaks down as follows:

6 division games (home-and-away against the three division foes)

4 conference common-division games (division vs. division within the conference)

4 non-conference games (division vs. division across conferences)

2 additional conference games (determined by division place finish from previous season)

Everyone knows that the division games are most important, followed by the conference games, followed by the non-conference games, right? Not so fast. Those non-conference games turn out to be pretty important in determining the division championship because every team in your division plays them, and the “record against common opponents” tiebreaker is considered before the “record within the conference” tiebreaker. To determine exactly how important each type of game is in 1) winning the division and 2) making the playoffs, I did another simulation study. Here, I set the power rankings of each team equal and eliminated home field advantage. This made every game a 50-50 chance for either team to win. I then pre-set the outcome of one division game (e.g., Chiefs beat the Broncos) and then simulated the whole NFL season 50,000 times leaving the outcome of every other game to chance. I recorded how much of an advantage winning the division game made. I then repeated the exercise with each of the other three types of games described above. Below, I’ve reported how much winning each type of game helps in terms of (winning the division) and [making the playoffs], normalized to a baseline of winning a non-conference game:

Division game: (1.23) [1.07]

Conference Common-Opponent game (1.00) [1.03]

Additional Conference game (0.98) [1.03]

Non-Conference game (1.00) [1.00]

This analysis indicates that the division games are most important both for winning the division and making the playoffs. After those, there is very little difference in the importance of the remaining games. In fact, if winning the division is your primary goal, the non-conference games are slightly more important than the two games that are determined by your placing from the pervious year.

A couple caveats should be mentioned. By making all the teams equal strength, we are going to get more ties than a typical NFL season, so the differences of the various types of games are exaggerated here. So in reality, the small differences between any non-division games are actually even smaller. Secondly, this study ignores the correlation of a team’s strength from one year to another. For example in 2006 the Chiefs play Jacksonville and Miami because all three teams finished second in their division in 2005. Let’s say all the teams in the AFC finish in exactly the same place as in 2006. All of those second place teams are competing for wild card places and those head-to-head wins could come in handy. However, even though there is some correlation in a team’s strength from year-to-year, there is usually enough shake up in the standings that this shouldn’t introduce too much error.

One think’s for sure, there are not many other places you can get this kind of information.

Doom, Gloom, and NFL Tiebreakers

Thursday, September 21st, 2006

One of the enduring traditions of the NFL is the week 2 ritual in which the media, with a tone containing some mixture of grave solemness and outright glee, pronounce dead, or at least mortally wounded, the playoff chances of various teams that start the season at 0-2. The litany of the 2006 ritual was the frequently mentioned statistic that teams that start the season at 0-2 have qualified for the playoffs only 13% of the time. My playoff forecasting software validates the historical lesson; it predicts that the average chance of making the playoffs for the eleven 0-2 teams is 15%. But not all 0-2 teams are equally doomed. Several 0-2 teams still have decent chances of making the playoffs including 2005 NFC playoff qualifiers Carolina, Washington, and Tampa Bay and AFC contenders Kansas City and Miami. You can look at the week 2 playoff forecasts in the preceeding post to get the specific odds of each team.

 I’m going to devote the rest of this column to the exciting topic of NFL tiebreaker rules. Most fans who have been in a tight playoff race in the last few years are aware of these and may even have a vague notion of the order of the tiebreakers committed to memory. However, I’ve been hanging around fan message boards long enough to see that many devoted and informed fans don’t fully understand them. They come into play more often than one would think and in some cases, you have to go up to six levels deep into the tiebreakers to resolve the tie. I’m going to focus on the aspects of the tiebreaking rules that most often cause confusion. One area that always causes confusion is 3-way ties. Let’s consider a couple hypothetical examples. Example A: The following 3 teams are tied at 10-6 and vying for two open wild card spots:

Denver 10-6 San Diego 10-6Pittsburgh 10-6 To break this 3-way tie, you must first break the tie between Denver and San Diego, since they are in the same division. Always break ties within a division before considering ties outside the division. Let’s assume San Diego wins that tiebreaker. They would then face the Steelers in a 2-way tiebreaker. Now if San Diego wins that tiebreaker, they would become the number 5 seed. Then Pittsburgh and Denver would have to go through a tiebreaking procedure to determine which one would claim the final playoff seed.

Example B: The following teams are tied at 10-6 for and vying for a single wild card spot:

Jacksonville 10-6

Miami 10-6  

 If Miami defeated Jacksonville in a head-to-head match up earlier in the season, do they have an iron-clad tiebreaker advantage over Jacksonville? Not necessarily. If another team is tied with them, say San Diego, and neither team played San Diego, then the head-to-head tie breaker is not applicable. The other tiebreaker that causes confusion is the strength of victory tiebreaker. Some fans mistakenly think that this is the point spread by which you won your games. Instead, it is the WLT percentage of the opponents that you have defeated. It is like strength of schedule, except it only considers the teams you defeated, not the teams you lost to.

My recommendation when it comes to tiebreakers is to use my software to analyze them. That’s how I do it, because it is faster and much less prone to mistakes than figuring them manually. Let’s say you were interested in who would win the Division between the Cowboys and the Redskins and there are 4 weeks left in the NFL season, including one game in which they play each other. You have a particular scenario in which both the Redskins and the Cowboys end up having the same record. Who would make the playoffs? You can analyze this scenario by selecting the winners of each of the seven games of interest by moving the sliders in the “Advanced Analysis” tab all the way toward each of the projected winners. Then click “Forecast Remainder of Season”. The software will determine the winner of the tiebreaker for that scenario. If the odds of one of the teams winning is less than 100%, that means that the tiebreaker is at the strength-of-victory or strength-of-schedule level and the final resolution depends on some unplayed games not involving these two teams. Next week, I’ll tell you a secret about the tiebreakers (they don’t work) and the surprising implications regarding the relative importance of various games on each team’s schedule.

 

A week 1 loss isn’t that bad, is it?

Friday, September 15th, 2006

So week 1 of the 2006 NFL season is but a distant memory. Most fans have shifted their attention to week 2. Most fans don’t include Chiefs fans (like me) who are still wondering when the NFL changed the rule to make it okay to lower your shoulder into a sliding QB and get up and celebrate afterwards. Before we leave week 1, let’s look at the playoff implications of the outcome of week 1 games. They are larger than you might think.

First, consider that 9-7 probably leaves you sitting at home. So a week one loss means that you are one-seventh of the way to being eliminated from the playoffs. The average chance of making the playoffs for each team at the beginning of the season is 6/16 or 37.5% . If you are one-seventh of the way to driving those odds to zero, you can figure that a week one loss dropped your team’s odds of making the playoffs by 5%. That is a pretty significant drop in odds, but for most teams, the loss was even more costly than that, because in the NFL, nothing is average.

In the table below, I’ve calculated the difference in week 1 and preseason playoff odds for all 16 losing teams. In the first column, I’ve listed their drop in odds in absolute percentage points. In the second column, I’ve listed the percentage of their initial chances that have been “lost”. For example, I predicted in preseason that the Titan’s odds of making the playoffs were 6.88%; after their week 1 loss to the Jets, they have dropped to 1.98%. So in absolute percentage points, their odds have dropped by 4.9%. The have also lost (6.88-1.98)/6.88 = 71% of their initial chances of making the playoffs by virtue of their week one loss. Their season is practically over.

Carolina          -15.9    -24.2
Washington      -11.4    -23.3
Kansas City     -11.2    -22.2
Denver             -8.9      -14.3
Tampa Bay      -8.8      -26.1
Detroit             -8.4      -42.4
Miami              -8.2      -16.7
Green Bay       -7.9      -43.2
Cleveland        -7.7      -44.4
NYG                -6.9      -17.3
Buffalo            -5.8      -41.9
Tennessee        -4.9      -71.2
Dallas              -4.9      -9.5
Oakland           -4.8      -43.0
Houston           -3.2      -40.8
San Francisco  -3.1      -48.4

How could one loss cause so much carnage? Let’s look take the Titans again as an example, since it is pretty unlikely that we will get too many more chances to talk about them again this season. According to the Vegas odds makers, the “average” Titans’ season was projected at 5 wins. In order to make the playoffs, they would need a season much better than average, and even then, the best chance of making the playoffs was to sneak in at 9-7. Now also consider that the Titans were favored on Sunday. My software predicted that they had a 59% chance of beating the Jets at home. By losing, the average predicted wins of the season drops from around 5 to 4.4, and most of those 9-7 seasons in which they barely qualified are now 8-8 seasons with them sitting at home. Add in the fact that the Colts and Jags won their games, and that another wild card contender won (either the Steelers or the Bengals) and it’s not difficult to see how the Titans are on the verge of mathematical elimination after week 1.

For every loser, there is a winner. Here is the same list for the winners.

Atlanta             16.4     21.3
Minnesota        12.4     10.6
Chicago           11.2     21.3
San Diego        11.0     33.7
Cincinnati        10.3     12.3
Philadelphia    7.8       22.2
New England   6.7       9.7
Indianapolis    6.3       8.2
Seattle             6.3       7.7
Pittsburgh        6.2       9.7
Baltimore        5.9       20.6
Jacksonville    5.6       22.3
New Orleans   5.3       31.4
St. Louis          4.5       18.3
Arizona           3.3       34.7
NYJ                 2.6       44.7

Some caveats are in order. It is absolutely ridiculous to be doing playoff predictions based on the outcome of one game. This is the first year that I’ve done predictions this early; I normally wait until four to six games have been played to get a better idea of how good the teams really are. I’m also well aware that football is played on a field, not a computer, so the Titans and every other NFL team still control their own destiny. However, this little study emphasizes one of the key attractions of the NFL:  every game matters. And if every game matters, then the entire season could be hinging on any given snap of the ball. Just ask a Chiefs fan.