footballcommentary.com
A model-based approach to football strategy.
| August 2, 2006 |
| Contents |
| Introduction |
| Example: Atlanta at Tampa Bay |
| Evaluation of Individual Scenarios |
| The Simulation |
| Playoff Clinching and Elimination |
| The Tiebreaking Algorithm |
| Appendix |
In this article we present a computer program that applies the NFL tiebreaking rules to determine the playoff teams and seedings in each conference, given actual or hypothetical outcomes for all the games in the regular season.1 This determination, often tedious when done by hand, takes a computer only a fraction of a second. We exploit the program by embedding it in a simulation. The simulation produces an estimate of each team's probability of making the playoffs or of winning the Super Bowl, given probabilities for the outcomes of the remaining regular-season games.
One possible application of the simulation is to help bettors estimate the proper odds for each team to win the Super Bowl, at least late in the season when the tiebreaking rules can have an important effect on teams' prospects.
A second application is finding the value of a tie, relative to a win or loss. This information can be critical on those rare occasions when a regular-season game goes deep into overtime. For example, with 1:08 left in the overtime period of the 2005 Week 16 game between Atlanta and Tampa Bay, the Falcons faced 4th-and-2 at their own 24-yard line, and had to choose between punting and going for the first down. Atlanta's decision depends in part on how a tie affects their playoff prospects, but that information was unavailable to coach Jim Mora, who ultimately elected to punt. Armed with our simulation, the Falcons could have run it at the start of overtime to get the information they needed. We will do the analysis in the next section, as an example. Interestingly, the more refined analysis facilitated by the computer program will force us to alter some of the conclusions of our original analysis of Mora's decision.
Following the example, we explain how to run the computer program. NFL teams should feel free to use it. Others may also, provided that published results derived from the program include a link to this page. For those who want some understanding of the tiebreaking algorithm, but don't want to study the source code, we explain the algorithm in the final section.
As an application of the simulation, we will examine the game between Atlanta and Tampa Bay which we discussed in the introduction. In our original analysis we used Atlanta's probability of making the playoffs as the criterion, and concluded that for the Falcons, a tie had at most 7.7% of the value of a win.2 Because of the complexity of the playoff scenarios, we gave only an informal argument for the value of a tie to Tampa Bay; but with the computer program we can now state that, using the probability of making the playoffs as the criterion, a tie had 37% of the value of a win.
However, probability of making the playoffs is a poor criterion, because it doesn't distinguish among the different seedings. Under the NFL's playoff format, higher seeds have an advantage: division winners get a home playoff game, the second seed gets a bye followed by a home game, and the top seed gets a bye and home-field advantage throughout the conference playoffs. Here, we will use as the criterion the probability of winning the Super Bowl.
| Table 1 | |||
| Estimated probabilities of winning the Super Bowl | |||
| Atlanta Loses | Tie Game | Atlanta Wins | |
| Atlanta | 0 | 0.00045 | 0.00805 |
| Tampa Bay | 0.0678 | 0.0416 | 0.0381 |
The computer program makes the calculations easy. Using the file of game-outcome probabilities described
later,
we run the simulation three times, corresponding to the cases in which Atlanta loses, ties, or wins. The results are shown in Table 1. Since 0.00045 is 5.6% of the way from 0 to 0.00805, a tie has just 5.6% of the value of a win for
Atlanta.3
This is even smaller than the value of a tie in our original analysis, which implicitly treated all playoff slots as equal. The reason is that with a tie against Tampa Bay, Atlanta can aspire only to the #6 seed, whereas with a win over Tampa Bay it's possible for the Falcons to win their division. Still, recognizing the disparities among the playoff seedings only reinforces the conclusion that the Falcons have to play to win.
Changing the criterion from the probability of making the playoffs to the probability of winning the Super Bowl has a more interesting effect on Tampa Bay's situation. Since 0.0416 is 11.8% of the way from 0.0381 to 0.0678, a tie has just 11.8% of the value of a win for Tampa Bay, rather than the 37% we obtained by regarding all playoff seedings as equal. So, contrary to an assertion we made in our original analysis, a tie isn't very helpful to the Buccaneers either.
| Figure 1 |
| Distribution for TB Playoff Seeding |
| [Larger Image] |
|
|
The reason can be seen in Figure 1, which shows the distribution for Tampa Bay's playoff seeding depending on whether they lose (blue bar graph), tie (green), or win (red) against Atlanta. The symbol "X" indicates missing the playoffs altogether. Using the probability of making the playoffs as the criterion is equivalent to comparing the heights of the three bar graphs at X, and ignoring all the other information. (Indeed, the heights of the three bar graphs at X are 0.145, 0.092, and 0.003. Our conclusion about the value of a tie when we use the probability of making the playoffs as the criterion follows from the observation that 0.092 is 37% of the way from 0.145 to 0.003.) From the full bar graphs, we see that a win over Atlanta gives Tampa Bay a much better chance to be a division winner (seed #4 or better) rather than a wild card (seed #5 or #6), and even gives the Buccaneers a realistic chance to capture the #2 seed and a first-round bye.4
Since a tie has little value to Tampa Bay, it's not immediately clear that Atlanta should go for the first down. If Atlanta punts, then with time running short and little to gain from a tie, the Buccaneers might go for broke themselves, creating the possibility that Tampa Bay could turn the ball over on downs in their own end. In our original analysis we calculated that if the Falcons go for it on 4th down, they have about a 0.12 probability of winning the game. If instead the Falcons punt, there is perhaps a 0.15 probability of a Tampa Bay four-and-out, following which Atlanta would have perhaps a 0.5 probability of winning the game. So Atlanta's win probability if they punt is roughly 0.075. Punting also raises the probability of a tie by about 0.2. However, since a tie has only about 5% of the value of a win for Atlanta, this is equivalent to only an additional 0.01 of win probability, bringing it to 0.085. So, if both coaches have full knowledge of the implications of a tie, it still seems that the Falcons should go for it, although the decision is closer than we first suggested. (We're confident, though, that neither coach had full knowledge of the implications of a tie.)
| Table 2 |
| Index of Teams |
1 Buf 5 Balt 9 Hou 13 Den 2 Mia 6 Cin 10 Ind 14 KC 3 NE 7 Cle 11 Jax 15 Oak 4 Jets 8 Pitt 12 TN 16 SD 17 Dal 21 Chi 25 Atl 29 Ari 18 NYG 22 Det 26 Car 30 StL 19 Phil 23 GB 27 NO 31 SF 20 Wash 24 MN 28 TB 32 Sea |
The program for finding the playoff teams and seedings, given actual or hypothetical results for the regular season, is playoffteams.m. It calls the additional programs setup.m, standings.m, divtiebreaker.m, wctiebreaker.m, getwcteams.m, seeddivwinners.m, and getsbwinprobs.m, which are also used for the simulation described in the next section. The source code is written for MATLAB®.
The required inputs are contained in a data file named gameoutcomes.txt, with one row for every game in the regular season. Each row has three entries. The first two entries are the indices (from Table 2) of the teams in that game. The third entry in that row equals 1 if the team listed first won the game, 0 if the team listed first lost the game, and
place =
3 6 10 13 18 21 28 32
2 8 11 14 20 24 26 30
1 5 12 16 17 22 25 29
4 7 9 15 19 23 27 31
seeds =
10 32
13 21
6 28
3 18
11 26
8 20
in which the entries are the indices of the teams, from Table 2. Naturally, this output matches the actual 2005 standings and playoff seedings.
The source code for the simulation is
schedsim.m.
The required inputs are contained in a data file gameprobs.txt which, analogous to the file gameoutcomes.txt described in the previous section, has one row for every game in the regular season. Each row has three entries. The first two entries are the indices (from Table 2) of the teams in that game. If the game has been completed, the third entry in that row equals 1 if the team listed first won the game, 0 if the team listed first lost the game, and
28 25 -1
1 6 1
26 17 0
27 22 0
9 11 0
18 20 0
7 8 0
14 16 1
30 31 0
2 12 1
29 19 0.5
32 10 0.8
13 15 0.85
21 23 0.7
24 5 0.4
3 4 0.65
The second row in this excerpt says that the game between Buffalo and Cincinnati was won by Buffalo. (The row could have been entered equivalently as
| Table 3 |
| Probabilities of winning the SB if ATL-TB ends in a tie |
1 0 2 0 3 0.05345856 4 0 5 0 6 0.05943744 7 0 8 0.03164800 9 0 10 0.18000000 11 0.03456000 12 0 13 0.14054400 14 0.00035200 15 0 16 0 17 0.01761280 18 0.06063232 19 0 20 0.02974592 21 0.12983408 22 0 23 0 24 0.00461312 25 0.00044800 26 0.03707200 27 0 28 0.04158976 29 0 30 0 31 0 32 0.17845200 |
When the simulation
schedsim.m
is run with the data set
gameprobs.txt,
it produces the output shown in Table 3. The first column of Table 3 lists the teams according to the key shown in Table 2. The second column gives each team's estimated probability of winning the Super Bowl, if the Atlanta-Tampa Bay game ends in a tie. The probabilities sum to 1. To find the teams' playoff prospects if Tampa Bay wins, we change the
The most natural way to generate a scenario is to generate outcomes for the remaining regular-season games using the specified probabilities,5 determine the playoff seedings in each conference, and then continue the simulation through the playoffs to arrive at a Super Bowl winner. By generating many scenarios, and recording the fraction of the scenarios in which each team wins the Super Bowl, one obtains an estimate of each team's probability of winning the Super Bowl. However, we have chosen a different approach with less sampling error: conditional on the playoff seedings, we directly compute the teams' probabilities of winning the Super Bowl (in getsbwinprobs.m), rather than include the playoffs as part of the simulation. The result of each scenario is therefore not a Super Bowl winner, but rather each team's conditional probability of winning the Super Bowl, conditional on the results of the regular season. The average of these conditional probabilities, averaged over the generated scenarios, is the estimate of each team's probability of winning the Super Bowl.
| Table 4 | |
| Super Bowl win Probabilities by seed under the default assumptions | |
| seed | probability |
| #1 | 0.1800 |
| #2 | 0.1440 |
| #3 | 0.0576 |
| #4 | 0.0518 |
| #5 | 0.0346 |
| #6 | 0.0320 |
For simplicity, the default assumption is a constant win probability for the home team in a playoff game. To generate Table 3, we assumed that a home playoff team has win probability 0.6, and that on a neutral field in the Super Bowl, each team has win probability 0.5. This assumption already generates substantial variation in the different seeds' probabilities of winning the Super Bowl, as shown in Table 4. However, as we explain in the appendix, the program gives users the option of specifying individual win probabilities for some or all possible playoff pairings. This option is important if the goal is to estimate the teams' odds of winning the Super Bowl, for betting purposes. However, it's overkill if the goal is merely to get a useful estimate the value of a tie.
The simulation is currently set to generate 1000 scenarios. This takes about a minute, and produces results that are as accurate as necessary.
One can use the simulation to estimate the teams' probabilities of making the playoffs, rather than their probabilities of winning the Super Bowl. To do this, one simply changes the variable seedweighttype in schedsim.m from 2 to 1.
It would be easy to create a version of the computer program with which one could, in principle, determine which teams have clinched a playoff slot, and which teams have been eliminated: For each possible scenario for the remaining regular-season games, one calculates the playoff teams. A team that makes the playoffs in every scenario has clinched, whereas a team that doesn't make the playoffs in any scenario has been eliminated.
This approach works only in principle because if n games remain, there are 2n possible scenarios. With even two full weeks (32 games) left in the season, the number of scenarios is
We recommend simply running the simulation with, say, ten thousand trials. A team that makes the playoffs in every one of those trials has clinched a playoff slot for all practical purposes, if not necessarily literally. Similarly, a team that doesn't make the playoffs in any of those trials has (for all practical purposes) been eliminated.
The program for determining playoff seedings using the NFL tiebreaking procedures starts by computing (in setup.m) each team's overall won-lost-tied (W-L-T) percentage, W-L-T percentage in its division, W-L-T percentage in its conference, strength of victory, and strength of schedule.
To keep the algorithm simple, we treat overall W-L-T percentage as an honorary "tiebreaker." With that convention, the following is the sequence of tiebreakers (applied in divtiebreaker.m) for determining the standings within a division:
0. Overall won-lost-tied percentage.
1. Head-to-head (best won-lost-tied percentage in games among the teams).
2. Won-lost-tied percentage in the division.
3. Won-lost-tied percentage versus common opponents.
4. Won-lost-tied percentage in the conference.
5. Strength of victory.
6. Strength of schedule.
The following is the sequence of tiebreakers (applied in wctiebreaker.m) for finding the wild-card teams in a conference, or seeding the division winners:
0. Overall won-lost-tied percentage.
1. Head-to-head sweep. (Applicable only if one team has defeated each of the others or if one team has lost to each of the others).
2. Won-lost-tied percentage in the conference.
3. Won-lost-tied percentage in games against common opponents (minimum of four).
4. Strength of victory.
5. Strength of schedule.
In either sequence, if only two teams are involved, tiebreaker 1 reduces to ordinary head-to-head.
The NFL tiebreaking procedures include several additional tiebreakers having to do with points scored and allowed. These are not included in the computer program. In any application of the tiebreaking procedures to a group of teams, if the teams remain tied for all tiebreakers through strength of schedule, the program arbitrarily eliminates one of the teams.
Two subtleties of the tiebreaking procedures occasionally cause confusion. First, if a specific tiebreaker eliminates one or more teams, but two or more teams remain, then the remaining teams revert to step 0 of the relevant sequence of tiebreakers. Second, only one team from each division (the division's highest-ranking team that is not already in the playoffs) can compete for a particular wild-card slot.6
The first step in the computer program is to determine the standings in each division. The full ordering is required because, in principle, even the third-place finisher in a division can make the playoffs. To illustrate the steps, performed by standings.m, we will explain in detail how the program determines the final standings in the AFC East for the 2002 season. Because the program follows simple rules that work in all cases, it often performs steps that a person would recognize as unnecessary.
| Team | W | L | T | PCT | AFC | DIV |
| Buffalo | 8 | 8 | 0 | .500 | 5-7 | 2-4 |
| Miami | 9 | 7 | 0 | .562 | 7-5 | 2-4 |
| New England | 9 | 7 | 0 | .562 | 6-6 | 4-2 |
| N.Y. Jets | 9 | 7 | 0 | .562 | 6-6 | 4-2 |
The program first finds the first-place team. From the overall W-L-T percentages (tiebreaker 0) Buffalo is eliminated, leaving Miami, New England, and the Jets. Those three teams have identical overall W-L-T percentages (tiebreaker 0) and identical 2-2 records in games among the teams (tiebreaker 1). However, Miami is eliminated based on W-L-T percentage in the division (tiebreaker 2), leaving New England and the Jets. Those two teams have identical overall W-L-T percentages (tiebreaker 0), identical 1-1 records in games against each other (tiebreaker 1), and identical W-L-T percentages in the division (tiebreaker 2). However, the Jets have a better record versus common opponents (tiebreaker 3). So, the Jets win the division.
The program next determines the second-place finisher. The contenders are Buffalo, Miami, and New England. From the overall W-L-T percentages (tiebreaker 0) Buffalo is eliminated, leaving Miami and New England. Those two teams have identical overall W-L-T percentages (tiebreaker 0) and identical 1-1 records in games against each other (tiebreaker 1). However, Miami is eliminated based on W-L-T percentage in the division (tiebreaker 2), making New England the second-place finisher.
The contenders for third place are Buffalo and Miami. Comparing their overall W-L-T percentages (tiebreaker 0), Buffalo is eliminated, giving Miami third place. Buffalo finishes fourth.
Once the program has determined the final standings in each division, the next step is to determine the playoff seedings for the four division-winners in each conference. This is done by regarding those four teams as comprising a pseudo-division. The program seeddivwinners.m computes the "standings" in that pseudo-division, using wild-card tiebreakers rather than divisional tiebreakers. The ordering so derived gives the #1 through #4 seeds for the conference playoffs.
The final step is to find the wild-card teams. Within a conference, the contenders for the #5 seed are the second-place teams from the four divisions. We regard those four teams as forming a pseudo-division, and (in getwcteams.m, using wild-card tiebreakers) find the pseudo-division winner. That team is the #5 seed in the conference.
The four contenders for the #6 seed are the third-place finisher from the #5-seed's division, and the second-place teams from the other three divisions. Regarding these four contenders as forming a pseudo-division, and using wild-card tiebreakers, we find the pseudo-division winner. That team is the #6 seed in the conference.
As an illustration, we will list the steps that the program performs to find the 2004 NFC wild-card teams. As shown by the standings, the four second-place finishers are the Giants, Minnesota, New Orleans, and St. Louis. The Giants are eliminated based on overall W-L-T percentage (tiebreaker 0), leaving Minnesota, New Orleans, and St. Louis. In the only games among these three teams during the regular season, Minnesota defeated New Orleans, and New Orleans beat St. Louis. Therefore, none of the teams beat both of the others, nor did any of the teams lose to both of the others (tiebreaker 1). But among the three teams, St. Louis has the best W-L-T percentage in the conference (tiebreaker 2), and so gets the #5 seed.
The contenders for the #6 seed are the Giants, Minnesota, New Orleans, and Arizona (the third-place finisher in the NFC West). The Giants and Arizona are eliminated based on overall W-L-T percentage (tiebreaker 0). Then, since Minnesota defeated New Orleans (tiebreaker 1), Minnesota claims the #6 seed.
The simulation schedsim.m is currently set to assume that in a playoff game, the home team wins with probability 0.6, and that in the Super Bowl, each team's win probability is 0.5. Users can of course change the 0.6 to some other value while retaining the assumption that the win probability for the home team in a playoff game is the same regardless of the specific teams involved. However, the program also gives users the option of specifying individual win probabilities for some or all possible playoff pairings. To do this one must change the variable seedweighttype from 2 to 3, and create an input file called weightoverrides.txt. That file has a row for each playoff pairing for which the user wants to override the default probabilities. Each row must have five numbers. For example, the row
10 14 0.7 0.5 0.6
says that if Indianapolis (team 10) hosts Kansas City (team 14) in the playoffs, Indianapolis's win probability is 0.7; if the game is in Kansas City, Indianapolis's win probability is 0.5; and if the game is on a neutral field (which can't happen in this case because the teams are from the same conference), Indianapolis's win probability is 0.6. To override the default assumptions completely would require a row for every pair of teams that could possibly meet in the playoffs. At least until late in the regular season, when most teams have been eliminated from playoff contention, the file would be very large. We feel that the assumption of a constant home-team playoff win probability is adequate for many purposes.
FOOTNOTES
1 This computer program was inspired by communication with King Kaufman, sports columnist for Salon.
2 The actual relative value of a tie is slightly smaller, because our original analysis omitted two scenarios in which Atlanta makes the playoffs following a win against Tampa Bay: Atlanta secures a playoff berth if Washington loses, Dallas wins, Minnesota wins twice, and Chicago wins in Week 16; or if Dallas loses, Washington wins, Minnesota wins twice, the Giants win, and Chicago wins in Week 16.
3 The relative value of a tie, determined this way, is the value v that we explained in our article on overtime.
4 Remarkably, it's possible for Tampa Bay to earn a bye even after a loss to Atlanta. If Tampa Bay, Carolina, Minnesota, Chicago, Washington, and the Giants all finish 10-6, the NFC division winners are Washington, Minnesota, Tampa Bay, and Seattle. Tampa Bay then gets the #2 seed behind Seattle because they beat both Washington and Minnesota during the regular season. However, the probability of this scenario is less than 0.005.
5 In the simulation, the outcomes of the unplayed games are treated as independent. This is not strictly correct. For one thing, game outcomes affect perceptions of how good the teams are, and therefore affect our subjective probabilities for the outcomes of subsequent games. In addition, a team's motivation to win a later game can be influenced by the results of an earlier game, either because they become locked into a playoff seeding (for example Seattle in 2005 Week 17) or actually prefer a lower seeding (as was apparently the case for New England in 2005 Week 17). The simulation could be modified to incorporate these types of dependencies, but we doubt it would be worth the trouble.
6 This rule is sometimes critical. For example, in 2002, if all of the AFC's 9-7 teams had been eligible for the final wild-card slot, Miami would have made the playoffs. But since the Dolphins finished behind the Jets and Patriots in the AFC East, and the Patriots were not the #5 seed, the Dolphins were shut out.
Copyright © 2006 by William S. Krasker