footballcommentary.com

A model-based approach to football strategy.

August 2, 2006

[Home]

A Computer Program For Playoff Tiebreakers

Contents
Introduction
Example: Atlanta at Tampa Bay
Evaluation of Individual Scenarios
The Simulation
Playoff Clinching and Elimination
The Tiebreaking Algorithm
Appendix

In this article we present a computer program that applies the NFL tiebreaking rules to determine the playoff teams and seedings in each conference, given actual or hypothetical outcomes for all the games in the regular season.1 This determination, often tedious when done by hand, takes a computer only a fraction of a second. We exploit the program by embedding it in a simulation. The simulation produces an estimate of each team's probability of making the playoffs or of winning the Super Bowl, given probabilities for the outcomes of the remaining regular-season games.

One possible application of the simulation is to help bettors estimate the proper odds for each team to win the Super Bowl, at least late in the season when the tiebreaking rules can have an important effect on teams' prospects.

A second application is finding the value of a tie, relative to a win or loss. This information can be critical on those rare occasions when a regular-season game goes deep into overtime. For example, with 1:08 left in the overtime period of the 2005 Week 16 game between Atlanta and Tampa Bay, the Falcons faced 4th-and-2 at their own 24-yard line, and had to choose between punting and going for the first down. Atlanta's decision depends in part on how a tie affects their playoff prospects, but that information was unavailable to coach Jim Mora, who ultimately elected to punt. Armed with our simulation, the Falcons could have run it at the start of overtime to get the information they needed. We will do the analysis in the next section, as an example. Interestingly, the more refined analysis facilitated by the computer program will force us to alter some of the conclusions of our original analysis of Mora's decision.

Following the example, we explain how to run the computer program. NFL teams should feel free to use it. Others may also, provided that published results derived from the program include a link to this page. For those who want some understanding of the tiebreaking algorithm, but don't want to study the source code, we explain the algorithm in the final section.

Example: Atlanta at Tampa Bay

As an application of the simulation, we will examine the game between Atlanta and Tampa Bay which we discussed in the introduction. In our original analysis we used Atlanta's probability of making the playoffs as the criterion, and concluded that for the Falcons, a tie had at most 7.7% of the value of a win.2 Because of the complexity of the playoff scenarios, we gave only an informal argument for the value of a tie to Tampa Bay; but with the computer program we can now state that, using the probability of making the playoffs as the criterion, a tie had 37% of the value of a win.

However, probability of making the playoffs is a poor criterion, because it doesn't distinguish among the different seedings. Under the NFL's playoff format, higher seeds have an advantage: division winners get a home playoff game, the second seed gets a bye followed by a home game, and the top seed gets a bye and home-field advantage throughout the conference playoffs. Here, we will use as the criterion the probability of winning the Super Bowl.

Table 1
Estimated probabilities of
winning the Super Bowl
  Atlanta
Loses
Tie
Game
Atlanta
Wins
Atlanta 0 0.00045 0.00805
Tampa Bay 0.0678 0.0416 0.0381

The computer program makes the calculations easy. Using the file of game-outcome probabilities described later, we run the simulation three times, corresponding to the cases in which Atlanta loses, ties, or wins. The results are shown in Table 1. Since 0.00045 is 5.6% of the way from 0 to 0.00805, a tie has just 5.6% of the value of a win for Atlanta.3 This is even smaller than the value of a tie in our original analysis, which implicitly treated all playoff slots as equal. The reason is that with a tie against Tampa Bay, Atlanta can aspire only to the #6 seed, whereas with a win over Tampa Bay it's possible for the Falcons to win their division. Still, recognizing the disparities among the playoff seedings only reinforces the conclusion that the Falcons have to play to win.

Changing the criterion from the probability of making the playoffs to the probability of winning the Super Bowl has a more interesting effect on Tampa Bay's situation. Since 0.0416 is 11.8% of the way from 0.0381 to 0.0678, a tie has just 11.8% of the value of a win for Tampa Bay, rather than the 37% we obtained by regarding all playoff seedings as equal. So, contrary to an assertion we made in our original analysis, a tie isn't very helpful to the Buccaneers either.

Figure 1
Distribution for TB Playoff Seeding
[Larger Image]
Bar Graph

The reason can be seen in Figure 1, which shows the distribution for Tampa Bay's playoff seeding depending on whether they lose (blue bar graph), tie (green), or win (red) against Atlanta. The symbol "X" indicates missing the playoffs altogether. Using the probability of making the playoffs as the criterion is equivalent to comparing the heights of the three bar graphs at X, and ignoring all the other information. (Indeed, the heights of the three bar graphs at X are 0.145, 0.092, and 0.003. Our conclusion about the value of a tie when we use the probability of making the playoffs as the criterion follows from the observation that 0.092 is 37% of the way from 0.145 to 0.003.) From the full bar graphs, we see that a win over Atlanta gives Tampa Bay a much better chance to be a division winner (seed #4 or better) rather than a wild card (seed #5 or #6), and even gives the Buccaneers a realistic chance to capture the #2 seed and a first-round bye.4

Since a tie has little value to Tampa Bay, it's not immediately clear that Atlanta should go for the first down. If Atlanta punts, then with time running short and little to gain from a tie, the Buccaneers might go for broke themselves, creating the possibility that Tampa Bay could turn the ball over on downs in their own end. In our original analysis we calculated that if the Falcons go for it on 4th down, they have about a 0.12 probability of winning the game. If instead the Falcons punt, there is perhaps a 0.15 probability of a Tampa Bay four-and-out, following which Atlanta would have perhaps a 0.5 probability of winning the game. So Atlanta's win probability if they punt is roughly 0.075. Punting also raises the probability of a tie by about 0.2. However, since a tie has only about 5% of the value of a win for Atlanta, this is equivalent to only an additional 0.01 of win probability, bringing it to 0.085. So, if both coaches have full knowledge of the implications of a tie, it still seems that the Falcons should go for it, although the decision is closer than we first suggested. (We're confident, though, that neither coach had full knowledge of the implications of a tie.)

Evaluation of Individual Scenarios

Table 2
Index of Teams
 1 Buf     5 Balt    9 Hou    13 Den
 2 Mia     6 Cin    10 Ind    14 KC
 3 NE      7 Cle    11 Jax    15 Oak
 4 Jets    8 Pitt   12 TN     16 SD

17 Dal    21 Chi    25 Atl    29 Ari
18 NYG    22 Det    26 Car    30 StL
19 Phil   23 GB     27 NO     31 SF
20 Wash   24 MN     28 TB     32 Sea

The program for finding the playoff teams and seedings, given actual or hypothetical results for the regular season, is playoffteams.m. It calls the additional programs setup.m, standings.m, divtiebreaker.m, wctiebreaker.m, getwcteams.m, seeddivwinners.m, and getsbwinprobs.m, which are also used for the simulation described in the next section. The source code is written for MATLAB®.

The required inputs are contained in a data file named gameoutcomes.txt, with one row for every game in the regular season. Each row has three entries. The first two entries are the indices (from Table 2) of the teams in that game. The third entry in that row equals 1 if the team listed first won the game, 0 if the team listed first lost the game, and -1 if the game ended in a tie. When the program playoffteams.m is run using the file gameoutcomes.txt corresponding to the actual 2005 season, it produces the output

place =

     3     6    10    13    18    21    28    32
     2     8    11    14    20    24    26    30
     1     5    12    16    17    22    25    29
     4     7     9    15    19    23    27    31


seeds =

    10    32
    13    21
     6    28
     3    18
    11    26
     8    20

in which the entries are the indices of the teams, from Table 2. Naturally, this output matches the actual 2005 standings and playoff seedings.

The Simulation

The source code for the simulation is schedsim.m. The required inputs are contained in a data file gameprobs.txt which, analogous to the file gameoutcomes.txt described in the previous section, has one row for every game in the regular season. Each row has three entries. The first two entries are the indices (from Table 2) of the teams in that game. If the game has been completed, the third entry in that row equals 1 if the team listed first won the game, 0 if the team listed first lost the game, and -1 if the game ended in a tie. If the game has not been played, the third entry in the row is the probability that the team listed first will win. The file gameprobs.txt is currently set up for computing the implications of a tie between Atlanta and Tampa Bay. For example, the section of the data file corresponding to the Week 16 games is

    28    25     -1
     1     6     1
    26    17     0
    27    22     0
     9    11     0
    18    20     0
     7     8     0
    14    16     1
    30    31     0
     2    12     1
    29    19    0.5
    32    10    0.8
    13    15    0.85
    21    23    0.7
    24     5    0.4
     3     4    0.65

The second row in this excerpt says that the game between Buffalo and Cincinnati was won by Buffalo. (The row could have been entered equivalently as 6 1 0.) That game and the games corresponding to the next eight rows in the excerpt had been decided by the time Atlanta and Tampa Bay began their overtime session. The last row in the excerpt says that in the game between New England and the Jets, which had not been played when Atlanta and Tampa Bay went into overtime, New England's win probability was 0.65. The first row of the excerpt indicates a tie between Tampa Bay and Atlanta.

Table 3
Probabilities of
winning the SB if
ATL-TB ends in a tie
  1            0
  2            0
  3   0.05345856
  4            0
  5            0
  6   0.05943744
  7            0
  8   0.03164800
  9            0
 10   0.18000000
 11   0.03456000
 12            0
 13   0.14054400
 14   0.00035200
 15            0
 16            0
 17   0.01761280
 18   0.06063232
 19            0
 20   0.02974592
 21   0.12983408
 22            0
 23            0
 24   0.00461312
 25   0.00044800
 26   0.03707200
 27            0
 28   0.04158976
 29            0
 30            0
 31            0
 32   0.17845200

When the simulation schedsim.m is run with the data set gameprobs.txt, it produces the output shown in Table 3. The first column of Table 3 lists the teams according to the key shown in Table 2. The second column gives each team's estimated probability of winning the Super Bowl, if the Atlanta-Tampa Bay game ends in a tie. The probabilities sum to 1. To find the teams' playoff prospects if Tampa Bay wins, we change the -1 to 1 in the data set and rerun the simulation. To find the teams' playoff prospects if Tampa Bay loses, we change the -1 to 0 and rerun the simulation.

The most natural way to generate a scenario is to generate outcomes for the remaining regular-season games using the specified probabilities,5 determine the playoff seedings in each conference, and then continue the simulation through the playoffs to arrive at a Super Bowl winner. By generating many scenarios, and recording the fraction of the scenarios in which each team wins the Super Bowl, one obtains an estimate of each team's probability of winning the Super Bowl. However, we have chosen a different approach with less sampling error: conditional on the playoff seedings, we directly compute the teams' probabilities of winning the Super Bowl (in getsbwinprobs.m), rather than include the playoffs as part of the simulation. The result of each scenario is therefore not a Super Bowl winner, but rather each team's conditional probability of winning the Super Bowl, conditional on the results of the regular season. The average of these conditional probabilities, averaged over the generated scenarios, is the estimate of each team's probability of winning the Super Bowl.

Table 4
Super Bowl win
Probabilities
by seed under the
default assumptions
seed probability
#1 0.1800
#2 0.1440
#3 0.0576
#4 0.0518
#5 0.0346
#6 0.0320

For simplicity, the default assumption is a constant win probability for the home team in a playoff game. To generate Table 3, we assumed that a home playoff team has win probability 0.6, and that on a neutral field in the Super Bowl, each team has win probability 0.5. This assumption already generates substantial variation in the different seeds' probabilities of winning the Super Bowl, as shown in Table 4. However, as we explain in the appendix, the program gives users the option of specifying individual win probabilities for some or all possible playoff pairings. This option is important if the goal is to estimate the teams' odds of winning the Super Bowl, for betting purposes. However, it's overkill if the goal is merely to get a useful estimate the value of a tie.

The simulation is currently set to generate 1000 scenarios. This takes about a minute, and produces results that are as accurate as necessary.

One can use the simulation to estimate the teams' probabilities of making the playoffs, rather than their probabilities of winning the Super Bowl. To do this, one simply changes the variable seedweighttype in schedsim.m from 2 to 1.

Playoff Clinching and Elimination

It would be easy to create a version of the computer program with which one could, in principle, determine which teams have clinched a playoff slot, and which teams have been eliminated: For each possible scenario for the remaining regular-season games, one calculates the playoff teams. A team that makes the playoffs in every scenario has clinched, whereas a team that doesn't make the playoffs in any scenario has been eliminated.

This approach works only in principle because if n games remain, there are 2n possible scenarios. With even two full weeks (32 games) left in the season, the number of scenarios is 232 = 4,294,967,296. Some simplification is possible, of course. For example, to determine whether a particular team has been eliminated, it suffices to examine only those scenarios in which that team wins its remaining games. Further simplifications aren't so obvious, though, because some tiebreakers (such as strength of victory) hinge on the outcomes of a large number of games. Consequently, this approach is computationally impractical with a personal computer if more than about eighteen games remain.

We recommend simply running the simulation with, say, ten thousand trials. A team that makes the playoffs in every one of those trials has clinched a playoff slot for all practical purposes, if not necessarily literally. Similarly, a team that doesn't make the playoffs in any of those trials has (for all practical purposes) been eliminated.

The Tiebreaking Algorithm

The program for determining playoff seedings using the NFL tiebreaking procedures starts by computing (in setup.m) each team's overall won-lost-tied (W-L-T) percentage, W-L-T percentage in its division, W-L-T percentage in its conference, strength of victory, and strength of schedule.

To keep the algorithm simple, we treat overall W-L-T percentage as an honorary "tiebreaker." With that convention, the following is the sequence of tiebreakers (applied in divtiebreaker.m) for determining the standings within a division:

0. Overall won-lost-tied percentage.
1. Head-to-head (best won-lost-tied percentage in games among the teams).
2. Won-lost-tied percentage in the division.
3. Won-lost-tied percentage versus common opponents.
4. Won-lost-tied percentage in the conference.
5. Strength of victory.
6. Strength of schedule.

The following is the sequence of tiebreakers (applied in wctiebreaker.m) for finding the wild-card teams in a conference, or seeding the division winners:

0. Overall won-lost-tied percentage.
1. Head-to-head sweep. (Applicable only if one team has defeated each of the others or if one team has lost to each of the others).
2. Won-lost-tied percentage in the conference.
3. Won-lost-tied percentage in games against common opponents (minimum of four).
4. Strength of victory.
5. Strength of schedule.

In either sequence, if only two teams are involved, tiebreaker 1 reduces to ordinary head-to-head.

The NFL tiebreaking procedures include several additional tiebreakers having to do with points scored and allowed. These are not included in the computer program. In any application of the tiebreaking procedures to a group of teams, if the teams remain tied for all tiebreakers through strength of schedule, the program arbitrarily eliminates one of the teams.

Two subtleties of the tiebreaking procedures occasionally cause confusion. First, if a specific tiebreaker eliminates one or more teams, but two or more teams remain, then the remaining teams revert to step 0 of the relevant sequence of tiebreakers. Second, only one team from each division (the division's highest-ranking team that is not already in the playoffs) can compete for a particular wild-card slot.6

The first step in the computer program is to determine the standings in each division. The full ordering is required because, in principle, even the third-place finisher in a division can make the playoffs. To illustrate the steps, performed by standings.m, we will explain in detail how the program determines the final standings in the AFC East for the 2002 season. Because the program follows simple rules that work in all cases, it often performs steps that a person would recognize as unnecessary.

Team W L T PCT AFC DIV
Buffalo 8 8 0 .500 5-7 2-4
Miami 9 7 0 .562 7-5 2-4
New England 9 7 0 .562 6-6 4-2
N.Y. Jets 9 7 0 .562 6-6 4-2

The program first finds the first-place team. From the overall W-L-T percentages (tiebreaker 0) Buffalo is eliminated, leaving Miami, New England, and the Jets. Those three teams have identical overall W-L-T percentages (tiebreaker 0) and identical 2-2 records in games among the teams (tiebreaker 1). However, Miami is eliminated based on W-L-T percentage in the division (tiebreaker 2), leaving New England and the Jets. Those two teams have identical overall W-L-T percentages (tiebreaker 0), identical 1-1 records in games against each other (tiebreaker 1), and identical W-L-T percentages in the division (tiebreaker 2). However, the Jets have a better record versus common opponents (tiebreaker 3). So, the Jets win the division.

The program next determines the second-place finisher. The contenders are Buffalo, Miami, and New England. From the overall W-L-T percentages (tiebreaker 0) Buffalo is eliminated, leaving Miami and New England. Those two teams have identical overall W-L-T percentages (tiebreaker 0) and identical 1-1 records in games against each other (tiebreaker 1). However, Miami is eliminated based on W-L-T percentage in the division (tiebreaker 2), making New England the second-place finisher.

The contenders for third place are Buffalo and Miami. Comparing their overall W-L-T percentages (tiebreaker 0), Buffalo is eliminated, giving Miami third place. Buffalo finishes fourth.

Once the program has determined the final standings in each division, the next step is to determine the playoff seedings for the four division-winners in each conference. This is done by regarding those four teams as comprising a pseudo-division. The program seeddivwinners.m computes the "standings" in that pseudo-division, using wild-card tiebreakers rather than divisional tiebreakers. The ordering so derived gives the #1 through #4 seeds for the conference playoffs.

The final step is to find the wild-card teams. Within a conference, the contenders for the #5 seed are the second-place teams from the four divisions. We regard those four teams as forming a pseudo-division, and (in getwcteams.m, using wild-card tiebreakers) find the pseudo-division winner. That team is the #5 seed in the conference.

The four contenders for the #6 seed are the third-place finisher from the #5-seed's division, and the second-place teams from the other three divisions. Regarding these four contenders as forming a pseudo-division, and using wild-card tiebreakers, we find the pseudo-division winner. That team is the #6 seed in the conference.

As an illustration, we will list the steps that the program performs to find the 2004 NFC wild-card teams. As shown by the standings, the four second-place finishers are the Giants, Minnesota, New Orleans, and St. Louis. The Giants are eliminated based on overall W-L-T percentage (tiebreaker 0), leaving Minnesota, New Orleans, and St. Louis. In the only games among these three teams during the regular season, Minnesota defeated New Orleans, and New Orleans beat St. Louis. Therefore, none of the teams beat both of the others, nor did any of the teams lose to both of the others (tiebreaker 1). But among the three teams, St. Louis has the best W-L-T percentage in the conference (tiebreaker 2), and so gets the #5 seed.

The contenders for the #6 seed are the Giants, Minnesota, New Orleans, and Arizona (the third-place finisher in the NFC West). The Giants and Arizona are eliminated based on overall W-L-T percentage (tiebreaker 0). Then, since Minnesota defeated New Orleans (tiebreaker 1), Minnesota claims the #6 seed.

Appendix: Specifying Individual Playoff-Game Win Probabilities

The simulation schedsim.m is currently set to assume that in a playoff game, the home team wins with probability 0.6, and that in the Super Bowl, each team's win probability is 0.5. Users can of course change the 0.6 to some other value while retaining the assumption that the win probability for the home team in a playoff game is the same regardless of the specific teams involved. However, the program also gives users the option of specifying individual win probabilities for some or all possible playoff pairings. To do this one must change the variable seedweighttype from 2 to 3, and create an input file called weightoverrides.txt. That file has a row for each playoff pairing for which the user wants to override the default probabilities. Each row must have five numbers. For example, the row

10  14  0.7  0.5  0.6  

says that if Indianapolis (team 10) hosts Kansas City (team 14) in the playoffs, Indianapolis's win probability is 0.7; if the game is in Kansas City, Indianapolis's win probability is 0.5; and if the game is on a neutral field (which can't happen in this case because the teams are from the same conference), Indianapolis's win probability is 0.6. To override the default assumptions completely would require a row for every pair of teams that could possibly meet in the playoffs. At least until late in the regular season, when most teams have been eliminated from playoff contention, the file would be very large. We feel that the assumption of a constant home-team playoff win probability is adequate for many purposes.


FOOTNOTES

   1 This computer program was inspired by communication with King Kaufman, sports columnist for Salon.

   2 The actual relative value of a tie is slightly smaller, because our original analysis omitted two scenarios in which Atlanta makes the playoffs following a win against Tampa Bay: Atlanta secures a playoff berth if Washington loses, Dallas wins, Minnesota wins twice, and Chicago wins in Week 16; or if Dallas loses, Washington wins, Minnesota wins twice, the Giants win, and Chicago wins in Week 16.

   3 The relative value of a tie, determined this way, is the value v that we explained in our article on overtime.

   4 Remarkably, it's possible for Tampa Bay to earn a bye even after a loss to Atlanta. If Tampa Bay, Carolina, Minnesota, Chicago, Washington, and the Giants all finish 10-6, the NFC division winners are Washington, Minnesota, Tampa Bay, and Seattle. Tampa Bay then gets the #2 seed behind Seattle because they beat both Washington and Minnesota during the regular season. However, the probability of this scenario is less than 0.005.

   5 In the simulation, the outcomes of the unplayed games are treated as independent. This is not strictly correct. For one thing, game outcomes affect perceptions of how good the teams are, and therefore affect our subjective probabilities for the outcomes of subsequent games. In addition, a team's motivation to win a later game can be influenced by the results of an earlier game, either because they become locked into a playoff seeding (for example Seattle in 2005 Week 17) or actually prefer a lower seeding (as was apparently the case for New England in 2005 Week 17). The simulation could be modified to incorporate these types of dependencies, but we doubt it would be worth the trouble.

   6 This rule is sometimes critical. For example, in 2002, if all of the AFC's 9-7 teams had been eligible for the final wild-card slot, Miami would have made the playoffs. But since the Dolphins finished behind the Jets and Patriots in the AFC East, and the Patriots were not the #5 seed, the Dolphins were shut out.


Copyright © 2006 by William S. Krasker