Sports rating system

Last updated

A sports rating system is a system that analyzes the results of sports competitions to provide ratings for each team or player. Common systems include polls of expert voters, crowdsourcing non-expert voters, betting markets, and computer systems. Ratings, or power ratings, are numerical representations of competitive strength, often directly comparable so that the game outcome between any two teams can be predicted. Rankings, or power rankings, can be directly provided (e.g., by asking people to rank teams), or can be derived by sorting each team's ratings and assigning an ordinal rank to each team, so that the highest rated team earns the #1 rank. Rating systems provide an alternative to traditional sports standings which are based on win–loss–tie ratios.

Contents

College football players in the United States UCF at the Texas goal line.jpg
College football players in the United States

In the United States, the biggest use of sports ratings systems is to rate NCAA college football teams in Division I FBS, choosing teams to play in the College Football Playoff. Sports ratings systems are also used to help determine the field for the NCAA men's and women's basketball tournaments, men's professional golf tournaments, professional tennis tournaments, and NASCAR. They are often mentioned in discussions about the teams that could or should receive invitations to participate in certain contests, despite not earning the most direct entrance path (such as a league championship). [1]

Computer rating systems can tend toward objectivity, without specific player, team, regional, or style bias. Ken Massey writes that an advantage of computer rating systems is that they can "objectively track all" 351 college basketball teams, while human polls "have limited value". [2] Computer ratings are verifiable and repeatable, and are comprehensive, requiring assessment of all selected criteria. By comparison, rating systems relying on human polls include inherent human subjectivity; this may or may not be an attractive property depending on system needs.

History

Sports ratings systems have been around for almost 80 years, when ratings were calculated on paper rather than by computer, as most are today. Some older computer systems still in use today include: Jeff Sagarin's systems, the New York Times system, and the Dunkel Index, which dates back to 1929. Before the advent of the college football playoff, the Bowl Championship Series championship game participants were determined by a combination of expert polls and computer systems.

Theory

Sports ratings systems use a variety of methods for rating teams, but the most prevalent method is called a power rating. The power rating of a team is a calculation of the team's strength relative to other teams in the same league or division. The basic idea is to maximize the amount of transitive relations in a given data set due to game outcomes. For example, if A defeats B and B defeats C, then one can safely say that A>B>C.

There are obvious problems with basing a system solely on wins and losses. For example, if C defeats A, then an intransitive relation is established (A > B > C > A) and a ranking violation will occur if this is the only data available. Scenarios such as this happen fairly regularly in sports—for example, in the 2005 NCAA Division I-A football season, Penn State beat Ohio State, Ohio State beat Michigan, and Michigan beat Penn State. To address these logical breakdowns, rating systems usually consider other criteria such as the game's score and where the match was held (for example, to assess a home field advantage). In most cases though, each team plays a sufficient amount of other games during a given season, which lessens the overall effect of such violations.

From an academic perspective, the use of linear algebra and statistics are popular among many of the systems' authors to determine their ratings. Some academic work is published in forums like the MIT Sloan Sports Analytics Conference, others in traditional statistics, mathematics, psychology, and computer science journals.

If sufficient "inter-divisional" league play is not accomplished, teams in an isolated division may be artificially propped up or down in the overall ratings due to a lack of correlation to other teams in the overall league. This phenomenon is evident in systems that analyze historical college football seasons, such as when the top Ivy League teams of the 1970s, like Dartmouth, were calculated by some rating systems to be comparable with accomplished powerhouse teams of that era such as Nebraska, USC, and Ohio State. This conflicts with the subjective opinion that claims that while good in their own right, they were not nearly as good as those top programs. However, this may be considered a "pro" by non-BCS teams in Division I-A college football who point out that ratings systems have proven that their top teams belong in the same strata as the BCS teams. This is evidenced by the 2004 Utah team that went undefeated in the regular season and earned a BCS bowl bid due to the bump in their overall BCS ratings via the computer ratings component. They went on to play and defeat the Big East Conference champion Pittsburgh in the 2005 Fiesta Bowl by a score of 35-7. A related example occurred during the 2006 NCAA men's basketball tournament where George Mason were awarded an at-large tournament bid due to their regular season record and their RPI rating and rode that opportunity all the way to the Final Four.

Goals of some rating systems differ from one another. For example, systems may be crafted to provide a perfect retrodictive analysis of the games played to-date, while others are predictive and give more weight to future trends rather than past results. This results in the potential for misinterpretation of rating system results by people unfamiliar with these goals; for example, a rating system designed to give accurate point spread predictions for gamblers might be ill-suited for use in selecting teams most deserving to play in a championship game or tournament.

Rating considerations

Home advantage

France national basketball team fans France national team fans.jpg
France national basketball team fans

When two teams of equal quality play, the team at home tends to win more often. The size of the effect changes based on the era of play, game type, season length, sport, even number of time zones crossed. But across all conditions, "simply playing at home increases the chances of winning." [3] A win away from home is therefore seen more favorably than a win at home, because it was more challenging. Home advantage (which, for sports played on a pitch, is almost always called "home field advantage") is also based on the qualities of the individual stadium and crowd; the advantage in the NFL can be more than a 4-point difference from the stadium with the least advantage to the stadium with the most. [4]

Strength of schedule

Strength of schedule refers to the quality of a team's opponents. A win against an inferior opponent is usually seen less favorably than a win against a superior opponent. Often teams in the same league, who are compared against each other for championship or playoff consideration, have not played the same opponents. Therefore, judging their relative win–loss records is complicated.

We looked beyond the record. The committee placed significant value on Oregon's quality of wins.

—  College football playoff committee chairman Jeff Long, press conference, week 12 of the 2014 season, [5] after ranking 9–1 Oregon above 9–0 Florida State

The college football playoff committee uses a limited strength-of-schedule algorithm that only considers opponents' records and opponents' opponents' records [6] (much like RPI).

Points versus wins

A key dichotomy among sports rating systems lies in the representation of game outcomes. Some systems store final scores as ternary discrete events: wins, draws, and losses. Other systems record the exact final game score, then judge teams based on margin of victory. Rating teams based on margin of victory is often criticized as creating an incentive for coaches to run up the score, an "unsportsmanlike" outcome. [7]

Still other systems choose a middle ground, reducing the marginal value of additional points as the margin of victory increases. Sagarin chose to clamp the margin of victory to a predetermined amount. [8] Other approaches include the use of a decay function, such as a logarithm or placement on a cumulative distribution function.

In-game information

Beyond points or wins, some system designers choose to include more granular information about the game. Examples include time of possession of the ball, individual statistics, and lead changes. Data about weather, injuries, or "throw-away" games near season's end may affect game outcomes but are difficult to model. "Throw-away games" are games where teams have already earned playoff slots and have secured their playoff seeding before the end of the regular season, and want to rest/protect their starting players by benching them for remaining regular season games. This usually results in unpredictable outcomes and may skew the outcome of rating systems.

Team composition

Teams often shift their composition between and within games, and players routinely get injured. Rating a team is often about rating a specific collection of players. Some systems assume parity among all members of the league, such as each team being built from an equitable pool of players via a draft or free agency system as is done in many major league sports such as the NFL, MLB, NBA, and NHL. This is certainly not the case in collegiate leagues such as Division I-A football or men's and women's basketball.

Cold start

At the beginning of a season, there have been no games from which to judge teams' relative quality. Solutions to the cold start problem often include some measure of the previous season, perhaps weighted by what percent of the team is returning for the new season. ARGH Power Ratings is an example of a system that uses multiple previous years plus a percentage weight of returning players.

Rating methods

Permutation of standings

Several methods offer some permutation of traditional standings. This search for the "real" win–loss record often involves using other data, such as point differential or identity of opponents, to alter a team's record in a way that is easily understandable. Sportswriter Gregg Easterbrook created a measure of Authentic Games, which only considers games played against opponents deemed to be of sufficiently high quality. [9] The consensus is that all wins are not created equal.

I went through the first few weeks of games and redid everyone’s records, tagging each game as either a legitimate win or loss, an ass-kicking win or loss, or an either/or game. And if anything else happened in that game with gambling repercussions – a comeback win, a blown lead, major dysfunction, whatever — I tagged that, too.

—  Bill Simmons, sportswriter, Grantland [10]

Pythagorean

Pythagorean expectation, or Pythagorean projection, calculates a percentage based on the number of points a team has scored and allowed. Typically the formula involves the number of points scored, raised to some exponent, placed in the numerator. Then the number of points the team allowed, raised to the same exponent, is placed in the denominator and added to the value in the numerator. Football Outsiders has used [11]

The resulting percentage is often compared to a team's true winning percentage, and a team is said to have "overachieved" or "underachieved" compared to the Pythagorean expectation. For example, Bill Barnwell calculated that before week 9 of the 2014 NFL season, the Arizona Cardinals had a Pythagorean record two wins lower than their real record. [12] Bill Simmons cites Barnwell's work before week 10 of that season and adds that "any numbers nerd is waving a “REGRESSION!!!!!” flag right now." [13] In this example, the Arizona Cardinals' regular season record was 8-1 going into the 10th week of the 2014 season. The Pythagorean win formula implied a winning percentage of 57.5%, based on 208 points scored and 183 points allowed. Multiplied by 9 games played, the Cardinals' Pythagorean expectation was 5.2 wins and 3.8 losses. The team had "overachieved" at that time by 2.8 wins, derived from their actual 8 wins less the expected 5.2 wins, an increase of 0.8 overachieved wins from just a week prior.

Trading "skill points"

Originally designed by Arpad Elo as a method for ranking chess players, several people have adapted the Elo rating system for team sports such as basketball, soccer and American football. For instance, Jeff Sagarin and FiveThirtyEight publish NFL football rankings using Elo methods. [14] Elo ratings initially assign strength values to each team, and teams trade points based on the outcome of each game.

Solving equations

Researchers like Matt Mills use Markov chains to model college football games, with team strength scores as outcomes. [15] [16] Algorithms like Google's PageRank have also been adapted to rank football teams. [17] [18]

List of sports rating systems

Bowl Championship Series computer rating systems

In collegiate American football, the following people's systems were used to choose teams to play in the national championship game.

Further reading

Bibliographies

Academic work

Related Research Articles

<span class="mw-page-title-main">Elo rating system</span> Method for calculating relative skill levels of players

The Elo rating system is a method for calculating the relative skill levels of players in zero-sum games such as chess. It is named after its creator Arpad Elo, a Hungarian-American physics professor.

A tournament is a competition involving at least three competitors, all participating in a sport or game. More specifically, the term may be used in either of two overlapping senses:

  1. One or more competitions held at a single venue and concentrated into a relatively short time interval.
  2. A competition involving a number of matches, each involving a subset of the competitors, with the overall tournament winner determined based on the combined results of these individual matches. These are common in those sports and games where each match must involve a small number of competitors: often precisely two, as in most team sports, racket sports and combat sports, many card games and board games, and many forms of competitive debating. Such tournaments allow large numbers to compete against each other in spite of the restriction on numbers in a single match.
<span class="mw-page-title-main">Peach Bowl</span> Annual American college football postseason game

The Peach Bowl is an annual college football bowl game played in Atlanta, Georgia, since December 1968. Since 1997, it has been sponsored by Chick-fil-A and is officially known as the Chick-fil-A Peach Bowl. From 2006 to 2013, it was officially referred to as simply the Chick-fil-A Bowl. The winner of the bowl game is awarded the George P. Crumbley Trophy, named after the game's founder George Crumbley.

<span class="mw-page-title-main">Bowl Championship Series</span> American college football playoff series

The Bowl Championship Series (BCS) was a selection system that created four or five bowl game match-ups involving eight or ten of the top ranked teams in the NCAA Division I Football Bowl Subdivision (FBS) of American college football, including an opportunity for the top two teams to compete in the BCS National Championship Game. The system was in place for the 1998 through 2013 seasons and in 2014 was replaced by the College Football Playoff.

<span class="mw-page-title-main">Bowl game</span> Category of football games in North America

In North America, a bowl game, or simply bowl, is one of a number of postseason college football games that are primarily played by teams belonging to the NCAA's Division I Football Bowl Subdivision (FBS). For most of its history, the Division I Bowl Subdivision had avoided using a playoff tournament to determine an annual national champion, which was instead traditionally determined by a vote of sports writers and other non-players. In place of such a playoff, various cities across the United States developed their own regional festivals featuring postseason college football games. Prior to 2002, bowl game statistics were not included in players' career totals. Despite attempts to establish a permanent system to determine the FBS national champion on the field, various bowl games continue to be held because of the vested economic interests entrenched in them.

A sports league is a group of individual athletes, sports teams or clubs who form a league to compete against each other and gain points in a specific sport. At its simplest, it may be a local group of amateur athletes who form teams among themselves and compete periodically, at its most complex, it can be an international professional league making large amounts of money and involving dozens of teams and thousands of players.

Jeff Sagarin is an American sports statistician known for his development of a method for ranking and rating sports teams in a variety of sports. His Sagarin Ratings have been a regular feature in the USA Today sports section from 1985 to 2023, have been used by the NCAA Tournament Selection Committee to help determine the participants in the NCAA Men's Division I Basketball Championship tournament since 1984, and were part of the college football Bowl Championship Series throughout its history from 1998 to 2014.

<span class="mw-page-title-main">Kenneth Massey</span> American sports statistician (born 1975)

Kenneth Massey is an American sports statistician known for his development of a methodology for ranking and rating sports teams in a variety of sports. His ratings have been a part of the Bowl Championship Series since the 1999 season. He is an assistant professor of mathematics at Carson–Newman University in Tennessee.

In a group tournament, unlike a knockout tournament, there is no scheduled decisive final match. Instead, all the competitors are ranked by examining the results of all the matches played in the tournament. Typically, points are awarded for each match, with competitors ranked based either on total number of points or average points per match.

In American college football, the 2006 BCS computer rankings are a part of the Bowl Championship Series (BCS) formula that determines who plays in the BCS National Championship Game as well as several other bowl games. Each computer system was developed using different methods which attempts to rank the teams' performance. For 2006, the highest and lowest rankings for a team are dropped and the remaining four rankings are summed. A team ranked #1 by a computer system is given 25 points, #2 is given 24 points and so forth. The summed values are then divided by 100. The values are then ranked by percentage. This percentage ranking is then averaged with the Coaches Poll and Harris Poll average rankings, each receiving equal weight, and the results become the BCS Rankings.

In American college football, the 2007 BCS computer rankings are a part of the Bowl Championship Series (BCS) formula that determines who plays in the BCS National Championship Game as well as several other bowl games. Each computer system was developed using different methods which attempts to rank the teams' performance. For 2007, the highest and lowest rankings for a team are dropped and the remaining four rankings are summed. A team ranked #1 by a computer system is given 25 points, #2 is given 24 points and so forth. The summed values are then divided by 100. The values are then ranked by percentage. This percentage ranking is then averaged with the Coaches Poll and Harris Poll average rankings, each receiving equal weight, and the results become the BCS Rankings.

The Bowl Championship Series (BCS) was a selection system used between 1998 and 2013 that was designed, through polls and computer statistics, to determine a No. 1 and No. 2 ranked team in the NCAA Division I Football Bowl Subdivision (FBS). After the final polls, the two top teams were chosen to play in the BCS National Championship Game which determined the BCS national champion team, but not the champion team for independent voting systems. This format was intended to be "bowl-centered" rather than a traditional playoff system, since numerous FBS Conferences had expressed their unwillingness to participate in a play-off system. However, due to the unique and often esoteric nature of the BCS format, there had been controversy as to which two teams should play for the national championship and which teams should play in the four other BCS bowl games. In this selection process, the BCS was often criticized for conference favoritism, its inequality of access for teams in non-Automatic Qualifying (non-AQ) Conferences, and perceived monopolistic, "profit-centered" motives. In terms of this last concern, Congress explored the possibility on more than one occasion of holding hearings to determine the legality of the BCS under the terms of the Sherman Anti-Trust Act, and the United States Justice Department also periodically announced interest in investigating the BCS for similar reasons.

In sports, strength of schedule (SOS) refers to the difficulty or ease of a team's/person's opponent as compared to other teams/persons. This is especially important if teams in a league do not play each other the same number of times.

<span class="mw-page-title-main">Fordham Rams football</span> Intercollegiate American football team for Fordham University

The Fordham Rams football program is the intercollegiate American football team for Fordham University, located in the borough of The Bronx in New York City. The team competes in the NCAA Division I Football Championship Subdivision (FCS) and are members of the Patriot League. Fordham's first football team was fielded 142 years ago in 1882; the team plays its home games on campus at 7,000-seat Coffey Field.

<span class="mw-page-title-main">2012 BCS National Championship Game</span> College football game

The 2012 Allstate BCS National Championship Game was a postseason college football bowl game between the Alabama Crimson Tide and the LSU Tigers, and determined the national champion of the 2011 NCAA Division I FBS football season on Monday, January 9, 2012, at the Mercedes-Benz Superdome in New Orleans, Louisiana. The game was part of the 2011–2012 Bowl Championship Series and a rematch of regular season foes. Alabama beat LSU 21–0 to win their 14th national championship, marking the first shutout in a national championship game since the 1992 Orange Bowl and the first ever shutout in a BCS bowl game. The game had the third-lowest TV rating, 14.01, in the 14-year history of the BCS National Championship game.

The 2008 Texas vs. Texas Tech football game was a Big 12 Conference college football game played between the Texas Longhorns and Texas Tech Red Raiders at Jones AT&T Stadium in Lubbock, Texas. The game was played on November 1 and was one of the most memorable games in the two teams' rivalry. Heading into the game, both teams were undefeated at 8–0. Texas entered game as the top-ranked team in the nation, led by coach Mack Brown. The Red Raiders, headed by coach Mike Leach, were ranked sixth. The Red Raiders stunned the Longhorns 39–33 on a last-second touchdown pass. The game appeared over on the previous play, but Texas dropped a potential interception. The game is one of the greatest upsets in the rivalry's history and was crucial in producing a three-way tie in the Big 12 at the end of the season.

The plus-one system, also known as a 4-team playoff, is the system used to determine the National Champion in the Football Bowl Subdivision of NCAA football in the United States. The format is of a 4-team playoff, where two bowl games act as semi-final games, and the winners of these games participate in the National Championship Game.

<span class="mw-page-title-main">College Football Playoff</span> Postseason tournament in American college football

The College Football Playoff (CFP) is an annual postseason knockout invitational tournament to determine a national champion for the National Collegiate Athletic Association (NCAA) Division I Football Bowl Subdivision (FBS), the highest level of college football competition in the United States. It culminates in the College Football Playoff National Championship game. The inaugural tournament was held at the end of the 2014 NCAA Division I FBS football season under a four-team format. The CFP expands to include twelve teams for the 2024 season.

<span class="mw-page-title-main">Mountain West Conference Football Championship Game</span> College football championship game

The Mountain West Conference Football Championship Game is an annual postseason college football game played to determine the champion of the Mountain West Conference (MW).

The Colley Matrix is a computer-generated sports rating system designed by Dr. Wesley Colley. It is one of more than 40 polls, rankings, and formulas recognized by the NCAA in its list of national champion selectors in college football.

References

  1. ↑ Fagan, Ryan (2011-03-09). "Sorting through teams on one big bubble". Sporting News . Retrieved 2011-03-24. This is a look at 20 of the teams (in alphabetical order) residing on this year's big ol' bubble. We've included three statistical rankings. The RPI (ratings percentage index, taken from collegeRPI.com) is considered the standard and is provided to committee members during the selection process. The two other ranking indexes include margin of victory in their formulas—the Pomeroy ratings (at kenpom.com) and Sagarin ratings (via USA Today)—aren't new but have played an increased role in discussions about potential seeds during this college basketball season.
  2. ↑ Ken Massey [@masseyratings] (November 3, 2014). "@kenpomeroy human polls have limited value. Computer systems can objectively track all the teams. http://www.masseyratings.com/cb/compare.htm #all351" (Tweet). Retrieved 9 Nov 2014– via Twitter.
  3. ↑ Jamieson, Jeremy P. (2010). "The Home Field Advantage in Athletics: A Meta-Analysis" (PDF). Journal of Applied Social Psychology. 40 (7): 1819–1848. doi:10.1111/j.1559-1816.2010.00641.x . Retrieved 11 November 2014.
  4. ↑ Barnwell, Bill (December 20, 2013). "Safe at Home". Grantland . Retrieved November 11, 2014.
  5. ↑ Russo, Ralph D. (11 November 2014). "Oregon up to 2 in playoff rankings; TCU to 4th". Associated Press. Retrieved 12 November 2014.
  6. ↑ Stewart Mandel [@slmandel] (November 12, 2014). "Committee doesn't use an SOS ranking. It looks at opponents' record and opponents' opponents record" (Tweet). Retrieved 12 Nov 2014– via Twitter.
  7. ↑ Richards, Darryl (2001). "BCS removes margin-of-victory element". Fox Sports . Retrieved 12 November 2014.
  8. ↑ Sagarin, Jeff (Fall 2014). "NCAAF Jeff Sagarin Ratings". USA Today . Retrieved 12 November 2014.
  9. ↑ Easterbrook, Gregg (18 November 2014). "More flags on D spins scoreboards". ESPN . Retrieved 19 November 2014.
  10. ↑ Simmons, Bill (24 October 2014). "Week 8 Picks: A Gambling Epiphany". Grantland . Retrieved 19 November 2014.
  11. ↑ Schatz, Aaron; Alamar, Ben; Barnwell, Bill; Bill Connelly; Doug Farrar (2011). Football Outsiders Almanac 2011: The Essential Guide to the 2011 NFL and College Football Seasons. CreateSpace. p. xviii. ISBN   978-1-4662-4613-3.
  12. ↑ Barnwell, Bill (November 5, 2014). "NFL at the Half: Breaking Down the Numbers". Grantland . Retrieved January 7, 2015.
  13. ↑ Simmons, Bill (7 November 2014). "Revisiting the Y2K-Compliant Quarterbacks" . Retrieved 10 November 2014.
  14. ↑ Silver, Nate (4 September 2014). "Introducing NFL Elo Ratings". FiveThirtyEight . Retrieved 10 November 2014.
  15. ↑ Mills, Matt (21 December 2014). "Using Continuous-Time Markov Chains to Rank College Football Teams". The Spread. Retrieved 21 December 2014.
  16. ↑ Jason, Brian. "1xBet Côte d'Ivoire" . Retrieved 16 October 2023.
  17. ↑ "Ranking NFL teams using Network Science". LinkedIN . 17 March 2016. Retrieved 17 March 2016.
  18. ↑ "Modifying Google's Page Ranking Algorithm to rank teams". Reddit . 21 December 2014. Retrieved 22 December 2014.
  19. ↑ Weng, Ruby C.; Lin, Chih-Jen (2011). "A Bayesian Approximation Method for Online Ranking" (PDF). Journal of Machine Learning Research. 12: 267–300.
  20. ↑ "Wayne Winston: Analytics in the World of Sports". Indiana University Bloomington - Kelley School of Business - Operations & Decisions Technologies. Nov 25, 2013. Retrieved 8 Nov 2014.
  21. ↑ "Numbers game". Washington Times . April 13, 2004. Retrieved 8 Nov 2014.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.