Pairwise Comparison System
What it is: In the early 1990s, the NCAA Men's Ice Hockey Committee instituted a system designed to objectively compare teams to each other. The methodology evolved over time, getting more and more precise until becoming as it is today. Also, the criteria used has fluctuated somewhat over time. Three are currently used.
- The three criteria: The current criteria for comparing one team to another consists of:
- Ratings Percentage Index (RPI), adjusted for a variety of factors (see below)
- Record vs. Common Opponents
- Head-to-Head record
The most notable change to the selection criteria over the years has been the removal of Record in Last 16 (or 20) Games. In addition, as of 2013-14, the Record vs. TUC criterion was removed, and replaced with a sliding-scale "Quality Win Bonus."
How it's applied: Each "Team Under Consideration" (TUC) is compared to every other "Team Under Consideration" (see below for TUC definition), using the three criteria. Within each "comparison," one point is awarded for winning each criterion. One point is also awarded for each head-to-head win. The team with the most "criteria points" at the end of this process, wins that comparison. If the comparison ends in a tie, it's broken by determining which team has the better RPI. This procedure is repeated for every possible TUC pair. The final number represented in the Pairwise Comparison Rating chart is the amount of "comparison wins" (PCWs) each team has.
How the teams are ultimately selected: Using the chart, the teams are listed in order on the basis of most "comparisons" won. When taking out the teams that qualify automatically (by virtue of winning their conference tournament), the remaining top teams are then selected to fill out the 16-team field. If there is a tie in the amount of total comparisons won, this tie is also broken by comparing the two teams' RPI. (Note: That method of breaking ties is not outlined anywhere, and has simply been ascertained through experience and observation. Likewise, the ordering of teams in the chart -- based on total comparisons won -- is also not outlined anywhere. Other methods have been used in the past that, while practically amount to the same thing, are not exactly the same. See below, and this article.)
History: To understand this further, it's important to know the history of the system. There came a time when the hockey community decided it wanted to take subjectivity out of the process. The Pairwise Comparison system was born. Originally, the system was designed as a way to objectively compare teams that were close in RPI, i.e. "on the bubble" of getting into the tournament. Once that bubble was ascertained by the committee (a subjective process in a sense, but not practically), the committee checked the individual comparisons among the teams, and figured out who was "winning the comparisons" against each other. It was only third-party sources -- after learning of this methodology's details -- that originally totaled up all the "comparison wins" and presented them in a chart in ranking form. This kind of chart was ultimately popularized by U.S. College Hockey Online, the first Internet-only college hockey media organization, which went on-line in 1996. Some time over the next seven years, life imitated art -- in other words, the committee's methods morphed, and it began to actually utilize the chart, as is, without doing any micro-observation of the individual comparisons.
