PAR-ty With Marty: a Deep Dive Into How Power Rankings Work

Hey everyone! Over the years, a common question asked about Power Rankings is how PAR (Points Above Replacement) is calculated. The formula itself is fairly straightforward, but relies heavily on opponent performance to calculate. It’s gone through a few different iterations over time with PAR scores varying slightly. The current formula is what I consider the “final” version, something I plan to stick with for the foreseeable future, so it’s finally time to share all of the details. Before we get into the formula, I want to establish what the current PAR system was made to do. 

I created this with a few goals in mind:
​

• The formula would truly represent PAR by calculating how a replacement player would perform in a given schedule. We should be able to point at a player and say “this person got x more points than someone should have gotten.” An issue with PAR formulas before this one was that they didn’t truly represent what PAR was supposed to stand for.
• Strength of schedule would be a strong consideration in determining PAR. Players should be rewarded for facing harder players.
• The formula would be dynamic, accounting for more information as the season progresses. Sometimes, a player is lucky or unlucky early in the season, and more games will allow for this randomness to account for less over time.
• DQ wins aren’t considered matches played, but players are still punished for receiving a DQ loss to discourage them.

With that, the formula used would require a few assumptions made:

• Matches played are considered to be between players of equal or similar skill level. This is to ignore PR discrepancies primarily, which can otherwise complicate the formula. Fortunately, we’ve observed over the years that the PR system does a good job of keeping most players within similar win rates with low variance. There are outliers in some seasons, but players are fairly matched for the most part.
• Game win rates follow a normal distribution and fall between 25% and 75%. While there are a few players that manage to achieve higher and lower win rates than this (mostly due to playing so few games), Hearthstone’s design makes it extremely difficult to have win rates that stray far from 50%. The 25% and 75% limits are arbitrary, but match the average score of a THL match: 3-1. This is mainly so players aren’t rewarded or punished for playing against opponents that have unusually high or low game win rates.
• Players who haven’t played other matches are assumed to have a 50% game win rate. This is to simplify calculations on expected win rate.

​Now, onto the formula:

Where X is points (excluding DQ wins), matches played exclude DQ wins, and E(X) is the expected amount of points against a player’s opponents (also excluding DQ wins), shown in the formula below.

Figuring out how many points given a win is easy, as the winner of a match always gets 4 points. When a player loses, however, they can receive 0, 1, or 2 points. We can ignore a player getting 0 points as it will always have an expected value of 0. There are 3 ways for a player to get 1 point and 6 ways for them to get 2 points, as order matters here (Remember, the order always has the last game result as a loss with these scenarios!).

​The challenge with the above formula is finding the expected game win rate of a player, E(Gwinrate). This is done by looking at the average opponent win rate after removing games played against the player. Once we have this average, we rescale the win rate using the assumed 25%-75% range and find a player’s expected game win rate against these opponents.​

Match win rate can then be found via a cumulative distribution function (CDF), as winning and losing follows a binomial distribution. We can find the CDF of getting two or fewer wins (losing a match), then subtract that from 1 to get the odds of winning a match. The reason why this works is because although we don’t play any games after someone wins 3 times in a match, we’re effectively looking for someone to have at least 3 wins in 5 games. The sum represents all possibilities of getting 0, 1, or 2 wins in a match (there is no overlap here, so we’re not double counting). Subtracting that sum from 1, therefore, will give us what we’re looking for.

​Once we have the expected game win rate and expected match win rate for a player, we can calculate their PAR and compare them to other players! As the weeks go on, the values used to found PAR are also adjusted to account for changes in opponent game win rates. It’s a dynamic system that rewards players for winning against stronger players.

The system is by no means perfect, as THL will always have randomness involved. However, there is evidence to suggest that the system is robust. Take the average score given a loss. According to the formula, the average match (read: 50% game win rate and match win rate) should result in the loser scoring 1.125 points. This is in line with the average amount of points a player tends to get when losing in THL, which tends to sway between 1.10 and 1.15 points in all series each qualifying season (some seasons weren’t best of 5, but the PAR formula was adjusted those seasons to account for this). We also see historically strong players finishing with higher PAR scores most seasons, in line with what you would expect. Over time as match randomness flattens out, we see players often considered THL’s best with the highest PAR  and PAR per match over THL’s lifetime. It’s a good sign when the formula matches with the eye test. 

How does PAR look across different series? In general, PR series see less variance in PAR among players,, while non-PR series see higher variance. This makes sense, as players in a PR series are expected to play against players of equal skill determined by PR in a designated seed. A player’s match win rate doesn’t matter when deciding who they face, so their opponent win rate will naturally follow a narrower win-loss spread, and therefore, a narrower PAR spread. Non-PR series, meanwhile, purposely set up the best players to face off against each other. This kind of scheduling leads to a more difficult schedule for the very best, who then benefit from being expected to lose more against oher top players. PAR displays a feast-or-famine system that reflects exactly how the series is played out. 

For team power rankings, we use a similar approach to player power rankings. We sum the PAR of all players on a given roster. The more a team’s players are overperforming, the higher the team’s PAR score. The idea here is to estimate roster strength first and foremost, meaning the system never considers subs or previously rostered players. If a player isn’t supposed to be playing games for a team every week, then they aren’t included in a team’s PAR.

Team power rankings across different series follow a similar pattern seen with player power rankings. It’s extremely difficult to shelter players in a PR series the same way you can in a non-PR series, so teams tend to have flatter PAR scores. In non-PR series where teams can do this, however, the best teams will appear far ahead of everyone else. It’s the same feast-or-famine system that we’ve come to expect from these kinds of series.

Occasionally, you’ll find a few interesting nuances in team power rankings. Sometimes, for instance, you’ll see a team skyrocket or plummet from their previous position without any real change in their performance. Usually, this is because they’ve replaced a player on their roster. When a player with DQs gets replaced, for example, a team experiences addition by subtraction. Likewise, when a player with a high PAR gets replaced by a new player with no matches played that season, a team can drop off of the power rankings entirely. You’ll also see PAR scores for a team change during bye weeks. The players may not be playing more matches, but their opponents are, shifting their opponent win rates and impacting the amount of points each player is expected to score.

Overall, we’ve been pretty satisfied with how the current PAR system has performed over the last few years. The system also provides us with the option to create other kinds of content, like predicting matches and ranking the strongest teams of all time in each series. None of these are in the works yet, but maybe this post inspires a few readers. If you’re looking to mess around with this a bit more, then you know who to ask.

​- MartyB