Reference - VsX comprehensive summary (1927 to 2023)

[Hockey Outsider]

Here's a thread from 2015 where I look at a method called "VsN" - Vs N - new adjusted scoring system

"N" stands for the number of teams in the league. So the benchmark for 2024 would be the average of the 1st through 32nd highest scorer in the league. The benchmark for 1954 would be the average of the 1st through 6th highest scorer.

Conceptually, that approach seems to make sense. It's drawing on a large enough pool of players that the impact of any outliers should be minimal. And the benchmark scales up/down based on the size of the league. The reason it never caught on is because the results felt too punitive towards older players. You can read through the results/commentary.

In terms of how to deal with watered down periods (ie WWII, early 1970s) - there are two broad approaches. We can either keep the system objective (no manual adjustments), but the results are obviously less meaningful (because players like Bill Cowley will have their result artificially inflated). The other option is we make some manual adjustments to the benchmark (which is less objective, but gives us a more meaningful result, because it bakes into the calculation the adjustment that all of us are making in our heads). Generally speaking, I prefer to be approximately right (rather than precisely wrong), so I favour the latter approach.

All that being said - I invested a lot of time in 2017 on automating the spreadsheets that drive the various VsX calculations. Now it's trivial to do the annual updates and/or to adjust the benchmarks. (Speaking frankly - I'm too busy these days to spend hours updating these spreadsheets. All of the updates for the end of the 2024 season took less than thirty minutes. It actually took longer to copy and paste the data into HFBoards-acceptable format, than to do the calculations). If anyone wants to see what the results look like under another set of benchmarks, let me know, as it's trivial for me to run the numbers.

 

[daver]

Here's a thread from 2015 where I look at a method called "VsN" - Vs N - new adjusted scoring system

"N" stands for the number of teams in the league. So the benchmark for 2024 would be the average of the 1st through 32nd highest scorer in the league. The benchmark for 1954 would be the average of the 1st through 6th highest scorer.

Conceptually, that approach seems to make sense. It's drawing on a large enough pool of players that the impact of any outliers should be minimal. And the benchmark scales up/down based on the size of the league. The reason it never caught on is because the results felt too punitive towards older players. You can read through the results/commentary.

In terms of how to deal with watered down periods (ie WWII, early 1970s) - there are two broad approaches. We can either keep the system objective (no manual adjustments), but the results are obviously less meaningful (because players like Bill Cowley will have their result artificially inflated). The other option is we make some manual adjustments to the benchmark (which is less objective, but gives us a more meaningful result, because it bakes into the calculation the adjustment that all of us are making in our heads). Generally speaking, I prefer to be approximately right (rather than precisely wrong), so I favour the latter approach.

All that being said - I invested a lot of time in 2017 on automating the spreadsheets that drive the various VsX calculations. Now it's trivial to do the annual updates and/or to adjust the benchmarks. (Speaking frankly - I'm too busy these days to spend hours updating these spreadsheets. All of the updates for the end of the 2024 season took less than thirty minutes. It actually took longer to copy and paste the data into HFBoards-acceptable format, than to do the calculations). If anyone wants to see what the results look like under another set of benchmarks, let me know, as it's trivial for me to run the numbers.

(1) Not taking league size into consideration is statistically unreasonable. It assumes there hasn't been an increase in the talent pool that generally has coincided, and is reflected, with the expansion of the league.

(2) Taking league size into consideration can become statistically unreasonable if taken at statistical face value i.e. multiplying by the # of teams so Top 5 from the O6 becomes Top 30 in the current era.

(3) Presuming it is "harder" for a GOAT talent to separate themselves from the pack with more teams/talent is unreasonable. Generally speaking, GOAT talent raises the top not because they are necessarily that much more talented than their peers but their competitive level/mental strength is also freakish. We have seen enough recent GOAT talent in many sports to dismiss the "harder to separate yourself" theory due to more talent.

I believe #2 is the point of contention that opens the door to many discussions and possible solutions to find the best method.

Using the VsX theory, I compare the PPG of a player to the average PPG of the #1/#2 scorer (with a correction for outliers). The presumption here is that, excluding outliers, the #1/#2 scorers in any season (from 1947 onwards), are generally going to finish close to those positions in most other seasons. Using a 20 year sample flattens out the strong years and the weak years and moves outlier season corrections to the margins.

Here are two statistical claims that can be made:

On average, the #3 scorer in the period from '47 to '67 finished a similar % behind #1/#2 to the #5 scorer in the period from '00 to '20. On average, the #5 scorer in the period from '47 to '67 finished a similar % behind #1/#2 to the #10 scorer in the period from '00 to '20.

Conclusion:

Generally speaking, it is more impressive to finish in the Top 3/5/10 in the current era than it is in the O6. Of course this does not apply blindly to every single season by every player. In ranking players from the '47 to '67 period vs. those from the '00 to the '20, it is not surprising that more O6 had Top 3/Top 5 and Top 10 finishes on their resumes (e.g. about 80 O6 players had a Top 10 finish vs. about 60 current players). There were more opportunities for the "pack" of elite scorers in the O6 to finish in the Top 10.

Crosby has raw similar career point and PPG finishes to Hull and Beliveau but was significantly closer to #1/#2 than they were. He is closer to Howe in PPG finishes than he was to Hull and Beliveau (as are McDavid and Jagr).

Does this move Crosby up a level from Hull and Beliveau? No, I do not think you can move players up and down levels using this method but I think it is reasonable to give Crosby an edge in this regard. It is also reasonable to not give him an edge as noted in the final sentence.

Broadly speaking, Howe was clearly the best offensive player from pre-expansion and Hull and Beliveau are somewhat clear #2/#3 among forwards (according to HOH).

Wayne and Mario hit heights that we can comfortably say were not hit before or since.

Jagr, Crosby and McDavid, through varying degrees of performances, have hit offensive levels that are closer to peak/prime Howe level than they are to any other #5 forward candidate.

So pre-expansion:

Howe
Hull/Beliveau

Post expansion:

Wayne/Mario/Orr
Jagr/Crosby/McDavid


From a strict statistical perspective,

Wayne/Mario/Orr
Howe/Jagr/Crosby/McDavid
Hull/Beliveau

I am hesitant to automatically move Hull/Beliveau down a tier as we simply cannot assume anything if they had happened to play in any other era. They were #2 in the era they played in; there is no real reason (at least statistically) to believe they perform any lower (or higher) in another era.

At some point, we have to accept that a GOAT talent from any era is a GOAT talent all-time.

 

[daver]

According to Vilica from the other thread: Historical relevance of Kucherov and MacKinnon's 2024 season?

here are how the last two years are rated using VsX:

McDavid 22/23 - 135

Kucherov 23/24 - 120 (13% behind)

MacKinnon 23/24 - 117 (15% behind)


In raw numbers,

McDavid 22/23 - 153

Kucherov 23/24 - 144 (6% behind)

MacKinnon 23/24 - 140 (9% behind)


That doesn't pass the smell test given league GPG was slightly lower this past season and the PPGs of the #10/#25/#50 scorers were almost identical from 22/23 to 23/24.

Is there something I am missing here that VsX captures?

I am assuming that the baseline of the #2? scorer is skewing the numbers.