Methods of comparing offense between eras

[TheDevilMadeMe]

This is a rough draft, but I think it's about time we talk about this, before we start using Vs2/VsX in situations where it really isn't appropriate.

Adjusted points (points adjusted to overall league scoring)

What it does:

• The goal of hockey is to win games. In a higher scoring environment, you need to score more goals (on average) to win, therefore every goal scored is worth less. Basic "adjusted points" simply adjust a player's point total based on the average number of goals scored each season.

Pros:

• Mathematically elegant. In a mathematical sense, the formula strictly determines how much each goal is worth and adjusts a player's scoring total
• Adjusted points take into account overall league scoring, therefore they are the best method to determine the offensive value of the average player.

Cons:

• Adjusted points perfectly account for the total amount of scoring, but not the distribution of scoring. It has been mathematically proven that in some eras, first line players score a higher or lower percentage of the league's offense. Adjusted points do not take this into account and will therefore underrate star players from certain eras (in particular the 1980s) and overrate star players from other eras.
• Hockey-reference's version of the stat introduces questionable variables that cause results that don't pass the smell test. Their numbers are pretty good post-expansion (where the goals to assists ratio and the size of rosters are relatively constant), but should not be used for pre-expansion players.
• Unfortunately, nobody else has really calculated adjusted points for pre-expansion players.

My take:

• Adjusted points are a good method of comparing the offense of average (non first line) post-expansion players. For ATD purposes where post-expansion non first-liners will rarely see power play time you are better off using adjusted even strength scoring. This would be a very valuable tool if someone would take the time to come up with an appropriate formula that applies to pre-expansion.

Adjusted even strength scoring

What it does:

• Same thing as adjusted points, but for even strength scoring only. Adjusts a player's point totals to the overall league scoring environment

Pros:

• Adjusted points are most useful for "average" players, and such players not only would probably not be used on an ATD power player, but also would probably have received inconsistent powerplay time in real life. Therefore, adjusted even strength scoring will probably be more useful around here than overall adjusted scoring, when it is available.

Cons:

• Only available for post expansion players

My take:

• The best method for comparing the offensive value of post-expansion players who rarely received first PP time.

Percentage methods (Vs2, VsX)

What it does:

• Attempts to solve the problem presented by adjusted stats - the problem that first line or star players score a disproportionate amount of league offense during different eras. It does this by pinning all comparisons of offense to star players - the second best scoring in the league in every year (or in the case of VsX, the 2nd best scoring except when we have reason to modify the standard).

Pros:

• Advantage over adjusted stats: By selecting one of the league's top scorers as the standard and comparing all other players to him, the percentage method does what it is designed to do - track the offense of the league's best players.
• Advantage over top 10 or top 20 finishes: Assuming the standard selected is reasonable, Vs2 or VsX is much less affected by the level competition. If a player is 90% as good as the standard, it doesn't matter how may or how few other players performed as well that given year.
• This appears to be the best method at taking into account the influx of European talent in the early 1990s.

Cons:

• Of all the methods, this one seems to be the most affected by league size. It is mathematically proven that the number of players meeting certain benchmarks increased dramatically when the league expanded in 1967.
• The probable reason: the Original 6 era had only 18 spots available for first liners, and players usually played the full power play. Any player who didn't get one of these spots would score significantly lower by this method than he would in a larger league where he got those offensively opportunities.
• This method should only be used for players who spent their entire primes as scoring line players who received first unit power play time. Caution needs to be used when applying the method to second tier stars from the Original 6 era, guys who would absolutely be good enough to play on the first unit in an expanded league, but who didn't because of the smaller league. How do you figure out when you have one of those guys? If he didn't regularly finish in the top 10-15 in in scoring, he probably didn't have one of the prime 18 spots in the league.

My take:

• The best method we have for comparing the offensive value of scoring stars across the eras.
• Also a great method for comparing the offense of other scoring line post-expansion players with each other, even if they weren't necessarily stars.
• Pre-expansion players who did not spend their entire primes in offensive roles should NOT be compared using this method.

Top 10/Top 20 points

What it does:

• A list of the top 10 or top 20 scorers in the league in any given year. Duh.

Pros:

• The least affected by league size of all the methods. A 10th place finish is a 10th place finish no matter the size of the league.
• Ease of use: Top 10 finishes are easily available on hockey-reference and top 20 finishes are easy to calculate.
• East of use 2: When attempting a crude comparison with a player who didn't play in the NHL, Top 10/20 finishes are easier to fudge/estimate.

Cons:

• The most affected by the talent pool. A 10th place finish can be vastly more impressive in a deeper talent pool. This is where the percentage method is a major improvement.
• Arbitrary cutoffs: Top 10 finishes make a 10th place finish look much better than an 11th. Top 20 finishes make a 20th place finish look much better than a 21st.

My take:

• An adequate "default" method if no better method is applicable, so long as you then make note of the talent pool.
• Unfortunately, until someone comes up with a form of "adjusted points" that passes muster for pre-expansion players, this is probably still the best method for comparing the offense of second tier pre-expansion players.

Click to read more...

 

[Bear of Bad News]

Bringing this one to the surface, now that it's been moved to "By The Numbers".

 

[Averick*]

I started a thread on this topic. The way I did it was that I put things in terms of a ratio of individual stats compared to total goals scored per game in a season. I picked 1982 as the baseline year because that was both the year when there was the highest goals per game in a season and it was also the year Gretzky set the record for most goals in a season. In other years, both proximate to that year and over the breadth of time, goals were more difficult to come by for a whatever reasons. To make things apples-to-apples, I used ratios to tweak individual stats to put them in terms of a base year (ie 1982).

The other thing that I had to do was put it in terms of an 80 game season (the # of games in 1982). That was an additional ratio tweak.

 

[TheDevilMadeMe]

I started a thread on this topic. The way I did it was that I put things in terms of a ratio of individual stats compared to total goals scored per game in a season. I picked 1982 as the baseline year because that was both the year when there was the highest goals per game in a season and it was also the year Gretzky set the record for most goals in a season. In other years, both proximate to that year and over the breadth of time, goals were more difficult to come by for a whatever reasons. To make things apples-to-apples, I used ratios to tweak individual stats to put them in terms of a base year (ie 1982).

The other thing that I had to do was put it in terms of an 80 game season (the # of games in 1982). That was an additional ratio tweak.

Basically a form of adjusted points, right? It seemed to show that pre-consolidation star forwards are going to greatly underrated by adjusted points, perhaps even moreso than 80s star forwards.

 

[matnor]

Here are two threads that partially address the issues raised in the OP:

http://hfboards.mandatory.com/showthread.php?t=801447
http://hfboards.mandatory.com/showthread.php?t=878672

I think the VsX-system is fairly good but I don't really understand why X is most often set to 2. It seems to me like it would be a much better idea to set X to 5 or 10 to deal with the problem with outliers. Maybe someone can explain it to me?

 

[Czech Your Math]

I think it's best to separate pre-expansion and post-expansion, then try to integrate the two based on the known differences in the effective pool of players available.

It seems like a larger sample, such as 1N (N=number of teams, so keeps opportunity fixed) would be more reliable and less subject to random variation or influence.

If you try to predict how many "simple adjusted" points the top 1N players will score, based on just a handful of factors, one can generally estimate within a standard deviation of the mean (and very often much less) just about every time. The main factors which I've identified so far are the impact of non-Canadians in the top 1N (what % the top 1N would decrease without their points), the amount of parity in the league (particularly the disparity in GF), and the frequency of PP opportunities. The non-Canadian presence may itself cause ~5-10% higher adjusted points from the top 1N players. Whether the increase in points indicates it's less difficult depends on the reason for the increase.

 

[Canadiens1958]

First Liners on the PP

The question of using first liners on the PP should be revisited.

Pre 1967 expansion teams regularly used specialists or depth players on the PP.

Prime example would be Yvan Cournoyer with the Montreal Canadiens who was the 4th RW on the depth chart. For Cournoyer the data is available:

http://www.hockey-reference.com/players/c/cournyv01.html

Overpass posted data about the New York Rangers using a five forward PP during Camille Henry's rookie season with him as a PP specialist. Going back into the thirties this was common. Four and five forward PPs were more effective offensively and allowed the dmen pairings to rest. Important when playing with only four dmen.

Post 1967 expansion, teams often used players from depth lines as PP specialists.

Paul Gardner would be one such example with the Penguins and earlier in his career:

http://www.hockey-reference.com/teams/PIT/1983.html

John Bucyk is an interesting study. Prime, Bruin first liner he saw very limited PP time. Post prime as a depth player he saw a lot more PP time:

http://www.hockey-reference.com/players/b/bucykjo01.html

Others past and recent, J.F. Sauve, Yvon Lambert, Tomas Holmstrom - 122/243 goals on the PP, 13:35 career TOI. All far from top line status but key PP members.

Just a small sampling from a very large group.

 

[hatterson]

The question of using first liners on the PP should be revisited.

Pre 1967 expansion teams regularly used specialists or depth players on the PP.

So if I follow, the argument is based on what scenario gives you a bigger advantage?

1.) Playing Sidney Crosby at ES for ~15-16 minutes and on the PP for ~4 and playing Brandon Sutter at ES for ~13 minutes and on the PP for ~1.

OR

2.) Playing Crosby at ES for ~19-20 minutes an on the PP for ~1 and playing Brandon Sutter at ES for ~10 minutes and on the PP for ~4

 

[Canadiens1958]

The Game

So if I follow, the argument is based on what scenario gives you a bigger advantage?

1.) Playing Sidney Crosby at ES for ~15-16 minutes and on the PP for ~4 and playing Brandon Sutter at ES for ~13 minutes and on the PP for ~1.

OR

2.) Playing Crosby at ES for ~19-20 minutes an on the PP for ~1 and playing Brandon Sutter at ES for ~10 minutes and on the PP for ~4

No argument. Just recognizing how the game has been played and coached to this day generating the resulting data.

Specifically the last two O6 regular seasons saw app 30% and 40% of the PP goals scored by specialists and depth players.

Very often the players used on the PP are dictated by game situations and not by picking an ideal line-up as your scenario presents.

The top offensive line forces the defending team to commit more penalties than depth players do. If the pre PP shift that generated the penalty ran long the offensive line would be too tired to start the PP unless a time-out was used. So depth players or specialists are used to start the PP. The coach is concerned with his post PP match-ups. This dictates which players get ice time during the second half of the PP.

 

[TheDevilMadeMe]

Here are two threads that partially address the issues raised in the OP:

http://hfboards.mandatory.com/showthread.php?t=801447
http://hfboards.mandatory.com/showthread.php?t=878672

I've seen those threads and they look promising. Keep in mind that the OP of this thread originated on the All Time Draft board, so it was originally a critique of the various methods of comparing offense that are used in the ATD. For me at least, it's easier to figure out the strengths and weaknesses of a method when I've seen it applied a few times. I'm also (in case you couldn't tell) much better at critiquing methods that other people come up with than creating them myself.

I do think your methods are well worth exploring. Perhaps you'll join the ATD next year?

I think the VsX-system is fairly good but I don't really understand why X is most often set to 2. It seems to me like it would be a much better idea to set X to 5 or 10 to deal with the problem with outliers. Maybe someone can explain it to me?

I had similar thoughts to you actually - if you set X at 5, you basically never have to worry about outliers, like you do if you set X at 2.

This is the conversation from the linked vs 2 thread. It's a little hard to see, but I linked to threads on adjusted even strength scoring, Vs2, VsX, and Top 10 and Top 20 points in the titles of the sections.

I would recommend reading or at least skimming the whole thread, but for your specific question, look at posts 21 until... i dunno post 33 or so.
http://hfboards.mandatory.com/showthread.php?t=1099631

I pretty much went out and said that Vs5 and Vs10 were superior to Vs2 because they are much gentler curves that aren't prone to much fluctuation between year to year. The main reason to use Vs2 seems to be this:

Problem with this is, pre-expansion, you are going to see some unholy scores for the top3 scorers a few times. Particularly from 49-55, the scoring lists were awful beyond the top few (this is the inverse of the problem you perceived last time when mackell had crap scores in two years he was a top-10 scorer)

I still think Vs5 is probably better when comparing post-expansion players, but it could run into issues pre-expansion as 70s showed.

Hockey Outsider suggested using the average of the 2nd-4th best scorer:

My take on the "Vs. #2" is here: http://hfboards.mandatory.com/showthread.php?t=589911&highlight=

I used the average of 2nd through 4th place. My rationale is that if you base a ranking on a single position (whether it's 2nd or 5th or 10th), there will always be outliers. By taking an average of a few positions, the numbers will be less likely to be skewed by a particularly good/bad performance. It also mitigates the need for you to arbitrarily identify and exclude "outlier" seasons.

If you use a lower position (say average of 5th to 10th), the numbers become seriously skewed pre-expansion (in many years, there may not have been a lot of elite talent beyond the top few spots).

All of the results (current as of 2008 or whenever I last updated this) are in the link. I think they look intuitively reasonable.

The issue with that is that there are still years like 1989, when the top 4 were all outliers (but #5 wasn't).

One possible solution is to do what a poster who is no longer here suggested: take the average of the 2nd - 18th best scorers as X. 18 being selected as the number of 1st line spots available in the O6. This would appear to possibly run into the same issue 70s and HO were discussed earlier though - in some pre-expansion years, the guys farther away from the top might just not be that strong and shouldn't be used as the standard.

 

[TheDevilMadeMe]

I think it's best to separate pre-expansion and post-expansion, then try to integrate the two based on the known differences in the effective pool of players available.

It seems like a larger sample, such as 1N (N=number of teams, so keeps opportunity fixed) would be more reliable and less subject to random variation or influence.

If you try to predict how many "simple adjusted" points the top 1N players will score, based on just a handful of factors, one can generally estimate within a standard deviation of the mean (and very often much less) just about every time. The main factors which I've identified so far are the impact of non-Canadians in the top 1N (what % the top 1N would decrease without their points), the amount of parity in the league (particularly the disparity in GF), and the frequency of PP opportunities. The non-Canadian presence may itself cause ~5-10% higher adjusted points from the top 1N players. Whether the increase in points indicates it's less difficult depends on the reason for the increase.

I'm against any proposal that tries to tie the strength of talent directly to the number of teams.

 

[Hardyvan123]

I think it's best to separate pre-expansion and post-expansion, then try to integrate the two based on the known differences in the effective pool of players available.

It seems like a larger sample, such as 1N (N=number of teams, so keeps opportunity fixed) would be more reliable and less subject to random variation or influence.

If you try to predict how many "simple adjusted" points the top 1N players will score, based on just a handful of factors, one can generally estimate within a standard deviation of the mean (and very often much less) just about every time. The main factors which I've identified so far are the impact of non-Canadians in the top 1N (what % the top 1N would decrease without their points), the amount of parity in the league (particularly the disparity in GF), and the frequency of PP opportunities. The non-Canadian presence may itself cause ~5-10% higher adjusted points from the top 1N players. Whether the increase in points indicates it's less difficult depends on the reason for the increase.

I'm starting to use a vsX against Canadian players, as well as a separate one for all players in the league and maybe blend the 2, more information and different ways of looking at it may give a clearer picture. Canadians are a constant in the NHL so it's a good baseline.

 

[TheDevilMadeMe]

I'm starting to use a vsX against Canadian players, as well as a separate one for all players in the league and maybe blend the 2, more information and different ways of looking at it may give a clearer picture. Canadians are a constant in the NHL so it's a good baseline.

I would be interested to see the results were "X" were tied to, say, the 2nd or 5th best Canadian scorer in a year.

You will be happy to note, however, that the Vs2 method does at least somewhat take into account the influx of European talent. IIRC, approximately 1.5 times as many players meet certain benchmarks per year after the early 90s as do before then.

 

[matnor]

I've seen those threads and they look promising. Keep in mind that the OP of this thread originated on the All Time Draft board, so it was originally a critique of the various methods of comparing offense that are used in the ATD. For me at least, it's easier to figure out the strengths and weaknesses of a method when I've seen it applied a few times. I'm also (in case you couldn't tell) much better at critiquing methods that other people come up with than creating them myself.

I do think your methods are well worth exploring. Perhaps you'll join the ATD next year?

I'm not sure I'll have the time to participate in the ATD, but I lurk that forum quite a lot so who knows... Anyways, I'm not sure the methods I proposed in those threads are that good. As you say, people need to use them for themselves before they can judge them. I suppose I could upload some spreadsheets with data if there is any interest in it.

I had similar thoughts to you actually - if you set X at 5, you basically never have to worry about outliers, like you do if you set X at 2.

This is the conversation from the linked vs 2 thread. It's a little hard to see, but I linked to threads on adjusted even strength scoring, Vs2, VsX, and Top 10 and Top 20 points in the titles of the sections.

I would recommend reading or at least skimming the whole thread, but for your specific question, look at posts 21 until... i dunno post 33 or so.
http://hfboards.mandatory.com/showthread.php?t=1099631

I pretty much went out and said that Vs5 and Vs10 were superior to Vs2 because they are much gentler curves that aren't prone to much fluctuation between year to year. The main reason to use Vs2 seems to be this:

I still think Vs5 is probably better when comparing post-expansion players, but it could run into issues pre-expansion as 70s showed.

Your argument is along the same line as mine so I think we're in agreement here. But I can see the counter-argument that due to the limited number of top line spots in the O6-era, the "depth" of the top-10 finisher may be worse. This is tricky, I don't really have a solution...

Hockey Outsider suggested using the average of the 2nd-4th best scorer:

The issue with that is that there are still years like 1989, when the top 4 were all outliers (but #5 wasn't).

One possible solution is to do what a poster who is no longer here suggested: take the average of the 2nd - 18th best scorers as X. 18 being selected as the number of 1st line spots available in the O6. This would appear to possibly run into the same issue 70s and HO were discussed earlier though - in some pre-expansion years, the guys farther away from the top might just not be that strong and shouldn't be used as the standard.

I don't think there is any reason to take an average of a percentile method. The beauty of a percentile method is that it deals with outliers very efficiently which is why many statisticians prefer, e.g., medians over means.

 

[Hardyvan123]

I would be interested to see the results were "X" were tied to, say, the 2nd or 5th best Canadian scorer in a year.

You will be happy to note, however, that the Vs2 method does at least somewhat take into account the influx of European talent. IIRC, approximately 1.5 times as many players meet certain benchmarks per year after the early 90s as do before then.

Yes for instance in the upcoming vsX for centers I would look and compare the two models, player x versus all players (so 2nd best whatever nationality) and then player x versus the 2nd best Canuck.

I have one for Gilmour that i will post in the Gilmour versus Clarke thread, although it didn't make that much of a difference as Wayne and Mario were 1 and 2 a lot of Gilmour's earlier years.

vsX has more of an impact with non Canadian goal scorers than it does with point getters for what it's worth.

 

[BraveCanadian]

I do think your methods are well worth exploring. Perhaps you'll join the ATD next year?

I'm not sure I'll have the time to participate in the ATD, but I lurk that forum quite a lot so who knows...

I think you'd be a really valuable addition to the ATD matnor.

I thought I was going to give it up this year but I find myself waiting for it to start again.

I'm a little divided on the topic at hand.

I think something like vs5 might make sense but I also believe that something floating might make sense as well.

For example, taking the average scoring of 3 x the number of teams worth of the top scorers to get a sense of how a player performs in relation to the "average first liner".

In that sense you'd be more comparing the player to his replacement instead of his relation to the top and you'd also be taking into account the opportunities for first line ice time and powerplay time etc. It would be a slightly different perspective than we are normally working with.

 

[TheDevilMadeMe]

For example, taking the average scoring of 3 x the number of teams worth of the top scorers to get a sense of how a player performs in relation to the "average first liner".

In that sense you'd be more comparing the player to his replacement instead of his relation to the top and you'd also be taking into account the opportunities for first line ice time and powerplay time etc. It would be a slightly different perspective than we are normally working with.

It's mathematically sound and it will tell you how close a player was to an average first liner. My question though is why do we care how close he was to an average first liner, when the quality of an average first liner is going to change, mostly based on the number of teams?

 

[BraveCanadian]

It's mathematically sound and it will tell you how close a player was to an average first liner. My question though is why do we care how close he was to an average first liner, when the quality of an average first liner is going to change, mostly based on the number of teams?

I see it kind of like a measure of how much the team benefited having that player as opposed to the average.

Similar to the above replacement value that people have started using for goaltenders.

It would just be another way of looking at their production instead of strictly how close were they to the leader (however the leader mark is defined)

 

[Czech Your Math]

I'm against any proposal that tries to tie the strength of talent directly to the number of teams.

There's no attempt to tie the strength of talent to the number of teams. Measuring a group of players in proportion to the size of the league keeps opportunity essentially fixed.

I'm starting to use a vsX against Canadian players, as well as a separate one for all players in the league and maybe blend the 2, more information and different ways of looking at it may give a clearer picture. Canadians are a constant in the NHL so it's a good baseline.

I agree that using "Canadian players only" for various metrics yields useful information. I'm just not fond of using a single player as the basis for the metric, since such a metric is much more prone to random variations.

I think something like vs5 might make sense but I also believe that something floating might make sense as well.

For example, taking the average scoring of 3 x the number of teams worth of the top scorers to get a sense of how a player performs in relation to the "average first liner".

In that sense you'd be more comparing the player to his replacement instead of his relation to the top and you'd also be taking into account the opportunities for first line ice time and powerplay time etc. It would be a slightly different perspective than we are normally working with.

I think you're on the right track here.

 

[matnor]

I think you'd be a really valuable addition to the ATD matnor.

I thought I was going to give it up this year but I find myself waiting for it to start again.

Thanks, we'll see...

I'm a little divided on the topic at hand.

I think something like vs5 might make sense but I also believe that something floating might make sense as well.

For example, taking the average scoring of 3 x the number of teams worth of the top scorers to get a sense of how a player performs in relation to the "average first liner".

In that sense you'd be more comparing the player to his replacement instead of his relation to the top and you'd also be taking into account the opportunities for first line ice time and powerplay time etc. It would be a slightly different perspective than we are normally working with.

That's similar to the adjusted scoring in comparison to the top 10% I posted above. I think the numbers look reasonable. But I also think there is some truth to this:

It's mathematically sound and it will tell you how close a player was to an average first liner. My question though is why do we care how close he was to an average first liner, when the quality of an average first liner is going to change, mostly based on the number of teams?

So, I'm a bit undecided...

 

[Boom Boom Bear]

Post 1967 expansion, teams often used players from depth lines as PP specialists.

Paul Gardner would be one such example with the Penguins and earlier in his career:

http://www.hockey-reference.com/tea...call seeing, & the stats really back this up.

 

[BraveCanadian]

Thanks, we'll see...

Do it.

That's similar to the adjusted scoring in comparison to the top 10% I posted above. I think the numbers look reasonable. But I also think there is some truth to this:

Yes it is very similar and I think the results of your top 10% were pretty reasonable too.

I just think I'd be interested in looking how much the top players exceed an "average" player in the same role.

So, I'm a bit undecided...

I like it tied to the number of teams somewhat if only because it would implicitly average out PP opportunities and team strengths etc.

For example in the 80s Lemieux was getting tons of PP opportunities and Gretzky was getting relatively few.. by looking at how much they exceeded the "average" first liner you would implicitly be meeting in the middle on those types of things.

Based on your top 10% findings I think that the results would be reasonable but coming from a different perspective than we are usually focused on: ie. how much they exceed the average person with average opportunities instead of us being focused on just the top players exclusively.

I also like it because, similarly to your top 10% idea, it should work across eras without any caveats for strange outlier years where a cluster of players were way ahead of the pack (1989 anyone?) etc.

I might run through it one of these days if I ever have time just to see the results.

 

[Averick*]

I think it's best to separate pre-expansion and post-expansion, then try to integrate the two based on the known differences in the effective pool of players available.

It seems like a larger sample, such as 1N (N=number of teams, so keeps opportunity fixed) would be more reliable and less subject to random variation or influence.

If you try to predict how many "simple adjusted" points the top 1N players will score, based on just a handful of factors, one can generally estimate within a standard deviation of the mean (and very often much less) just about every time. The main factors which I've identified so far are the impact of non-Canadians in the top 1N (what % the top 1N would decrease without their points), the amount of parity in the league (particularly the disparity in GF), and the frequency of PP opportunities. The non-Canadian presence may itself cause ~5-10% higher adjusted points from the top 1N players. Whether the increase in points indicates it's less difficult depends on the reason for the increase.

The problem with scrutinizing expansion is that you have to look at it relative to the talent pool. In other world if the number of teams increases by 5% but this occurs around the same time that the talent pool expands by > 5%, this issue doesn't carry a lot of weight.

 

[HarrisonFord]

When in doubt, compare standard deviations above the mean between players.

 

[TheDevilMadeMe]

When in doubt, compare standard deviations above the mean between players.

This would greatly favor eras like the 1970s when there was a huge disparity between haves and have-nots.