Adjusted stats - how valuable?

[Plural]

How much weight do you give adjusted stats? hockey-reference.com gives pretty good summary of adjusted points/goals/assista and bunch of other statistic also.

I was browsing through there and found out that Jaromir Jagr is 4th and Teemu Selanne is 5th on adjusted goals in career and Jagr is 3rd in adjusted points in career. Only behind Howe and Gretzky.

Selanne's highest points in season ends up in 122 points. It is exactly the same as Malkin, Crosby and Ovechkin has. Ovechkin has the second highest goals in season by 72 adjusted goals right after Brett Hull's number 1 place with 78.

Are these stats more important than the actual statistics that are kept by NHL or are they just a tool to look offensive seasons in perspective?

Click to read more...

 

[revolverjgw]

They're obviously imperfect but they paint a much clearer picture than raw totals.

Look at a guy like Bossy with his bloated 80s numbers. Adjust them, line his seasons up with the predicted adjusted totals of last seasons scorers, and it turns out that, year by year, he ends up pretty much exactly in the same place in the 11-12 leaderboards as he did in his respective seasons back then, dropping a spot or two in a few years which is exactly what WOULD happen in a deeper league with more competition. The adjusted totals pretty much depict exactly the kind of player I would expect him to be today. They don't punish him too much, they don't bump him up, they put in the same relative position with totals that aren't ridiculously exaggerated.

Look at his totals unadjusted, you'd assume he was a consistently more dominant goal scorer than Stamkos was last season. He wasn't close. Neither was Denis freakin' Maruk in '82.

(similarly, go ahead and adjust Maruk's 60 goal season down and you get 44 goals, which would place him 4th last season, which is totally reasonable... he was 3rd in 82. It's predictable that in a deeper league with more stars, he'd almost certainly drop at least a spot and end up right where the adjusted number puts him)

This is just a very simple analysis, there's other variables that could swing results either way. But as a rough guide and a quick reference, adjusted stats are pretty good, way better than not adjusting them.

 

[Sureves]

They're obviously imperfect but they paint a much clearer picture than raw totals.

Look at a guy like Bossy with his bloated 80s numbers. Adjust them, line his seasons up with the predicted adjusted totals of last seasons scorers, and it turns out that, year by year, he ends up pretty much exactly in the same place in the 11-12 leaderboards as he did in his respective seasons back then, dropping a spot or two in a few years which is exactly what WOULD happen in a deeper league with more competition. The adjusted totals pretty much depict exactly the kind of player I would expect him to be today. They don't punish him too much, they don't bump him up, they put in the same relative position with totals that aren't ridiculously exaggerated.

Look at his totals unadjusted, you'd assume he was a consistently more dominant goal scorer than Stamkos was last season. He wasn't close. Neither was Denis freakin' Maruk in '82.

(similarly, go ahead adjust Maruk's 65 goal season down and you get 44 goals, which would place him 4th last season, which is totally reasonable... he was 3rd in 82. It's predictable that in a deeper league with more stars, he'd almost certainly drop at least a spot and end up right where the adjusted number puts him)

Pretty much this. Adjusted stats are very good, but not perfect.

 

[Plural]

They're obviously imperfect but they paint a much clearer picture than raw totals.

Look at a guy like Bossy with his bloated 80s numbers. Adjust them, line his seasons up with the predicted adjusted totals of last seasons scorers, and it turns out that, year by year, he ends up pretty much exactly in the same place in the 11-12 leaderboards as he did in his respective seasons back then, dropping a spot or two in a few years which is exactly what WOULD happen in a deeper league with more competition. The adjusted totals pretty much depict exactly the kind of player I would expect him to be today. They don't punish him too much, they don't bump him up, they put in the same relative position with totals that aren't ridiculously exaggerated.

Look at his totals unadjusted, you'd assume he was a consistently more dominant goal scorer than Stamkos was last season. He wasn't close. Neither was Denis freakin' Maruk in '82.

(similarly, go ahead adjust Maruk's 65 goal season down and you get 44 goals, which would place him 4th last season, which is totally reasonable... he was 3rd in 82. It's predictable that in a deeper league with more stars, he'd almost certainly drop at least a spot and end up right where the adjusted number puts him)

Thanks. Good post.

Kind of makes me wonder tough, if adjusted stats put Ovechkin's 65 goal campaign at the second best all-time. Was it actually as good as they imply? I was born in the late 80's so i did not get to see the "run & gun" hockey at that decade. I only see the numbers and read post's in the HoF section.

Or, i am a BIG BIG fan of Selanne, but since OV and Crosby broke in to the league i have felt like they are offensively much superior to Selanne is his prime. Adjusted stats imply different. It actually puts him in the exact same amount of points, which is 122, with Malkin, Crosby and OV. Somehow i feel that it is hard to think of him as a offensive equal to those three.

 

[begbeee]

Ovechkin's 65' season is definitely one of the most impressive season in history (individually speaking).
Maruk's 60 goals campaign is not comparable by any stretch and raw stats dont tell you this. Just adjusted.

Adjusted stats aren't perfect and you can't take off some goals from player who actually physically scored 60 goals - he put rubber 60 times behind a goalie, but that doesnt mean 60 in 80's were same as the 60 in 2006.

 

[Dalton]

I am not a fan of adjusted stats.

First of all. A guy scores x amount of goals over a certain time frame in a specific set of circumstances. It is pure speculation rife with questionable assumptions to assume that he would score y amount of goals in a different time frame or set of circumstances.

I think it's pub math. Fun for debating but absolutley unreliable.

Second of all we have a very recent, very large, comprehensive and unchallenged study that included hockey players among other sports and many other professions that clearly demonstrated that arithmetic means are not accurate for assessing human performance.

The authors showed that in fact we can expect a distribution of perfomance among a group to roughly follow an 80-20 rule. That is (roughly) 20% of the performers are responsible for (roughly) 80% of the output. Within that elite group we would see a similar distribution. I suspect this is intuitively obvious to many.

They showed that the elite were hugely influential on the mean.
This means that a mean showing a particular season or era being high scoring or low scoring is largely due to the outliers. It does not mean that it was easier or more difficult to score. To prove this you would have to look at the worst goalies and ascertain that it wasn't simply a case of poorer goaltending at the bottom end and also look at the highest scorers to assure that there wasn't a surge of scoring at that end. Perhaps both events were in play.

I proposed looking at the percentage of the league totals that these top and bottom performers achieved and comparing them to other seasons. This type of analysis should reveal the impact of the extreme ends of goal tending and scoring. If in fact the top scorers and bottom goalies percentages of league totals are an aberration compared to other seasons then we must conclude that the season was not easier for scoring goals per se. But that maybe the bottom goalies were especially bad or the top scorers were especially good by comparison to other seasons.

If the percentages of scores and goals allowed are consistent with other seasons then we can conclude that scoring was easier overall that season or era.

I looked at last season and I did see that the top 10 and 20% were responsible for much more points than the rest of the league which supports the authors conclusions in one case. I added what I felt was a typical Gretzky season and found a bigger difference even though the rest of the players results were unchanged. One super elite guy can make a huge difference. Two? What about a really bad goalie or two?

BTW one can also apply this looking at the poorest performers.

Dump the means. There is absolute proof that they do not give an accurate picture. I used fractions but perhaps someone else can come up with something simpler.

I have also suggested but not tested the notion of using the percentage resulting from a ppg or similar benign result as a baseline to compare seasons.

What does it mean if the percentage of pts in a given season that ppg represents is higher or lower than another season's ppg percentage? What would be a good standard for goalies? 3 gaa reresenting a goal per period?

In conclusion not only do I disagree with adjusted points ( if they use arithmetic means) I am backed by a huge study that proves they are questionable.

Read the study for yourself.

Here's a write up if you prefer.

This isn't the Copernican revolution but it is a sea change for the old guard. It isn't Gauss publishing his work on non-Euclidean geometry but it does have an impact on some very hard working people that deserve respect.

 

[BraveCanadian]

I am not a fan of adjusted stats.

First of all. A guy scores x amount of goals over a certain time frame in a specific set of circumstances. It is pure speculation rife with questionable assumptions to assume that he would score y amount of goals in a different time frame or set of circumstances.

I think it's pub math. Fun for debating but absolutley unreliable.

Second of all we have a very recent, very large, comprehensive and unchallenged study that included hockey players among other sports and many other professions that clearly demonstrated that arithmetic means are not accurate for assessing human performance.

The authors showed that in fact we can expect a distribution of perfomance among a group to roughly follow an 80-20 rule. That is (roughly) 20% of the performers are responsible for (roughly) 80% of the output. Within that elite group we would see a similar distribution. I suspect this is intuitively obvious to many.

They showed that the elite were hugely influential on the mean.
This means that a mean showing a particular season or era being high scoring or low scoring is largely due to the outliers. It does not mean that it was easier or more difficult to score. To prove this you would have to look at the worst goalies and ascertain that it wasn't simply a case of poorer goaltending at the bottom end and also look at the highest scorers to assure that there wasn't a surge of scoring at that end. Perhaps both events were in play.

I proposed looking at the percentage of the league totals that these top and bottom performers achieved and comparing them to other seasons. This type of analysis should reveal the impact of the extreme ends of goal tending and scoring. If in fact the top scorers and bottom goalies percentages of league totals are an aberration compared to other seasons then we must conclude that the season was not easier for scoring goals per se. But that maybe the bottom goalies were especially bad or the top scorers were especially good by comparison to other seasons.

If the percentages of scores and goals allowed are consistent with other seasons then we can conclude that scoring was easier overall that season or era.

I looked at last season and I did see that the top 10 and 20% were responsible for much more points than the rest of the league which supports the authors conclusions in one case. I added what I felt was a typical Gretzky season and found a bigger difference even though the rest of the players results were unchanged. One super elite guy can make a huge difference. Two? What about a really bad goalie or two?

BTW one can also apply this looking at the poorest performers.

Dump the means. There is absolute proof that they do not give an accurate picture. I used fractions but perhaps someone else can come up with something simpler.

I have also suggested but not tested the notion of using the percentage resulting from a ppg or similar benign result as a baseline to compare seasons.

What does it mean if the percentage of pts in a given season that ppg represents is higher or lower than another season's ppg percentage? What would be a good standard for goalies? 3 gaa reresenting a goal per period?

In conclusion not only do I disagree with adjusted points ( if they use arithmetic means) I am backed by a huge study that proves they are questionable.

Read the study for yourself.

Here's a write up if you prefer.

This isn't the Copernican revolution but it is a sea change for the old guard. It isn't Gauss publishing his work on non-Euclidean geometry but it does have an impact on some very hard working people that deserve respect.

Count me in the crowd that has been suspicious for a while of how well averages apply in adjustments to outliers.

I still think that it is helpful to keep in mind how the scoring level has changed over time but the top level players that we are generally concerned with around here are not average.

 

[Master_Of_Districts]

I am not a fan of adjusted stats.
...

You're wrong.

 

[Bear of Bad News]

You're wrong.

Care to elaborate? It seems like a post as thought-out as that one deserves more than a two-word response.

 

[TheDevilMadeMe]

The biggest issue with traditional adjusted stats is that they assume that the distribution of offense has remained fixed over time, and just the amount of total offense has changed. And this had been demonstrated as false.

CYM did a great job on the history board showing that scoring from first line players actually didn't increase from the late 70s into the 80s - the offensive explosion came almost entirely from lower lines scoring more.

Likewise, it's been known for some time that "dead puck era" first liners scored a higher percentage of offense than 80s first liners. Two reasons have been given:

1) A larger percentage of offense during the dead puck era and beyond comes from powerplays and first liners get most of the powerplay time. Overpass proved this to be true statistically.

2) Much of the increase in scoring in the 80s came from bad D and goaltending, and such goals would be distributed to all players much more equally than "skill goals.". Meaning the less skilled players would have a higher percentage of their offense taken away by the better D and goaltending that would later develop. This one has not been proven statistically, bit intuitively seems true

 

[wgknestrick]

The biggest issue with traditional adjusted stats is that they assume that the distribution of offense has remained fixed over time, and just the amount of total offense has changed. And this had been demonstrated as false.

CYM did a great job on the history board showing that scoring from first line players actually didn't increase from the late 70s into the 80s - the offensive explosion came almost entirely from lower lines scoring more.

Likewise, it's been known for some time that "dead puck era" first liners scored a higher percentage of offense than 80s first liners. Two reasons have been given:

1) A larger percentage of offense during the dead puck era and beyond comes from powerplays and first liners get most of the powerplay time. Overpass proved this to be true statistically.

2) Much of the increase in scoring in the 80s came from bad D and goaltending, and such goals would be distributed to all players much more equally than "skill goals.". Meaning the less skilled players would have a higher percentage of their offense taken away by the better D and goaltending that would later develop. This one has not been proven statistically, bit intuitively seems true

I totally agree. The key is adjusting for the proper things. Total league scoring is an end result and should not be adjusted for IMO. The inputs to that "result" should be adjusted for. Total league scoring just indicates something is different, but doesn't tell you exactly what that is or by how much. I see 3 main variables that change year to year (and there maybe more):


5v4 ice time ratio to 5v5 (primary factor IMO) - captures changes in officiating

5v5 league average SV% (secondary) - captures goaltender population skill

5v3 ice time ratio to 5v5 (tertiary) - captures changes in officiating


These would be the items that I would adjusting player totals to to try to capture all the real inputs. Unfortunately I don't think they are available for all years.

 

[seventieslord]

I think it's pub math. Fun for debating but absolutley unreliable.

And if it is, so what? Have you done better? Can we see it? And that study you posted still has no relevance to adjusted stats.

 

[Dalton]

And if it is, so what? Have you done better? Can we see it? And that study you posted still has no relevance to adjusted stats.

I need data I can use on excel compatible software.

I've repeatedly requested for info on how to get this data. Either some site has it availabe for dl or some member could elucidate for all of us how they aquired it.

I have suggested that this site could be a source of the data. I think these discussions and indeed this whole notion of discussing the analysis of stats will be short lived, frustrating and futile if posters with ideas can't access basic data to test their ideas.

As for the study- you keep saying that it doesn't apply and have yet to give a detailed reason why. OTOH I've posted an analysis of last season showing that it did apply.

If I did this correctly -The top 10% of players (89 of 890 players) were responsible for 5,830 of the 17,631 points generated in the NHL last year. The top 20% earned 9,762. 30%= 12,166. I feel pretty confident that these numbers support that study's conclusions that a few performers outperform all the rest.

The authors showed that this means arithmetic means would not give an accurate picture of overall performance of the whole group. Using many stats including hockey goals and penalties they demonstrated that the error was many magnitudes higher than if a power curve were used instead of a bell curve. Of course I would feel better if all the seasons were looked at.

Show me some data to prove the study is wrong. Are my calculations in error? My data?

 

[TheDevilMadeMe]

I need data I can use on excel compatible software.

I've repeatedly requested for info on how to get this data. Either some site has it availabe for dl or some member could elucidate for all of us how they aquired it.

I have suggested that this site could be a source of the data. I think these discussions and indeed this whole notion of discussing the analysis of stats will be short lived, frustrating and futile if posters with ideas can't access basic data to test their ideas.

Exactly what data are you looking for? Maybe someone could help if you were more specific

As for the study- you keep saying that it doesn't apply and have yet to give a detailed reason why. OTOH I've posted an analysis of last season showing that it did apply.

If I did this correctly -The top 10% of players (89 of 890 players) were responsible for 5,830 of the 17,631 points generated in the NHL last year. The top 20% earned 9,762. 30%= 12,166. I feel pretty confident that these numbers support that study's conclusions that a few performers outperform all the rest.

Top 10% of forwards or players overall? Because forwards only make up a little more than half the league.

Edit: and forwards who get prime powerplay time make up far less than that. If you want to show that a small number of outliers are responsible for a large % of offense, I would look at even strength only, and possibly add PP points per PP TOI.

The authors showed that this means arithmetic means would not give an accurate picture of overall performance of the whole group. Using many stats including hockey goals and penalties they demonstrated that the error was many magnitudes higher than if a power curve were used instead of a bell curve. Of course I would feel better if all the seasons were looked at.

The issue with correlating raw goals, assists, and points to performance is that this isn't basketball. Players aren't shooting at an empty net - there are players on the other side with the job of stopping goals. A higher defensive performance or goaltending performance will reduce the number of scoring instances, even when the scorer is performing at the same level. Adjusted stats, when used properly (and they aren't always), are an attempt to "even the playing field" between eras where the balance was tilted more towards offense or towards defense.

Show me some data to prove the study is wrong. Are my calculations in error? My data?

The study could be "correct" and still not apply the way you think it does to adjusted stats. I still haven't been convinced it applies to the vast majority of players. Okay, maybe there are issues with mathematically adjusting the stats of an outlier like Gretzky, but why can't you do it with the majority of players?

 

[seventieslord]

Okay, maybe there are issues with mathematically adjusting the stats of an outlier like Gretzky, but why can't you do it with the majority of players?

that's exactly it.

this keeps going back to "the study" how "it's not actually a bell curve, it looks more like this"... and my answer to that is, "so? what does that matter when talking about adjusted stats?"

 

[Dalton]

I'm looking for basic stats. Goals. assists, points, goals against etc.
The data I used was dl'd from TSN.com. It was everybody from last season AFAIK.

I want to look at the most basic data at first. Then I think it would be interesting to look at data to respond to questions that arise. KISS.

I might contribute to the attempt to even the playing field by using a method not dependant on means. How or if I can won't be apparent until the data is analyzed. As I said I need data. Until then it's just MHO.

Does that study apply the way I think? I really can't know until I have data, analyze it and put it up for criticism. I haven't really seen a definitive argument to persuade me it doesn't.

Does that study apply to adjusted stats? If they use arithmetic means it may well apply, absolutely. It cannot be disregarded without specific arguments supporting that course. Otherwise one is just falling into the trap that the study is debunking. I just looked at one season and found evidence that means don't apply.

Lastly I wish to repeat that this isn't number 42. I think it's just a way of looking at the numbers without changing the data or the data set. I just can't help feeling suspicious of any analysis that starts off by adding, changing or eliminating data. I feel this study re-inforces that opinion since it works on raw data. It repudiates dropping or adjusting outliers at both ends.

IMHO the best result of looking at the data this way would be interesting questions not necessarily answers. And I have some fun too.

 

[TheDevilMadeMe]

I'm looking for basic stats. Goals. assists, points, goals against etc.
The data I used was dl'd from TSN.com. It was everybody from last season AFAIK.

I want to look at the most basic data at first. Then I think it would be interesting to look at data to respond to questions that arise. KISS.

If you're just looking for basic stats, hockey-reference.com has them for all NHL seasons

 

[Micklebot]

Top 10% of forwards or players overall? Because forwards only make up a little more than half the league.

Edit: and forwards who get prime powerplay time make up far less than that. If you want to show that a small number of outliers are responsible for a large % of offense, I would look at even strength only, and possibly add PP points per PP TOI.

597 forwards played at least a game vs 297 defencemen; that's roughly 67% of all skaters being forwards. These #s are inline with teams icing 12 forwards and 6 d as well. I'd say that is substantially more then half. I do agree that you should probably break down the study by position, or at least include that as well to get a better idea.

I would think forwards getting prime PP time would probably be either half, or a quarter, depending on how you define prime,

 

[TheDevilMadeMe]

597 forwards played at least a game vs 297 defencemen; that's roughly 67% of all skaters being forwards. These #s are inline with teams icing 12 forwards and 6 d as well. I'd say that is substantially more then half. I do agree that you should probably break down the study by position, or at least include that as well to get a better idea.

I was assuming he was including goalies since he said "players."

I'm also not interested in players who played "at least a game," as much as a minimum number of games

Either way, you're right, substantially more than half the players are forwards.

I would think forwards getting prime PP time would probably be either half, or a quarter, depending on how you define prime,

I think it would be somewhere between 1/2 and 1/4. Some teams give their first unit most of the time; others have a more even split

 

[Filatov2Kovalev2Bonk]

They're interesting from a normalization standpoint, to create comparison points across eras, but ultimately don't mean much.

Not to put too fine a point on it, but if someone were to see 99 in a social event and told him he'd actually scored 162 points (or whatever) instead of 200+ becaue adjusted stats for total scoring and so on said as much...he'd probably laugh and buy you a beer or something.

They're great for comparitive purposes but not so much reality-wise. The game sheets tell you who scored however many points in a game. If you're going to subtract or add, then you'll need to find a game where that happened and then adjust goalie stats, +/-, PP and SH stats...seems dicey.

 

[TheDevilMadeMe]

They're interesting from a normalization standpoint, to create comparison points across eras, but ultimately don't mean much.

Not to put too fine a point on it, but if someone were to see 99 in a social event and told him he'd actually scored 162 points (or whatever) instead of 200+ becaue adjusted stats for total scoring and so on said as much...he'd probably laugh and buy you a beer or something.

They're great for comparitive purposes but not so much reality-wise. The game sheets tell you who scored however many points in a game. If you're going to subtract or add, then you'll need to find a game where that happened and then adjust goalie stats, +/-, PP and SH stats...seems dicey.

The point of adjusted stats is to compare players between eras. I don't think I've seem anyone use them for anything else; have you?

 

[Bear of Bad News]

Not to put too fine a point on it, but if someone were to see 99 in a social event and told him he'd actually scored 162 points (or whatever) instead of 200+ becaue adjusted stats for total scoring and so on said as much...he'd probably laugh and buy you a beer or something.

Has anyone suggested that there's anyone in the world who actually does this?

EDIT: TDMM beats me to the punch again.

 

[Czech Your Math]

I'm looking for basic stats. Goals. assists, points, goals against etc.

BehindTheNet.com has total league data in Excel format for the past 40+ seasons.

 

[Czech Your Math]

They're interesting from a normalization standpoint, to create comparison points across eras, but ultimately don't mean much.

Not to put too fine a point on it, but if someone were to see 99 in a social event and told him he'd actually scored 162 points (or whatever) instead of 200+ becaue adjusted stats for total scoring and so on said as much...he'd probably laugh and buy you a beer or something.

They're great for comparitive purposes but not so much reality-wise. The game sheets tell you who scored however many points in a game. If you're going to subtract or add, then you'll need to find a game where that happened and then adjust goalie stats, +/-, PP and SH stats...seems dicey.

They actually mean a great deal: They tell you how valuable goals & points were in that season, based on the average number of goals scored per game.

If you told Gretzky that 200 points in a 7.5 gpg environment were roughly as valuable as 160 points in a 6.0 gpg environment, he'd probably think about it and agree (assuming he has basic math skills).

 

[Czech Your Math]

First of all. A guy scores x amount of goals over a certain time frame in a specific set of circumstances. It is pure speculation rife with questionable assumptions to assume that he would score y amount of goals in a different time frame or set of circumstances.

There's no way of determining exactly how many goals/points a player would score in a different season. However, adjusted stats put the data in the context of value. 50 goals in a 7.5 gpg environment has equivalent value to 40 goals in a 6.0 gpg environment. You say KISS, and it's really that simple.

I think it's pub math. Fun for debating but absolutley unreliable.

The principle is very simple, whether or not you choose to acknowledge it.

I looked at last season and I did see that the top 10 and 20% were responsible for much more points than the rest of the league which supports the authors conclusions in one case. I added what I felt was a typical Gretzky season and found a bigger difference even though the rest of the players results were unchanged. One super elite guy can make a huge difference. Two? What about a really bad goalie or two?

Did you account for the fact that there are 2.7-2.75 points awarded per goal? To say a player was involved in X% of the goals is not the same as saying that he was responsible for X% of the goals.

One or two extreme outliers are not going to make a huge difference in a 20+ team league. Yes, outliers (whether players or teams) do affect the mean. However, there have been much larger fluctuations in league gpg than can be attributed to one or two players, or even one or two teams.