Adjusted stats- stop using them

[rallymaster19]

I’m all for comparing different players from different eras and across different positions. It’s interesting ranking Potvin against Coffey or Clarke against Lindros or of course Yzerman and Messier. But I don’t understand how “adjusted stats†are now commonly used when comparing players. Of course scoring a goal in 2003 was a lot harder to do than in 1983. The size of goaltender equipment is one reason for this. Physically larger players reducing free space on the ice surface and the dreaded trap are others.

But I now often see adjusted stats used to sway arguments one way or the other. For example, Sakic’s Hart season in 2001 was equivalent to or better than Yzerman’s 1989 season when “adjusted†for era because both register about 150-160 points for a 1980s base season. If you disregard all the other factors like PK time, quality of teammates, defensive analysis etc. (I’m not going to compare the two players myself, at least not today) and just look purely at the numbers, when adjusted they look about the same. But for anyone who actually watched hockey both those seasons, you cannot tell me 118 points in 2001 automatically equals 155 points in 1989.

People seem to believe the following:

Adjusted stats = A substitute for: (Actual stats + Observation)

This is not the case. There is NO mathematical formula that can accurately compare players’ points totals across different eras. I know there are several different methods people use to calculate adjusted stats. The one I hate/despise the most is also the simplest and goes like this:

Goals per game in 1989: 7.37
Goals per game in 2001: 5.51

Yzerman’s point total 1989: 155
Sakic’s point total 2001: 118

Therefore GPG (1989)/GPG (2001) * Sakic’s points total 2001= Sakic’s adjust points total 1989

7.37/5.51*118= 157.8 or 158 points

Therefore Sakic’s best year > Yzerman’s best year and the numbers prove it.


No, no and no. PLEASE STOP DOING THIS. There are too many variables that are unaccounted for when producing adjusted stats but regardless of how the different types of adjusted stats are calculated, there seems to be several important universal problems:
1. Adjusted stats always favour more recent times
These formulas overcompensate for the reduction in scoring in recent years. Malkin had a great season last year but no way in hell does his 113 points compare to Yzerman’s ’89 or Sakic’s ’01 seasons as adjusted stats would like you to believe. And Mark Recchi’s adjusted career numbers now look better than Esposito’s and Dionne’s. Really? Marcel Dionne has more than 300 actual points than Recchi but when adjusted Recchi’s got him beat. Really?
2. The quality of players is not accounted for
Certain eras simply have more depth in players than others. And staying with the 1989 and 2001 seasons as benchmarks, the top 10 scorers in 1989 were: Lemieux, Gretzky, Yzerman, Nicholls, Brown, Coffey, Mullen, Kurri, Carson and Robitaille. The 10 scorers in 2001 were: Jagr, Sakic, Elias, Allison, Kovalev, Straka, Bure, Weight, Forsberg and Palffy. I count 7 HHOFers/future HHOFers to 4 (which includes Bure). If you take out leeches Nicholls and Brown, the count becomes 9 to 4 with the addition of Hawerchuk and Messier in 1989. None of this discrepancy in depth is accounted for in adjusted stats and therefore Sakic racking up the points against inferior competition outweighs Yzerman finishing third amongst some of the best forwards to ever play the game.
3. The quantity of players is not accounted for
Expansion from a 21-league team in the 80s to a 30-team league and the repercussions of those decisions are not accounted for in adjusted statistics. The league is watered down to the point that about 40-45% of current players in the league today would not have made an NHL roster twenty years ago. As the Devils and almost every team has learned in the last 15 years, it is a lot easier to teach players to play defense than it is to score goals. That is why otherwise would-be AHLers are playing in the NHL and this has caused significant erosion in the talent of players today.


In the end, we all saw what we saw. When comparing guys, mention the raw stats with everything else (defensive game, special teams, trophies, cups etc.) and let people decide themselves how to compare across different times. It’s almost like a polygraph or lie detector test. If a suspect consents to it, the police can administer the test but it’s never admissible in court. Look at it on your own but dn’t bring up these highly questionable “adjusted stats†when ranking players in debates.

Click to read more...

 

[Dark Shadows]

I think you are Wayyyyy over judging the "Adjusted stats". Few of the History crew takes adjusted stats seriously as if they were stone cold fact, nor do we think they are very accurate. Its simply a tool used to help educate people who are silly enough to come in here and say "How can you compare Gordie Howe to Crosby or Ovechkin. I mean, Howe only scored 95 points once. Crosby/Ovechkin scored over a hundred multiple times, so they are much better!"

Get it? A tool to just help people "Bridge the gap" and realize that Howe's 95 points was much more than just a face value statistic that can be compared to modern players.

 

[Bear of Bad News]

rallymaster19, you are highly oversimplifying things (and you're adding in a strawman when you say "and the numbers prove it" above).

No one who knows what they are doing would ever say that, and no one who knows what they are doing would rely solely on adjusted statistics for anything.

If you would stop attributing falsities to these people, then perhaps what we do would make more sense.

 

[overpass]

1. Adjusted stats always favour more recent times
These formulas overcompensate for the reduction in scoring in recent years. Malkin had a great season last year but no way in hell does his 113 points compare to Yzerman’s ’89 or Sakic’s ’01 seasons as adjusted stats would like you to believe. And Mark Recchi’s adjusted career numbers now look better than Esposito’s and Dionne’s. Really? Marcel Dionne has more than 300 actual points than Recchi but when adjusted Recchi’s got him beat. Really?
2. The quality of players is not accounted for
Certain eras simply have more depth in players than others. And staying with the 1989 and 2001 seasons as benchmarks, the top 10 scorers in 1989 were: Lemieux, Gretzky, Yzerman, Nicholls, Brown, Coffey, Mullen, Kurri, Carson and Robitaille. The 10 scorers in 2001 were: Jagr, Sakic, Elias, Allison, Kovalev, Straka, Bure, Weight, Forsberg and Palffy. I count 7 HHOFers/future HHOFers to 4 (which includes Bure). If you take out leeches Nicholls and Brown, the count becomes 9 to 4 with the addition of Hawerchuk and Messier in 1989. None of this discrepancy in depth is accounted for in adjusted stats and therefore Sakic racking up the points against inferior competition outweighs Yzerman finishing third amongst some of the best forwards to ever play the game.
3. The quantity of players is not accounted for
Expansion from a 21-league team in the 80s to a 30-team league and the repercussions of those decisions are not accounted for in adjusted statistics. The league is watered down to the point that about 40-45% of current players in the league today would not have made an NHL roster twenty years ago. As the Devils and almost every team has learned in the last 15 years, it is a lot easier to teach players to play defense than it is to score goals. That is why otherwise would-be AHLers are playing in the NHL and this has caused significant erosion in the talent of players today.


In the end, we all saw what we saw. When comparing guys, mention the raw stats with everything else (defensive game, special teams, trophies, cups etc.) and let people decide themselves how to compare across different times. It’s almost like a polygraph or lie detector test. If a suspect consents to it, the police can administer the test but it’s never admissible in court. Look at it on your own but don’t bring up these highly questionable “adjusted stats” when ranking players in debates.

The method of adjusting stats that you attack simply keeps the win value of each goal and assist constant across different scoring levels. When there are 7 or 8 goals per win in the 1980s, it's fairly obvious that each individual goal is not as valuable as in recent years when there are 5 or 6 goals per win. In your Sakic and Yzerman example, Sakic's points very likely contributed to wins as much or more than Yzerman's.

It's true that not all eras are created equal in average player talent. But if you expect adjusted stats to adjust for this, you are asking too much from them. They adjust for win value, and I would argue that makes them better than unadjusted stats, if not perfect. If you think one era had more talent on average than another era, add your own mental adjustment.

On the other hand, people that use adjusted stats should be aware that they don't adjust for everything. Average talent in the league is probably the biggest thing they don't adjust for.

Finally, don't underrate the influx of European and American talent that entered the league in the 1980s and 1990s in terms of the average talent level of the league.

 

[Ohashi_Jouzu*]

Wow. Sounds like a great big rant directed at me, since I'm the only one to have brought up adjusted stats for Yzerman's 89 and Sakic's 01 in the other thread. Funny thing is, my point was that DESPITE being similar after "adjustment", I still think Yzerman's season was more impressive to watch than Sakic's... which he seems to agree with.

 

[Crosbyfan]

Simply put, adjusted stats are a better starting point for comparison.

 

[rallymaster19]

This post wasn’t directed at anyone specifically. Though I did post this after reading some of the comments on that Yzerman vs. Sakic thread, this really wasn’t about either player or who was better (although one day I will post my opinion on the topic; probably in another two months when we’re due for the next Yzerman vs Sakic thread). I’ve just noticed a trend more recently where when two players are almost evenly matched up against each other in all other areas, the deciding variable ends up being adjusted points or some sort of adjusted measure. I don’t agree with that being an appropriate choice. That’s where I was going with this. Hopefully, everyone can put an “in my opinion†in front of everything I said and it will sound less and less like a rant, and more and more like an actual topic of discussion.

 

[MS]

They're a tool and like any tool over-using or taking them as concrete fact is going to be fallacy.

However, there has to be some sort of adjustment to recognize that Stephan Lebeau's 1992-93 seasons is not better than Jarome Iginla's 2003-04 season, no matter how it looks on paper. Or that Paul Kariya's performance from 1995-2000 is not equatable to Bernie Nicholls' production from 1983-88.

 

[NOTENOUGHJTCGOALS]

I agree that stats are never a substitute for watching the games and forming your own opinion about a player.

However your 3 points could also be applied to non-adjusted stats. Bottom line: take every stat with a grain of salt.

There is a certain poster here to gets caught up in every flavour of the month ranking method. First it was raw stats, then he started believing adjusted stats were the only true measure of a players worth, then "point leeches", then Hart trophy shares. If you take his posts as a whole he could be arguing against himself from one month to the next next. There is no one statistical measure to use to prove which player was better. Adjusted stats are a good tool.

 

[canucks4ever]

It would be impossible for Joe Sakic to score 158 points and remain defensively responsible. In his second best season, which was 1996, Sakic put up 120 points. So playing in a high scoring season didn't make him superman.

 

[overpass]

It would be impossible for Joe Sakic to score 158 points and remain defensively responsible. In his second best season, which was 1996, Sakic put up 120 points. So playing in a high scoring season didn't make him superman.

Why should it be impossible for Sakic to do this? Look at it this way. Goalies in 1989 had an average save percentage of 0.879. In 2001 the average save percentage was 0.903, so shots went in 25% more often in 1989. If Sakic and his linemates play exactly the same way, take the same shots, but score 25% more often, Sakic has 147 points without changing his style of play at all.

That's not to say that he absolutely would have scored 147 points, but it shows the difference in eras to some degree. All points are not equally easy to get across eras.

 

[Ohashi_Jouzu*]

Why should it be impossible for Sakic to do this? Look at it this way. Goalies in 1989 had an average save percentage of 0.879. In 2001 the average save percentage was 0.903, so shots went in 25% more often in 1989. If Sakic and his linemates play exactly the same way, take the same shots, but score 25% more often, Sakic has 147 points without changing his style of play at all.

That's not to say that he absolutely would have scored 147 points, but it shows the difference in eras to some degree. All points are not equally easy to get across eras.

That's another thing about adjustments. In this case, it's the save percentage adjustment. You'd almost have to calculate the combined save percentages of the goalies he faced in one year, versus the save percent of those he faced in the other. 'Cause chances are there are a LOT of goalies dragging that average down that either player may never have faced. Especially since top teams tend to face the other team's top goalies almost exclusively (unless the backup has some miraculous track record against an opponent, there's an injury, or something else out of the ordinary).

 

[overpass]

That's another thing about adjustments. In this case, it's the save percentage adjustment. You'd almost have to calculate the combined save percentages of the goalies he faced in one year, versus the save percent of those he faced in the other. 'Cause chances are there are a LOT of goalies dragging that average down that either player may never have faced. Especially since top teams tend to face the other team's top goalies almost exclusively (unless the backup has some miraculous track record against an opponent, there's an injury, or something else out of the ordinary).

A perfect adjustment would do that, although I doubt it would give a very different result. Your last sentence seems to suggest that Sakic, playing on the President's trophy Avalanche, would have faced tougher goalies than the league average (although on the other hand he never went against Roy).

Really, any perfect adjustment would include a strength of schedule adjustment for opponents faced, but 95% of the time that's too much work for a relatively small gain in accuracy. A small league like the original 6 or an unbalanced league like the NHL immediately post-expansion could benefit from this adjustment, as could a comparison of a starting goaltender vs a backup goaltender.

 

[Granlund2Pulkkinen*]

You're so right.

I mean who says Joe Malone couldn't have 44 goals and 4 assists for 48 points in 20 games this season?

 

[Ohashi_Jouzu*]

A perfect adjustment would do that, although I doubt it would give a very different result. Your last sentence seems to suggest that Sakic, playing on the President's trophy Avalanche, would have faced tougher goalies than the league average (although on the other hand he never went against Roy).

Yeah, that highlights some of the potential difficulties, however I imagine the Red Wings also faced decent goalies for a long time, finishing 1st in the West something like 6 times (1st in their division something like a dozen times) during Yzerman's career. However...

Really, any perfect adjustment would include a strength of schedule adjustment for opponents faced, but 95% of the time that's too much work for a relatively small gain in accuracy. A small league like the original 6 or an unbalanced league like the NHL immediately post-expansion could benefit from this adjustment, as could a comparison of a starting goaltender vs a backup goaltender.

...I don't think we have to go for perfection here. I assume if people are coming up with league averages for things like save percentage, a spreadsheet is involved (can't recall having ever seen a league sv pct stat otherwise readily available, and the hockeyreference season finder sorts, but doesn't crunch the numbers). I'd just like to see the "average" taken from just the #1 goalies in any year. I expect (with no evidence to back this up, of course), that instead of the 25% difference mentioned earlier, it may be as low as 10 or 15%. Just a cursory look at the top 3 #1 guys in each of the years in question:

(2008/09: Thomas 0.933, Vokoun 0.926, Backstrom 0.923)
2000/01: Dunham: 0.923, Burke 0.922, Hasek 0.921
1995/96: Hasek 0.920, Puppa 0.918, Hebert 0.914
1988/89: Roy 0.908, Casey 0.900, Vernon 0.897

suggests that among the goalies that either of these guys actually faced, the difference is likely smaller than 25% by a (meaningful?) margin. About 7 of potentially 21 (I didn't verify that far) #1 guys in 88/89 were below that average of 0.879 mentioned. I count about 10/11 (of 30?) below the 0.903 average for 2000/01. I guess proportionally that's about the same.

So yeah, not calling for perfection. On the negative side, though, it becomes very subjective/potentially inaccurate erasing of names from a spreadsheet unless all the game matchups are verified. I'm just curious as to how/if those sv pct numbers change. Although, I guess all the scorers factor into the scoring numbers (be they goons or superstars), and strength of schedule is still omitted, so maybe it does all work out in the end.

edit: dunno if this will work, but I'll try linking these in...

88/89, goalies with >=40 games played here

00/01, goalies with >=41 games played here

Hehe, the difference may be bigger than I thought.

 

[Lososaurus]

Stats do need to be adjusted. If backchecking, defense and goaltending existed in the 80s, nobody would have scored over 120 points.

 

[Canadiens1958]

Adjusted Stats

Adjusted stats may be a tool to explain across eras if sufficient knowledge is lacking BUT they are definitely not a solid base for comparisons across eras.

BTW some of the weaknesses in this thread are fascinating and startling.

 

[RabbinsDuck]

Absolutely highlights why adjusted stats should not be taken at face value.

Responses state well why they are still a valuable tool -- I'll also use them frequently, but always give more credit to the one who actually accomplished it.

 

[Pear Juice]

It's a matter of information density. If I say that Player A is clearly better than Player B as he has more adjusted points, this statement does not contain very much information. I have analyzed one of countless of variables contributing to the overall performances of both players.

This is true for all observations, be it points, adjusted points, individual trophies, stanley cups, on-site observation, training reports etc. Even the ever-argued +/- statistic.

Watching a player perform obviously carries more information than reading the scoresheet, but it's not all black and white. None of these variables carries all the information, but like it or not, they all do carry information. We should not rely too much on any one observation, but neither should we dismiss anyone as irrelevant.

This is the reason why the discussions on this board usually are informative, as different people use different angles and thereby many different observations are collectively analyzed in the same discussion. If we all would weigh everything by the same scales, these discussions would be rather dull.

 

[Big Phil]

Stats do need to be adjusted. If backchecking, defense and goaltending existed in the 80s, nobody would have scored over 120 points.

Yikes!
Jagr 127 points in 1999
Thornton 125 points in 2006
Jagr 123 points in 2006
Jagr 121 points in 2001
Crosby 120 points in 2007

In the last 10 years alone there have been 5 players to acheive this feat and a couple others that were very close (Thornton again, Sakic, Ovy, Malkin)

Despite popular belief it WAS hard to get points in the '80s as well. Scoring was at a higher clip yes, but that shouldn't diminish that whole decade. Gretzky got 200 points religiously but few others got more than 130 not named Mario.

That being said for the OP I hate adjusted stats too. There are much more relevant ways to compare eras. When I did a Beliveau vs. Messier poll a while back a simple solution was judging how many times each player was in the top 10 in scoring. Beliveau to many people's surprise blew Messier out of the water despite the fact that Mess is the 2nd leading scorer of all time. The purest way to judge eras, is how a player performed vs. his peers. Plain and simple.

 

[rallymaster19]

I do look at adjusted stats myself but only for guys I never saw play. How do I compare Eddie Shore to Dennis Potvin or Ted Lindsay to Jari Kurri? I never saw Shore or Lindsay play so adjusted stats are good to look at to give me a ballpark figure of how they might rate offensively against each other. But if I were to compare Potvin and Borque or Bossy and Bure I wouldn’t need adjusted stats because there is the mental adjustment I can make in my head. As most posters have stated so far, they are a useful tool that can help make initial judgment on players. But assuming we’re all knowledgeable hockey fans here, do we really need an inevitably incorrect mathematical adjustment or could we all just use our mental adjustments?

The other thing is that adjusted stats, when derived from their mathematical formula, are whole numbers, and not approximations with upper and lower limits as they should be to indicate to the user they are only an estimation of adjusted values. When you look at the Yzerman/Sakic example, Sakic’s 158 is certainly greater than Yzerman’s 155. As an example to a situation similar to this, I’ve seen numerous times where someone will say Sakic’s was better simply because 158 is a larger number than 155. But I think people fail to understand the math behind adjustments and that 158 is not a statistically significant number by any means. I’m not going to do the math on this myself but if you were to find a 95% confidence interval for Sakic’s adjusted 1989 points total, it would come out to something like 130-165 points. Therefore, Sakic’s offensive totals exclusively may be better than Yzerman’s, same as Yzerman’s or worse than Yzerman’s. And in the end, the adjusted stats don’t tell you anything in any close or relatively close comparison because the numbers aren’t statistically significant to form a conclusion.

Now on the other hand, if you were to compare Sakic’s ’01 season with say Trevor Linden’s ’89 season, go ahead, the adjusted numbers will tell you Sakic would blow him out of the water on that comparison, like it should. But for anyone that watched hockey the last two decades, you didn’t need to see the adjusted stats to know Sakic ‘01 was better than Linden ‘89.

 

[overpass]

The other thing is that adjusted stats, when derived from their mathematical formula, are whole numbers, and not approximations with upper and lower limits as they should be to indicate to the user they are only an estimation of adjusted values. When you look at the Yzerman/Sakic example, Sakic’s 158 is certainly greater than Yzerman’s 155. As an example to a situation similar to this, I’ve seen numerous times where someone will say Sakic’s was better simply because 158 is a larger number than 155. But I think people fail to understand the math behind adjustments and that 158 is not a statistically significant number by any means. I’m not going to do the math on this myself but if you were to find a 95% confidence interval for Sakic’s adjusted 1989 points total, it would come out to something like 130-165 points. Therefore, Sakic’s offensive totals exclusively may be better than Yzerman’s, same as Yzerman’s or worse than Yzerman’s. And in the end, the adjusted stats don’t tell you anything in any close or relatively close comparison because the numbers aren’t statistically significant to form a conclusion.

Sure, adjusted stats could have confidence intervals, although that might just confuse most people.

But Yzerman's stats need a confidence interval just as much as Sakic's do. There's nothing special about 1989 stats that make them the gold standard.

 

[canucks4ever]

Yzerman scored 155 points when other people like Jari Kurri, Mark Messier, Dale hawerchuk or Paul Coffey failed to reach 115, thats dominance. The top 5 scorers that year were mario, gretz, thier linemates and yzerman.

 

[Canadiens1958]

Drifting

Adjusted stats become problematic when the users start drifting into predictions of certainty as opposed to simply explaining or comparing.

 

[God Bless Canada]

Adjusted stats don't work. They're an abject failure from the moment they're conceived.

For one thing, greatness in hockey, with few exceptions (save percentage, goals against average), is not to be measured with a calculator. Maybe it's just me: I want to know what a player did, I want to know how he actually played the game, what he meant to the game, what he brought to the table, and what he didn't bring to the table. I don't want to know what a calculator tells me he might have done.

They also fail to take into account changes in the game. No adjusted formula can accurately take into account the impact that Bobby Orr had on the game.

Why did scoring peak in the 80s? A lot of reasons. Shabby goaltending is certainly one of them. But the talented wasn't as diluted as it is now. A lot of teams can't ice two credible scoring lines anymore. Lots of good players, but not many good NHL scorers. It's what hockey people were looking for. They were putting a premium on offensive skill. The one-dimensional scorers had more of a place in the late 70s and the 80s than any other time in the game's history. And the offensive defenceman suddenly became a factor, too. Defencemen played an offensive game throughout their developmental careers, and they could do it once they reached the NHL.

That's not to say that you can blindly look at the numbers. You can't do that. In any circumstance. 100 points now is different than 100 points 20 years ago. I won't deny that. But part of the reason that 100 points was attainable was that teams put a premium on offence that they don't have now, the offensive defenceman was becoming a big factor (they aren't as much of a factor now), and you had the two best centres ever ripping the league apart.

But you can't say that "Yzerman would have had x points in 1989, and Sakic would have had x points in 2001." Yzerman had more than 150 points in 1989, won the Pearson Award for best player in the league, as voted by the players. Maybe if Yzerman has top-notch talent to play with, he hits 175 points. And maybe if he didn't play against shabby opposition (I would argue five of the six worst teams in the league in 88-89 were in the Norris) for 32 games, Yzerman winds up around 120 points.