Adjusted Save Percentage

[Hockey Outsider]

Save percentage is, in my opinion, the best statistic to evaluate a goalie with. Every goaltending statistic (save percentage, wins, GAA, shutouts, etc) is influenced by the goalie’s team, however save percentage is less team-dependent than the others. I think this is intuitively obvious to anyone who studies goaltending, but I’ll explain if anybody’s curious.

The problem with save percentage is that it’s highly era-dependent. The purpose of this study is to adjust save percentage so that it’s comparable across seasons. I have data for 1983-2009.

Career Adjusted Save Percentage (min 400 games)

Name	Adj GP	Adj SA	Adj Sv	Sv%
Dominik Hasek	741	21368	19768	92.5%
Patrick Roy	1040	29471	27114	92.0%
Roberto Luongo	517	16744	15360	91.7%
Martin Brodeur	1009	26215	23943	91.3%
Tomas Vokoun	486	14928	13628	91.3%
John Vanbiesbrouck	872	25316	23110	91.3%
Guy Hebert	488	15379	14029	91.2%
Jean-Sebastien Giguere	436	12837	11707	91.2%
Ed Belfour	957	25678	23406	91.2%
Andy Moog	686	19252	17548	91.1%
Kelly Hrudey	664	20724	18888	91.1%
Daren Puppa	423	12179	11096	91.1%
Curtis Joseph	923	27638	25168	91.1%
Ron Hextall	605	16727	15211	90.9%
Mike Richter	660	20122	18294	90.9%
Martin Biron	407	12147	11043	90.9%
Tom Barrasso	746	22001	19999	90.9%
Evgeni Nabokov	471	12872	11697	90.9%
Sean Burke	804	24389	22162	90.9%
Marty Turco	433	11044	10035	90.9%
Felix Potvin	636	18855	17121	90.8%
Jon Casey	401	11172	10143	90.8%
Dwayne Roloson	435	12861	11674	90.8%
Bob Essensa	404	11976	10869	90.8%
Mike Liut	466	12993	11789	90.7%
Nikolai Khabibulin	661	19627	17807	90.7%
Jeff Hackett	473	14210	12891	90.7%
Jose Theodore	474	14133	12810	90.6%
Chris Osgood	690	18559	16809	90.6%
Olaf Kolzig	703	20961	18980	90.5%
Don Beaupre	585	16927	15318	90.5%
Jocelyn Thibault	558	16424	14856	90.5%
Tommy Salo	511	14333	12949	90.3%
Patrick Lalime	400	11085	10012	90.3%
Grant Fuhr	787	22904	20680	90.3%
Ron Tugnutt	498	14570	13154	90.3%
Mike Vernon	768	20514	18518	90.3%
Arturs Irbe	556	16034	14473	90.3%
Glenn Healy	418	12188	10996	90.2%
Ken Wregget	559	17368	15653	90.1%
Greg Millen	416	12054	10859	90.1%
Bill Ranford	628	18796	16925	90.0%
Kirk McLean	616	17573	15817	90.0%

[TBODY] [/TBODY]

Patrick Roy is incredibly underrated from a save percentage perspective. His peak occurred in the high-scoring late eighties and early nineties. He towered over his peers with almost Hasek-like dominance, but his raw numbers aren’t impressive because his played during an era that was very unfriendly to goalies. I often see people argue that Brodeur is better than Roy due to a higher save percentage. That would like comparing stats from an eighties player to a modern player, and concluding that the former was better. Adjusted for era, Roy was a significantly better regular season goalie than every goalie aside from Hasek in the past three decades.

Click to read more...

 

[Hockey Outsider]

Best Peak

Name	sv%
Dominik Hasek	93.3%
Patrick Roy	93.1%
John Vanbiesbrouck	92.4%
Curtis Joseph	92.2%
Ed Belfour	92.2%
Tom Barrasso	92.1%
Martin Brodeur	92.1%
Roberto Luongo	92.1%
Kelly Hrudey	91.9%
Tomas Vokoun	91.8%
Andy Moog	91.8%
Sean Burke	91.8%
Ron Hextall	91.8%
Guy Hebert	91.7%
Jean-Sebastien Giguere	91.7%
Dwayne Roloson	91.5%
Nikolai Khabibulin	91.5%
Olaf Kolzig	91.4%
Felix Potvin	91.4%
Mike Richter	91.4%
Evgeni Nabokov	91.3%
Jose Theodore	91.3%
Mike Vernon	91.3%
Jon Casey	91.3%
Mike Liut	91.2%
Don Beaupre	91.2%
Grant Fuhr	91.1%
Chris Osgood	91.1%
Mike Dunham	91.0%
Martin Biron	91.0%
Marty Turco	91.0%
Byron Dafoe	90.9%
Kirk McLean	90.9%
Arturs Irbe	90.9%
Ron Tugnutt	90.9%
Jocelyn Thibault	90.8%
Tommy Salo	90.8%
Bill Ranford	90.7%
Stephane Fiset	90.6%
Greg Millen	90.3%
Ken Wregget	90.3%

[TBODY] [/TBODY]

A few observations:

- Remember what I said earlier about Roy being underrated? He was so far ahead of all the other goalies in the league (in the late eighties and early nineties) that it’s almost Hasek-like. I think the Hasek/Roy debate is a lot subtler than most people realize. Adjusted for era, Roy separated himself from the pack almost as much as Hasek; despite winning only two Stanley Cups, Hasek was almost always an outstanding playoff goalie.

- Chris Osgood, during his best five years, was a somewhat above-average goalie.

- This is the weighted average of each goalie’s best five seasons in terms of save percentage. They don’t have to be consecutive. I am only including seasons when the goalie played in at least 40 adjusted games.

 

[Hockey Outsider]

METHODOLOGY

To calculate adjusted games played:

1. Start with games played.
2. Adjustment #1: pro-rate games played to an 82 game schedule.

To calculate adjusted shots against:

1. Start with shots against
2. Adjustment #1: pro-rate shots against to an 82 game schedule
3. Adjustment #2: pro-rate shots against to a 29.0 shot per game environment

To calculate adjusted saves:

1. Start with shots against
2. Adjustment #1: pro-rate saves to an 82 game schedule
3. Adjustment #2: pro-rate saves to a 29.0 shot per game environment
4. Adjustment #3: pro-rate saves to a 90.5% save percentage environment

To calculate adjusted save percentage:

1. Divide adjusted shots against by adjusted saves

EXAMPLE

Let’s look at Ed Belfour’s 1993-94 season.

Games played: Eddie played 66.6 games. We pro-rate that to 82 games (the season was actually 84 games), so Belfour gets credit for 66.6 * 82 / 84 = 65.0 games.

Shots against: the Eagle faced 1,892 shots. We pro-rate that to 82 games (the season was actually 84 games), and we pro-rate that to 29 shots against (the season average was actually 29.9 shots), so Belfour faced 1,892 * 82 / 84 * 29 / 29.9 = 1,794 adjusted shots against.

Adjusted saves: the Billionaire made 1,714 saves. We pro-rate that to 82 games (the season was actually 84 games), we pro-rate that to 29 shots against (the season average was actually 29.9 shots) and we pro-rate that 90.5% save percentage (the season average was actually 89.5%), so Belfour made 1,714 * 82 / 84 * 29 / 29.9 * 90.5% / 89.5% = 1,643 adjusted saves.

Adjusted save percentage: 1,643 adjusted saves / 1,794 adjusted shots against = 91.6%.

DISCLAIMERS

Even if it’s adjusted for era, save percentage has the following flaws:

1. Save percentage doesn’t take shot quality into account. A goalie might face more difficult shots due to playing more time shorthanded compared to average, playing behind a weak, risk-taking or injured defense, or facing tougher opponents than average.

2. Save percentage doesn’t take puckhandling into account. The contributions made by Brodeur and Belfour (among others) are understated.

DATA SOURCE

1. My raw data comes from the file “all2.txt.gz†from the Hockey Analysis Group (a Yahoo group).

2. My source data shows adjusted games played. I’m pretty sure they’re calculating it based on minutes, divided by 60.

3. I have not in any way attempted to verify the accuracy of the information in the source file that I used. Nothing looks seriously out of line but I can’t guarantee that’s it completely correct.

 

[Canadiens1958]

Interesting.................

Very interesting approach.

However certain factors have to be weighed. Games played. Playing 40 games vs 70 games is a significant difference.Perhaps increments of 10 games or % of scheduled games/minutes. From memory since SV% was first recognized as a stat circa 1983-84 only one goalie that led the NHL in SV% also led the league in games played / appearances.

How far back - seventies, sixties, fifties would you extend your study?

 

[foame]

I appreciate studies like this. Great work.

Wouldn't the schedule (how many times teams play eachother) change the amount of avg shots a goalie would face?

However certain factors have to be weighed. Games played. Playing 40 games vs 70 games is a significant difference.Perhaps increments of 10 games or % of scheduled games/minutes. From memory since SV% was first recognized as a stat circa 1983-84 only one goalie that led the NHL in SV% also led the league in games played / appearances.

Three have done it: Hextall, Belfour, Hasek.

 

[Caeldan]

I'd take exception to the premise that save % is less team-dependent than the other goalie stats in your first post, for the reason you make in your disclaimer in your 3rd post - shot quality allowed.

I'd say to truly adjust - you'd need to figure out a way to normalize shot quality by team.
One rough idea I have would be something to the effect of looking at the shots/saves of other goalies playing for the same team - with a window of 3-5 seasons possible (placing the current season you're analyzing in the middle). Of course in cases where it was rare for a backup to play, you're going to have an incredibly small sample size to compare against.

I'm curious as to the rationale for the 90.5% and 29 shot per game numbers.

 

[HokieHockey]

It would be interesting to see a graph that shows how the save pct. curve has moved from year to year. I'd assume it's a normal distribution (few goalies have low numbers, lots have moderate, few have great). Plotting the average save pct. against year would show if there were any trends (dead puck era...) or if it was random movement.

Good work, you seem to know what you are doing.

 

[Hockey Outsider]

Very interesting approach.

However certain factors have to be weighed. Games played. Playing 40 games vs 70 games is a significant difference.Perhaps increments of 10 games or % of scheduled games/minutes. From memory since SV% was first recognized as a stat circa 1983-84 only one goalie that led the NHL in SV% also led the league in games played / appearances.

Thank you.

Yes, I think it's important to balance sv% with workload. I'm still working on the post about Brodeur vs Hasek/Roy, but I fully agree that durability is an asset.

How far back - seventies, sixties, fifties would you extend your study?

I'd like to fill in the data for as many years as I have data for. Right now, I'm only confident that the data I have is accurate for 1983-present. I'm going to try to contact the person who posted my data source file & see where he got his earlier numbers from.

 

[Hockey Outsider]

It would be interesting to see a graph that shows how the save pct. curve has moved from year to year. I'd assume it's a normal distribution (few goalies have low numbers, lots have moderate, few have great). Plotting the average save pct. against year would show if there were any trends (dead puck era...) or if it was random movement.

Good work, you seem to know what you are doing.

Thank you. This would require a fair amount of work but I plan to do this at some point over the next few days.

 

[Hockey Outsider]

I'd take exception to the premise that save % is less team-dependent than the other goalie stats in your first post, for the reason you make in your disclaimer in your 3rd post - shot quality allowed.

I agree that save percentage doesn't take shot quality into account. However, no other goalie stat takes shot quality into account either -- and the other goalie stats are usually influenced by the goalie's teammates abilities, that don't influence save percentage.

GAA: like save percentage, GAA doesn't take shot quality into account. A goalie that faces five breakaways will have a high GAA than a goalie that faces five shouts from the blue line (all other things being equal). The reason I like save percentage more than GAA is that there's a very direct relationship between shots faced and GAA; the more shots a goalie faces, the higher their GAA. (This is intuitively obvious to me, though I can run some numbers if anybody's curious). There's essentially zero relationship between the number of shots faced, and save percentage. Since the number of shots faced is primarily (though not entirely) determined by how well the goalie's team plays defensively, GAA significantly influenced by a factor that is beyond the goalie's control.

Wins: wins are equally determined by two things: goals scored, and goals allowed. IMO goalies have virtually no impact on goals scored. (Goals scored doesn't influence save percentage, there's essentially no relationship between those two numbers). The number of goals allowed is just GAA, which I talked about above. Wins don't tell you how well an individual goalie played because it's highly influenced by things that are beyond a goalie's control, namely the number of shots they face and the number of goals their team scores.

Shutouts: I think that shutouts are not a meaningful statistic. Does it really matter how many times a goalie allows zero goals? Isn't it more relevant and informative to consider how well they played in every game? To be clear, shutouts are great. However, you need to look at more than just shutouts to determine if a goalie has played well.

To be perfectly clear, the goal of a team is to win games. If you told me that I could have a 60-win team with a weak goalie, and a 20-win team with an excellent goalie, I'd choose the 60-win team every time. It doesn't necessarily mean that the goalie on the 60-win team is a good goalie though. (It doesn't necessarily mean that a goalie on a good team is a bad goalie, of course!)

I'd say to truly adjust - you'd need to figure out a way to normalize shot quality by team.
One rough idea I have would be something to the effect of looking at the shots/saves of other goalies playing for the same team - with a window of 3-5 seasons possible (placing the current season you're analyzing in the middle). Of course in cases where it was rare for a backup to play, you're going to have an incredibly small sample size to compare against.

The goaltending blog "Brodeur is a Fraud" tries to analyze goalie performance using this method.

You're right about the main problem with this method -- I think that the sample sizes are often so small that the results are not statistically meaningful.

The other problem with this method is that not all backups are created equal. Sometimes your backup may be Dominik Hasek, other times your backup is Dan Cloutier. Obviously, being 5% better than Hasek is far more impressive than being 5% better than Cloutier.

I'm curious as to the rationale for the 90.5% and 29 shot per game numbers.

I'm trying to put the numbers into the context of the post-lockout NHL. Over the past four seasons, the averages were 29.4 shots per game and a 90.6% save percentage. I rounded them.

Keep in mind that I'm applying the same adjustment factor to every goalie (i.e., every goalie gets normalized to a 90.5% environment) so this doesn't give any goalie an advantage relative to the others.

 

[seventieslord]

Dang, I was going to do this possibly over the holidays. And I was going to extend it all the way back to the 1950s.

A year ago this same thing happened when we both got the idea to add up every players' top-5, 10, and 20 finishes in goals too!

 

[seventieslord]

- Something doesn't look right. It seems that more recent goalies appear higher on this list more often, and the 80s goalies appear lower.

- Something also just doesn't seem right to me about the methodology. it's late and I wouldn't be able to articulate it that well. But wouldn't it make sense to just calculate a goalie's error rate (1 - sv%) for each season, and then compare it to the league average, coming up with a percentage above or below average, and then weigh that season by shots faced, and then add up all seasons?

The results generally look correct. I'd have bet my life savings that Hasek would be first, Roy 2nd, and Luongo 3rd. I thought Brodeur would appear more ordinary, but now that I look at it, it makes sense, plus he's as far away from Luongo as he is from Burke, as far away from Roy as he is from Theodore, and as far away from Hasek as he is from the very bottom.

 

[Hockey Outsider]

MARTIN BRODEUR

In my opinion, there have been three elite goalies from 1983-present: Brodeur, Hasek and Roy. I think that Belfour is generally underrated and isn't far behind the big three, but I'm going to focus on the three aforementioned netminders. Interestingly, Hasek and Roy are, by fairly wide margins, the leaders in career save percentage and peak save percentage. Brodeur supporters always say that Brodeur has special intangibles that make up for a comparatively weak save percentage.

Intangible #1: durability

Brodeur is almost certainly the most durable goalie of the modern era. Durability has value, because it means that you have an excellent goalie playing, rather than a mediocre backup. Does Brodeur’s durability make up for the superior performance of Hasek and Roy?

Over his ten best seasons (sorted by save percentage, minimum 40 games played), here’s how the goalies fared:

Goalie|Save %|Games
Brodeur|91.7%|69.8
Hasek|92.7%|61.0
Roy|92.6%|55.7

In the regular season, general managers shouldn’t care about how individual goalies perform, what really matters is how the team overall performs. When you have Brodeur on your team, what you’re really getting is 69.8 games at a 91.7% save percentage and 12.2 games from a backup goalie who posts an “X” save percentage. We can solve for “X” to find out what save percentage a backup would need to post, in order for a team to be indifferent to Brodeur and Hasek/Roy.

To be indifferent between Roy and Brodeur, a team would need to find a backup goalie with an 88.1% save percentage. In other words, a team with 69.8 games of Brodeur at 91.7%, and a backup for 12.2 games at 88.1%, is equal to 55.7 games of Roy at 92.6% percent, and a backup (also at 91.7%) for 26.3 games. The team would be better with Brodeur only if the backup goalie posted a lower save percentage than what I specified.

To be indifferent between Hasek and Brodeur, a team would need to find a backup goalie with an 84.8% save percentage.

This, to me, conclusively dismisses the argument that Brodeur’s advantage in durability makes up for Hasek and Roy’s superior play. In the modern NHL, goalies with an 88% save percentage are extremely rare; goalies with an 85% save percentage simply don’t exist. A competent general manager would be better off taking either Roy or Hasek, and playing their backup more, rather than taking Brodeur.

Intangible #2: puckhandling

Brodeur is arguably the greatest puckhandling goalie ever. Many of his supporters say that save percentage underrates Brodeur, because it fails to take into account all of the shots he prevents through his puckhandling.

How many shots would Brodeur need to prevent each game, through his puckhandling, to equal Roy and Hasek’s superior save percentage? Roy stopped 92.6% of his shots. Brodeur, over his ten best seasons averaged 26.64 saves on 29.07 shots. To determine how many extra shots Brodeur would need to prevent we can calculate [(26.64+ X) / (29.07+ X) = 92.6%], where X is the number of shots he prevents. We can do a similar calculation for Hasek and his 92.7% save percentage.

In order to match Roy’s save percentage, Brodeur would need to prevent 3.7 shots per game. In order to match Hasek’s save percentage, Brodeur would need to prevent 4.1 shots per game.

Is Brodeur’s puckhandling good enough to stop 4 shots per game? I think that’s a bit high. Keep in mind that every time Brodeur touches the puck, it’s not as if he saves a shot against. There are dozens of times per game when the puck is dumped into the Devils’ defensive zone, and defensemen would have been able to clear the out safely (and/or the opponents gather the puck but aren’t able to get a shot off) regardless of whether Brodeur touches the puck.

From 1994 to 2009, the Devils allowed an average of 25.7 shots per game. The league average during those years was 28.5 shots per game. If Brodeur saved the Devils 4 shots per game, that means that the Devils (minus Brodeur) were a below-average team defensively! That doesn’t pass the common sense test. The Devils, for the past fifteen years, have been a highly disciplined team, had two Hall of Fame defensemen, and were fairly conservative. There’s no way that they would be a below average defensive team without Brodeur, so I think it’s a stretch to say that his puckhandling stopped 4 shots per game.

Both intangibles, on their own, are insufficient to put Brodeur on the same level as Roy and Hasek. But what happens if we put both intangibles together?

 

[Hockey Outsider]

Putting both intangibles together

Both of Brodeur’s intangibles, individually, can’t possibly make up for Roy and Hasek’s superior ability. But what happens if we look at both of his intangibles at the same time? This analysis is tricky because we’re changing two variables at the same time (how good is the backup goalie and how many shots per game does Brodeur prevent)?

Brodeur vs Roy

Shots Prevented|Backup sv%
0 | 87.8%
1 | 89.3%
2 | 90.8%
3 | 92.1%
4 | 93.3%

This chart shows how good the backup goalie would need to be, in order for a general manager to be indifferent between Brodeur and Roy , given five different assumptions about how many shots Brodeur’s puckhandling prevents.

If you’re a “puckhandling sceptic” and think that Brodeur’s puckhandling abilities don’t have much of an impact on the game (i.e., preventing 1 shot per game), you’re better taking Roy. It would not be difficult to get a backup goalie with at least an 89.3% save percentage. Thus, Roy's superior puck-stopping ability more than offsets Brodeur's intangibles.

If you’re a “puckhandling believer” and think that Brodeur can greatly influence a game (i.e., preventing 3 shots per game), you’re better taking Brodeur. There’s no way you can reasonably expect to find a 92.1% backup goalie, so Brodeur’s durability and puckhandling more than offset Roy’s superior ability at stopping the puck.

Personally, I’m right in the middle. I think that Brodeur is essentially tied with Roy in the regular season, taking all the intangibles into account. However, Roy ranks higher on my all-time list due to a clearly superior playoff resume.

Brodeur vs Hasek

Shots Prevented|Backup sv%
0 | 83.7%
1 | 86.1%
2 | 88.4%
3 | 90.5%
4 | 92.5%

Even after we take the intangibles into account, it’s almost impossible to argue that Brodeur was a more valuable goalie than Hasek.

If we’re “puckhandling skeptics” and think that Brodeur’s puckhnandling prevents 2 or fewer shots against per game, Hasek is clearly the more valuable player. A good GM would pick Hasek and easily find an 88.4% backup goalie, and he’d be better off than picking Brodeur.

I’ve already dismissed 4 shots per game as an unrealistically high impact for stickhandling.

The only conceivable argument for taking Brodeur over Hasek is if you think he prevents 3 shots per game due to stickhandling, and a team would be unable to find a statistically average backup goalie. (Remember my numbers are all normalized to a 90.5% environment). I find that even this argument is a stretch – if Brodeur’s puckhandling is worth 3 shots per game, that implies that the Devils are just average defensively, which doesn't pass the common sense test; besides, there are lots of backup goalies who post average save percentages (or are within a few tenths of a percent in either direction) so at best a GM would be indifferent between the two.

The other intangible worth mentioning is shot quality. Based on watching hundreds of games from these goalies, I'm confident saying that Brodeur, on average, faced easier (lower-quality) shots compared to Hasek. Even if we ignore this factor, Hasek comes out ahead.

Overall conclusion

Brodeur is underrated if we only look at save percentage, because it fails to take two intangibles into account – his durability and his puckhandling ability. If we look at both of these intangibles, Brodeur and Roy are essentially even in the regular season. No possible amount of intangibles would make Brodeur more valuable than Hasek.

 

[seventieslord]

Is Brodeur’s puckhandling good enough to stop 4 shots per game? I think that’s a bit high. Keep in mind that every time Brodeur touches the puck, it’s not as if he saves a shot against. There are dozens of times per game when the puck is dumped into the Devils’ defensive zone, and defensemen would have been able to clear the out safely (and/or the opponents gather the puck but aren’t able to get a shot off) regardless of whether Brodeur touches the puck.

It's not just that, either. Every goalie nowadays handles the puck and prevents shots against. Brodeur needs to be measured by how many more shots he prevents than them. The best estimates I've seen indicate that it's about one per game.

 

[Canadiens1958]

Wins

I agree that save percentage doesn't take shot quality into account. However, no other goalie stat takes shot quality into account either -- and the other goalie stats are usually influenced by the goalie's teammates abilities, that don't influence save percentage.

GAA: like save percentage, GAA doesn't take shot quality into account. A goalie that faces five breakaways will have a high GAA than a goalie that faces five shouts from the blue line (all other things being equal). The reason I like save percentage more than GAA is that there's a very direct relationship between shots faced and GAA; the more shots a goalie faces, the higher their GAA. (This is intuitively obvious to me, though I can run some numbers if anybody's curious). There's essentially zero relationship between the number of shots faced, and save percentage. Since the number of shots faced is primarily (though not entirely) determined by how well the goalie's team plays defensively, GAA significantly influenced by a factor that is beyond the goalie's control.

Wins: wins are equally determined by two things: goals scored, and goals allowed. IMO goalies have virtually no impact on goals scored. (Goals scored doesn't influence save percentage, there's essentially no relationship between those two numbers). The number of goals allowed is just GAA, which I talked about above. Wins don't tell you how well an individual goalie played because it's highly influenced by things that are beyond a goalie's control, namely the number of shots they face and the number of goals their team scores.

Shutouts: I think that shutouts are not a meaningful statistic. Does it really matter how many times a goalie allows zero goals? Isn't it more relevant and informative to consider how well they played in every game? To be clear, shutouts are great. However, you need to look at more than just shutouts to determine if a goalie has played well.

To be perfectly clear, the goal of a team is to win games. If you told me that I could have a 60-win team with a weak goalie, and a 20-win team with an excellent goalie, I'd choose the 60-win team every time. It doesn't necessarily mean that the goalie on the 60-win team is a good goalie though. (It doesn't necessarily mean that a goalie on a good team is a bad goalie, of course!)



The goaltending blog "Brodeur is a Fraud" tries to analyze goalie performance using this method.

You're right about the main problem with this method -- I think that the sample sizes are often so small that the results are not statistically meaningful.

The other problem with this method is that not all backups are created equal. Sometimes your backup may be Dominik Hasek, other times your backup is Dan Cloutier. Obviously, being 5% better than Hasek is far more impressive than being 5% better than Cloutier.



I'm trying to put the numbers into the context of the post-lockout NHL. Over the past four seasons, the averages were 29.4 shots per game and a 90.6% save percentage. I rounded them.

Keep in mind that I'm applying the same adjustment factor to every goalie (i.e., every goalie gets normalized to a 90.5% environment) so this doesn't give any goalie an advantage relative to the others.

Your position on wins should be revisited. Allowing the first goal or two in a game is a major factor and changes the way the game is played. Having to play catch-up provides more high quality offensive opportunities for the opposition.

The impact of allowing two goals on thirty shots is identical in terms of SV% whether the two goals are in the first or last period but the impact on the game is significant.

One of the strengths of Ken Dryden was not allowing the early goal.
First/second goals have to be considered.

 

[BM67]

Overall conclusion

Brodeur is underrated if we only look at save percentage, because it fails to take two intangibles into account – his durability and his puckhandling ability. If we look at both of these intangibles, Brodeur and Roy are essentially even in the regular season. No possible amount of intangibles would make Brodeur more valuable than Hasek.

More good food for thought, keep it coming Hockey Outsider.

But...

These are not the only factors not accounted for unfortunately.

Brodeur is not only a great puck handler, but also has superior rebound control and is an above average pokechecker. A 4 shot a game difference may not be unreasonable when all are accounted for. Of course it's almost impossible to determine what the impact of these skills are compared to the average.

Also one needs to account for the impact of a missed pokecheck, and a turnover while wandering from the net have. If Brodeur prevents 1000 shots, but also gives up 18 high quality chances, only the high quality chances have an impact on his stats.

Is more shots faced per game the sign of lesser team defense? To me the Devils are more about preventing a shot from a high quality scoring area, than about preventing the shot at all.

"Matvichuk was an old school player born in Edmonton who learned his craft with the Dallas Stars and loved to dive and block shots, the kind of thing we didn't do a lot in Jersey." - Martin Brodeur, Beyond the Crease

Other factors such as the unbalanced schedule will also impact how much relevance the league average sv% has as well. A quick look at the 96-97 season indicates that Hasek faced an above average shooting Northeast Division, and thus his numbers are probably even more impressive than as viewed just against the league average.

 

[Eisen]

Save percentage is, in my opinion, the best statistic to evaluate a goalie with. Every goaltending statistic (save percentage, wins, GAA, shutouts, etc) is influenced by the goalie’s team, however save percentage is less team-dependent than the others. I think this is intuitively obvious to anyone who studies goaltending, but I’ll explain if anybody’s curious.

The problem with save percentage is that it’s highly era-dependent. The purpose of this study is to adjust save percentage so that it’s comparable across seasons. I have data for 1983-2009.

Career Adjusted Save Percentage (min 400 games)

Goalie|Save Percentage
Dominik Hasek | 92.5%
Patrick Roy | 92.0%
Roberto Luongo | 91.7%
Martin Brodeur | 91.3%
Tomas Vokoun | 91.3%
John Vanbiesbrouck | 91.3%
Guy Hebert | 91.2%
Jean-Sebastien Giguere | 91.2%
Ed Belfour | 91.2%
Andy Moog | 91.1%
Kelly Hrudey | 91.1%
Daren Puppa | 91.1%
Curtis Joseph | 91.1%
Ron Hextall | 90.9%
Mike Richter | 90.9%
Martin Biron | 90.9%
Tom Barrasso | 90.9%
Evgeni Nabokov | 90.9%
Sean Burke | 90.9%
Marty Turco | 90.9%
Felix Potvin | 90.8%
Jon Casey | 90.8%
Dwayne Roloson | 90.8%
Bob Essensa | 90.8%
Mike Liut | 90.7%
Nikolai Khabibulin | 90.7%
Jeff Hackett | 90.7%
Jose Theodore | 90.6%
Chris Osgood | 90.6%
Olaf Kolzig | 90.5%
Don Beaupre | 90.5%
Jocelyn Thibault | 90.5%
Tommy Salo | 90.3%
Patrick Lalime | 90.3%
Grant Fuhr | 90.3%
Ron Tugnutt | 90.3%
Mike Vernon | 90.3%
Arturs Irbe | 90.3%
Glenn Healy | 90.2%
Ken Wregget | 90.1%
Greg Millen | 90.1%
Bill Ranford | 90.0%
Kirk McLean | 90.0%

Patrick Roy is incredibly underrated from a save percentage perspective. His peak occurred in the high-scoring late eighties and early nineties. He towered over his peers with almost Hasek-like dominance, but his raw numbers aren’t impressive because his played during an era that was very unfriendly to goalies. I often see people argue that Brodeur is better than Roy due to a higher save percentage. That would like comparing stats from an eighties player to a modern player, and concluding that the former was better. Adjusted for era, Roy was a significantly better regular season goalie than every goalie aside from Hasek in the past three decades.

OK, I am a newbie, but could you tell me, why s% is era-dependant? I shouldn't make any difference if you face 40 or a hundred shots, since it's already in percentile. I can see why goals scored are era dependant, a style or a philosphy should lead to more shots and thus increase or decrease goals, but the s% should remain the same. What am I overlooking?

 

[Dennis Bonvie]

Save percentage is, in my opinion, the best statistic to evaluate a goalie with. Every goaltending statistic (save percentage, wins, GAA, shutouts, etc) is influenced by the goalie’s team, however save percentage is less team-dependent than the others. I think this is intuitively obvious to anyone who studies goaltending, but I’ll explain if anybody’s curious.

The problem with save percentage is that it’s highly era-dependent. The purpose of this study is to adjust save percentage so that it’s comparable across seasons. I have data for 1983-2009.

Career Adjusted Save Percentage (min 400 games)

Goalie|Save Percentage
Dominik Hasek | 92.5%
Patrick Roy | 92.0%
Roberto Luongo | 91.7%
Martin Brodeur | 91.3%
Tomas Vokoun | 91.3%
John Vanbiesbrouck | 91.3%
Guy Hebert | 91.2%
Jean-Sebastien Giguere | 91.2%
Ed Belfour | 91.2%
Andy Moog | 91.1%
Kelly Hrudey | 91.1%
Daren Puppa | 91.1%
Curtis Joseph | 91.1%
Ron Hextall | 90.9%
Mike Richter | 90.9%
Martin Biron | 90.9%
Tom Barrasso | 90.9%
Evgeni Nabokov | 90.9%
Sean Burke | 90.9%
Marty Turco | 90.9%
Felix Potvin | 90.8%
Jon Casey | 90.8%
Dwayne Roloson | 90.8%
Bob Essensa | 90.8%
Mike Liut | 90.7%
Nikolai Khabibulin | 90.7%
Jeff Hackett | 90.7%
Jose Theodore | 90.6%
Chris Osgood | 90.6%
Olaf Kolzig | 90.5%
Don Beaupre | 90.5%
Jocelyn Thibault | 90.5%
Tommy Salo | 90.3%
Patrick Lalime | 90.3%
Grant Fuhr | 90.3%
Ron Tugnutt | 90.3%
Mike Vernon | 90.3%
Arturs Irbe | 90.3%
Glenn Healy | 90.2%
Ken Wregget | 90.1%
Greg Millen | 90.1%
Bill Ranford | 90.0%
Kirk McLean | 90.0%

Patrick Roy is incredibly underrated from a save percentage perspective. His peak occurred in the high-scoring late eighties and early nineties. He towered over his peers with almost Hasek-like dominance, but his raw numbers aren’t impressive because his played during an era that was very unfriendly to goalies. I often see people argue that Brodeur is better than Roy due to a higher save percentage. That would like comparing stats from an eighties player to a modern player, and concluding that the former was better. Adjusted for era, Roy was a significantly better regular season goalie than every goalie aside from Hasek in the past three decades.

Just wondering, why isn't Henrik Lundquist on the list? Not enough time in the league?

 

[MXD]

Very interesting approach.

However certain factors have to be weighed. Games played. Playing 40 games vs 70 games is a significant difference.Perhaps increments of 10 games or % of scheduled games/minutes. From memory since SV% was first recognized as a stat circa 1983-84 only one goalie that led the NHL in SV% also led the league in games played / appearances.

How far back - seventies, sixties, fifties would you extend your study?

To be honest, I'm not sure this is AS relevant as you implies. And would really, but really help O-6 goalies.

- Something doesn't look right. It seems that more recent goalies appear higher on this list more often, and the 80s goalies appear lower.

- Something also just doesn't seem right to me about the methodology. it's late and I wouldn't be able to articulate it that well. But wouldn't it make sense to just calculate a goalie's error rate (1 - sv%) for each season, and then compare it to the league average, coming up with a percentage above or below average, and then weigh that season by shots faced, and then add up all seasons?

The results generally look correct. I'd have bet my life savings that Hasek would be first, Roy 2nd, and Luongo 3rd. I thought deur would appear more ordinary, but now that I look at it, it makes sense, plus he's as far away from Luongo as he is from Burke, as far away from Roy as he is from Theodore, and as far away from Hasek as he is from the very bottom.

Hummm... That list starts at 83-84, which means some guys in this list probably had better seasons (compared to the average NHL goaltender) the ones considered in this study. Two names immediately comes to mind : Pete Peeters and Mike Liut.

The problem is... well, I can't quite explain it, but I'm pretty sure Bob Froese (85-86) or Patrick Roy (89-90) prevented - for a lack of a better word - more goals per shot when compared to the average goalie than Dominik Hasek did. Did the annual SV% average (and Std.Dev.) is considered?

FOAME : You point about sv% and games would tend to penalize guys who are already penalized by the fact they were playing with better backups (thus, not earning as much shots and saves during their primes).

 

[Canadiens1958]

Workload

To be honest, I'm not sure this is AS relevant as you implies. And would really, but really help O-6 goalies.





Hummm... That list starts at 83-84, which means some guys in this list probably had better seasons (compared to the average NHL goaltender) the ones considered in this study. Two names immediately comes to mind : Pete Peeters and Mike Liut.

The problem is... well, I can't quite explain it, but I'm pretty sure Bob Froese (85-86) or Patrick Roy (89-90) prevented - for a lack of a better word - more goals per shot when compared to the average goalie than Dominik Hasek did. Did the annual SV% average (and Std.Dev.) is considered?

FOAME : You point about sv% and games would tend to penalize guys who are already penalized by the fact they were playing with better backups (thus, not earning as much shots and saves during their primes).

Playing fewer games means virtually fewer or no 3 games in 4 nights sequences plus skipping games against teams that are not a good match for a specific goalies style.

As for helping O6 goalies - it would reflect strategic use of goaltenders plus the fatigue factor across all eras.

 

[TrevorLinden16]

I found a study somewhere, though I can't seem to find it now, that divided the ice into 2 foot by two foot segments and measured for a season where shots came from. It then decided expected save percentage from each square and thus formulated an expected save percentage for all goalies, which was essentially based on shot quality. Then it took how much above or below the expected each goalie was and ranked them. Does anyone else know what I'm talking about?

 

[overpass]

Good work and analysis, Hockey Outsider. Of course your adjustments don't adjust for everything, as you made clear, but it's better than trying to compare without adjusting anything.

I just have a couple of points to add. I don't think your examination of durability is entirely fair to Roy. While it may be correct from a strict value standpoint to just look at average games started, it's worth noting that Roy spent his first few years in a league in which starting goalies started fewer games on average and backups got more work. He wasn't as durable as Brodeur, but if you adjust for era the gap is closer.

Second, I think one area you left unexamined is shot quality due to power plays against. This article on Puck Prospectus gives an overview of the impact power plays can have on a goalie's save percentage. Brodeur faced far fewer power plays than average over his career in New Jersey, and Roy also benefited from the same effect in Montreal.

I found a study somewhere, though I can't seem to find it now, that divided the ice into 2 foot by two foot segments and measured for a season where shots came from. It then decided expected save percentage from each square and thus formulated an expected save percentage for all goalies, which was essentially based on shot quality. Then it took how much above or below the expected each goalie was and ranked them. Does anyone else know what I'm talking about

Gabriel Desjardins did this on either his site, www.behindthenet.ca, or on his Behind the Net blog.

 

[Ohashi_Jouzu*]

Just wondering, why isn't Henrik Lundquist on the list? Not enough time in the league?

Exactly. The table clearly states "400 game minimum", whereas Lundqvist hasn't eclipsed the 300 mark yet.

 

[haakon84]

MARTIN BRODEUR
Intangible #2: puckhandling

Brodeur is arguably the greatest puckhandling goalie ever. Many of his supporters say that save percentage underrates Brodeur, because it fails to take into account all of the shots he prevents through his puckhandling.

How many shots would Brodeur need to prevent each game, through his puckhandling, to equal Roy and Hasek’s superior save percentage? Roy stopped 92.6% of his shots. Brodeur, over his ten best seasons averaged 26.64 saves on 29.07 shots. To determine how many extra shots Brodeur would need to prevent we can calculate [(26.64+ X) / (29.07+ X) = 92.6%], where X is the number of shots he prevents. We can do a similar calculation for Hasek and his 92.7% save percentage.

In order to match Roy’s save percentage, Brodeur would need to prevent 3.7 shots per game. In order to match Hasek’s save percentage, Brodeur would need to prevent 4.1 shots per game.

Is Brodeur’s puckhandling good enough to stop 4 shots per game? I think that’s a bit high. Keep in mind that every time Brodeur touches the puck, it’s not as if he saves a shot against. There are dozens of times per game when the puck is dumped into the Devils’ defensive zone, and defensemen would have been able to clear the out safely (and/or the opponents gather the puck but aren’t able to get a shot off) regardless of whether Brodeur touches the puck.

From 1994 to 2009, the Devils allowed an average of 25.7 shots per game. The league average during those years was 28.5 shots per game. If Brodeur saved the Devils 4 shots per game, that means that the Devils (minus Brodeur) were a below-average team defensively! That doesn’t pass the common sense test. The Devils, for the past fifteen years, have been a highly disciplined team, had two Hall of Fame defensemen, and were fairly conservative. There’s no way that they would be a below average defensive team without Brodeur, so I think it’s a stretch to say that his puckhandling stopped 4 shots per game.

Both intangibles, are their own, are insufficient to put Brodeur on the same level as Roy and Hasek. But what happens if we put both intangibles together?

I have a problem with this methodology. You have to factor in the rule changes. I would not think that Brodeur was able to prevent 4 shots per game from 1994 to 2009 but from 1994 to 2003 I think it is highly plausible. It may be plausible that post lock-out Brodeur prevents 2-3 shots a game over a far inferior puckhandling replacement.

I think you have to compare league averages for shots against post-trapezoid and pre-trapezoid. Not to mention delayed offsides.

If we look at pre-lockout and trapezoid I think it is very fair to say he prevented around 4 shots a game. Since Brodeur did not give up many goals in those seasons this would have a huge effect on his save %. Let's look at '97-'98 and Brodeur vs. Dunham.

Shots Allowed Per Game:
Brodeur: 22.8 shots allowed per game
Dunham: 26 shots allowed per game

Now when you factor in quality of opponents. Dunham faced below league average competition in his starts (78 points per game) yet allowed more shots per game. If the Devils were so stingy then you'd think they'd tighten up with their back up in net.

If we pro-rate Brodeur's save % with the added 4 shots a game that would leave him with 1788 shots against and a new save % of .927 (instead of .917). If we do this with the following 6 seasons it is around a .01 increase in save %. I don't think it would put the Devils as a below average defensive team either.

When you look at post-lockout New Jersey. It is fair to say they were in fact a below average defensive team (especially when you look at the names on defense). From '05-'08 they averaged an SRS of about 0. Meaning they were incredibly average. Brodeur's shot prevention made their defense look stronger than it was. I don't think it's unreasonable to say he prevents about 1-3 shots per game depending on replacement.

You also don't take into consideration face-offs and flow of the game I know the TCG is working on a study which better exemplifies this and I'm interesting to see where it leads. Brodeur tilted the ice to the opposition side allowing the Devils to have more shots for and against. If you look at when Hasek's teams shots for are close to or above their shots against, his save % drops.