Goalie effectiveness formula?

[VanIslander]

Didn't Darren Pang call Hasek the worst goalie ever at one point? (and I don't remember him prefacing it with "worst technical goalie".)

It proves my point: a good goalie is one that stops the puck. End point.

I too thought Hasek was a joke until the year before his first league MVP, when he kept making "fluke" save after "fluke" save until a friend of mine said I used the word "fluke" seven times in one game to describe Hasek's saves. I became self-conscious of my words after that, and soon came to see that he displayed the most incredible reflexes I've ever seen in a goalie. For a few years he was astonding, unbelievably so in fact. technique had nothing to do with it.

The great Soviet goalie Tikhov coached belfour in Chicago, ignoring Hasek, because, as the all-time great Soviet said: "I can't teach someone who breaks all the rules."

Stop the damn puck. That's the only standard. I'd love to see another creative goaltender. The Brodeurs of today are effective but damn boring.

 

[KOVALEV10*]

It proves my point: a good goalie is one that stops the puck. End point.

I too thought Hasek was a joke until the year before his first league MVP, when he kept making "fluke" save after "fluke" save until a friend of mine said I used the word "fluke" seven times in one game to describe Hasek's saves. I became self-conscious of my words after that, and soon came to see that he displayed the most incredible reflexes I've ever seen in a goalie. For a few years he was astonding, unbelievably so in fact. technique had nothing to do with it.

The great Soviet goalie Tikhov coached belfour in Chicago, ignoring Hasek, because, as the all-time great Soviet said: "I can't teach someone who breaks all the rules."

Stop the damn puck. That's the only standard. I'd love to see another creative goaltender. The Brodeurs of today are effective but damn boring.

Well thats what made Hasek stand out from the rest... not to many goalies had the talent that he had but IMO none brought in the excitment he did. Maybe Roy in 1986 but noone else period!

 

[trentmccleary]

The great Soviet goalie Tikhov coached belfour in Chicago, ignoring Hasek, because, as the all-time great Soviet said: "I can't teach someone who breaks all the rules."

I may be getting a little tired and a little drunk... but I was a Hawk fan then and I'm pretty sure you meant Tretiak (who wasn't exactly a normal full time goalie coach).

 

[Hockey Outsider]

Thanks for the DB and sorry for the long delay...

I have a couple of issues with your formula but it does show that much reflection went in to attempting to interpret the stats.

Thanks for the feedback, I really do appreciate it.

Issue 1:
The comparison of a goalie to the seasonal league average is valid, to a point, but the extension of that comparison to anything beyond the dataset (season) is beyond what the data can infer. For example, the fact that Dryden stands out among goalies in '76 in no way should be interpretted to mean that he would stand out among goalies of the '95. The data shows that Dryden was X goals saved (GS) above the league average in '76. The GS in no way relates to seasons outside the one in question and has little to no meaning outside its own dataset.

True to an extent. We never know how a goalie would have performed outside of their season. For their career stats, I'm showing how they performed against their peers. One could argue that the level of competition in Plante's era was higher than that of Roy's, or vice-versa.

Issue 2:
There is no adjustment for rule changes and how that may influence the GS calculation. In the '50's, players ceased to serve a full two minutes for penalties regardless of whether the opposing team scored or not. In the '70's, the blue line face-off dot was moved from the center of the ice, affording the luxury of prime attacking position, to make two dots, one on either side. In the late '80's, teams no longer played four-on-four when coincidental minors were called. The 90's brought the end of tag-up offsides forcing the forechecking team to refrain from pressing the defenders and stacking players in the neutral zone. Every one of these changes affected the way the game was played and how many chances to score were generated. I am sure there are more but this is off the top of my head. The point is that the game being played today is very different than the one being played in years past.

I agree that I haven't accounted for rule changes directly, but the formula accounts for them indirectly. For example, let's look at the rule in the 50's that made players serve a full 2 minutes for penalties. Over the course of the season that would increase the powerplay opportunities in the league and decrease the average save percentage. Since we're comparing goalies to the league average that year, their save percentage will be lower, but so will the league average. Since we compare them to the league average, the rule changes won't affect their rating.

Issue 3:
There is no allotment for the "timing" of a save. The GS formula assumes that all saves must be made. This is not the case. Soft goals allowed in lopsided games for both sides are not that meaningful while spectacular saves made when the team needs it most are invaluable. Grant Fuhr, for example, would allow certain goals in the first two periods but that was in stark contrast to what he could stop in the third period. By stipulating that all goals are equal, we neglect to credit "money" goalies who raise their ability to play when it is needed. If you set a standard of "all goals are equal" as a rating for goalies, then disregard this objection. Also, having re-read the thread, I see that the issue of intagibles has been brought up so I will leave it at that.

Goals Saved assumes that all goals are equal. Personally, I'm not sure if that statement is true or not, one can make a strong case either way.

In any case, I don't think it's possible to take into account "money" goalies, statistically. However it definitely is worth considering, and it should be considered an intangible.

I also may have issue with how the formula is applied even within a given dataset, but I have yet to read the SQN essay so I will leave that for now.

All criticisms aside, I do like the way you went out and stated your conclusions and put forth their result with confidence. Not many people would outright state that Sawchuk or Brodeur are overrated, even if they had cooked up a formula to back them up. It was entertaining and made me rethink how I rate goalies and how I, at one time, unsuccessfully tried to brew a formula that would reflect goaltending ability rather than collective team contribution. It was a good read.

Actually, I only meant that Brodeur was overrated the past 2 seasons... other than that he's had an excellent career.

Again, thanks for the comments and helpful suggestions.

 

[Greenback]

Issue 1:
...
True to an extent. We never know how a goalie would have performed outside of their season. For their career stats, I'm showing how they performed against their peers. One could argue that the level of competition in Plante's era was higher than that of Roy's, or vice-versa.

I guess I am a little reluctant to validate the career comparisons as there are too many variables to account for once you go beyond a single season.

Issue 2:
...
I agree that I haven't accounted for rule changes directly, but the formula accounts for them indirectly. For example, let's look at the rule in the 50's that made players serve a full 2 minutes for penalties. Over the course of the season that would increase the powerplay opportunities in the league and decrease the average save percentage. Since we're comparing goalies to the league average that year, their save percentage will be lower, but so will the league average. Since we compare them to the league average, the rule changes won't affect their rating.

That is correct for a given season but, as I stated earlier, my issue is with GS being applied beyond a single season.

Issue 3:
...
Goals Saved assumes that all goals are equal. Personally, I'm not sure if that statement is true or not, one can make a strong case either way.

In any case, I don't think it's possible to take into account "money" goalies, statistically. However it definitely is worth considering, and it should be considered an intangible.

I would agree. Unfortunately, sources of evaluation in this regard become historical anecdotes and scouting reports rather than statistics.


This subject has always fascinated me. If we want to see how Hall or Sawchuk would do in this day and age, we would have to adjust their equipment, training habits, etc. It becomes daunting when you start to consider the details that must be adjusted to make a proper comparison.


On the whole Hockey Outsider, I like your way of thinking. You're new here (I am rather green on this MB as well), but it was this thread that compelled me to sign up. I have been on other hockey boards and, while they may be talking hockey, very few a) know what they are talking about and/or b) are willing to examine traditionally held beliefs about the game.

Here's to many discussions on this MB. They may be fruitless. They may be pointless. They may even be serious. In the end, may they always be entertaining.

 

[PecaFan]

In the '70's, the blue line face-off dot was moved from the center of the ice, affording the luxury of prime attacking position, to make two dots, one on either side.

I said PARDON? I'm assuming that's a typo, you're a few decades off.

 

[DownFromNJ]

Also, I would be very interested to consider how Brodeur's puckhandling affects the Devil's ability to score goals and/or give up fewer shots. I've toyed around with a few ideas but I haven't been able to come up with anything. Any ideas?

Brodeur's puckhandling ability is so above the level of everyone else that it changes the Devil's style of play. The Devils defensive system is entirely different when Scott Clemmenson is in net. Brodeur saves his defensemen from hits, and gets the puck away from the zone.

The Flyers were able to beat Brodeur by changing their system to cut down on his ability to stick handle, and were able to do so because of the injury-weakened NJ defensemen.

Goaltenders cannot be quantatively measured by their stats. This isn't baseball. You can quantatively measure baseball players by their stats because each event is an individiual and unique occurance that can be measured. In hockey, this is not the case. Sv% doesn't work because all shots are not alike. GAA and Wins are affected by teams. Some goalies create shots for themself. Thats my problem with Luongo. When I watch him, he puts himself in situations (by not controlling his rebounds) where players get extra shots against him. Thats why he broke the league record for saves in a season. Florida's defense wasn't that horrible (there were worse defenses who gave up more shots). But Luongo created more shots (and saved most of them, but still let in a substantial amount of goals).

Don't try to bring sabremetrics into hockey. I'm the biggest proponent of sabremetrics in baseball, but the statistics of the two sports are worlds apart.

 

[Greenback]

In the '72 Summit Series, the offside face-off dots were still in the middle of the ice.

 

[ChemiseBleuHonnete]

Take scoring chances, then count how many of those ended up as a shot on goal... Also how many of the scoring chances shots that have been stopeed. Then you can make a scoring chance saving %. That's a very good indicator. The rest goes into a routine shot saving percentage. Then, make a new stat called momentum shifting goals against. Momentum shifting goals are those that puts the opposite team within 1 goal of your team's lead, every goals that puts the opposite team to a 2 goals lead or any goal allowed in the last 5 minutes of regulation time.

A goalies stat should be :
Wins, Losses, (no tie assuming there won't be ties anymore in the nhl), Routine shot saving % (RSS%), Scoring chance saving % (SCS%) and Momentum shifting goals against average (MSGAA). With those stats, you should get a pretty good translation of actual play into stats.

 

[Hockey Outsider]

I guess I am a little reluctant to validate the career comparisons as there are too many variables to account for once you go beyond a single season.

That is correct for a given season but, as I stated earlier, my issue is with GS being applied beyond a single season.

I understand this concern... actually this point is very similar to what a friend told me a few weeks ago. But I think this is true of all career statistics. There's no way to know if Gretzky would have scored the same number of points in a different era (even if we just look at his "normalized" stats). I feel confident saying that, for example, Patrick Roy was 469 goals above average in his career... whether he would be able to maintain that performance in a different era is debateable.

I would agree. Unfortunately, sources of evaluation in this regard become historical anecdotes and scouting reports rather than statistics.

Yes!! Couldn't agree with you more. I think that clutch play is legitimate to take into account. However, most of the time it's just a couple of anecdotes, positive or negative, that will make or break a player's entire reputation.

For a great (short) article about "clutch players" go to http://ifyffe.bravejournal.com/ and scroll down to "How to build an entire career on 22 games, the Bill Ranford way".

This subject has always fascinated me. If we want to see how Hall or Sawchuk would do in this day and age, we would have to adjust their equipment, training habits, etc. It becomes daunting when you start to consider the details that must be adjusted to make a proper comparison.

On the whole Hockey Outsider, I like your way of thinking. You're new here (I am rather green on this MB as well), but it was this thread that compelled me to sign up. I have been on other hockey boards and, while they may be talking hockey, very few a) know what they are talking about and/or b) are willing to examine traditionally held beliefs about the game.

Here's to many discussions on this MB. They may be fruitless. They may be pointless. They may even be serious. In the end, may they always be entertaining.

Well said! Look forward to talking with you more on here.

 

[Hockey Outsider]

Goaltenders cannot be quantatively measured by their stats. This isn't baseball. You can quantatively measure baseball players by their stats because each event is an individiual and unique occurance that can be measured. In hockey, this is not the case.

A goalie's job is to stop shots. While there are other ways goalies can contribute to their team (puckhandling, clutch saves), we can measure how effectively a goalie is in stopping shots. This is their job.

As for the point about hockey not being a discrete game like baseball, I agree. However it doesn't make it impossible to measure things, it just makes it harder. And more fun.

Sv% doesn't work because all shots are not alike.

True to an extent but starting goalies face 1,500-2,000 shots per year. This is a large enough sample size (ie we can find a confidence interval of a few tenths of a percent) to judge their performance. For modern seasons we can refer to Alan Ryder's shot quality. For other years we will have to either disregard shot quality, or address it as an intangible.

Some goalies create shots for themself. Thats my problem with Luongo. When I watch him, he puts himself in situations (by not controlling his rebounds) where players get extra shots against him. Thats why he broke the league record for saves in a season. Florida's defense wasn't that horrible (there were worse defenses who gave up more shots). But Luongo created more shots (and saved most of them, but still let in a substantial amount of goals).

I've watched a lot of Panthers games the past few seasons and respectfully disagree with that conclusion.

The Panthers allowed more shots than any other team. They allowed 108 more shots more than the 2nd worst team and allowed 528 shots more than average. I find it hard to believe that Luongo's alleged lacked of rebound control caused 528 shots.

Don't try to bring sabremetrics into hockey. I'm the biggest proponent of sabremetrics in baseball, but the statistics of the two sports are worlds apart.

As I said before, baseball and hockey are very different sports. However, there are still many things that can be objectively measured and analyzed in hockey. My belief is that we can objectively learn all that we can, but it's still important to take subjective or intangible elements into account. As I said earlier, hockey is certainly harder to anaylze than baseball. But it's also more fun.

 

[trentmccleary]

Goaltenders cannot be quantatively measured by their stats. This isn't baseball. You can quantatively measure baseball players by their stats because each event is an individiual and unique occurance that can be measured. In hockey, this is not the case. Sv% doesn't work because all shots are not alike. GAA and Wins are affected by teams.

But in the end, stats are the only way to make sense of so many separate events (like say... 2,000 shots).
Scouting is nice in combination, but on it's own doesn't account for the prettiest/tools-iest players in the game blowing their careers because they're stupid.

I feel confident saying that, for example, Patrick Roy was 469 goals above average in his career... whether he would be able to maintain that performance in a different era is debateable.

Maybe I'm overcomplicating things, but couldn't a % of [goals saved/total goals allowed] or [goals saved/total goals allowed + goals saved] correct for that a little more?
Goalies could get more or less shots and allow more or less goals on average in different eras.

Also... the potential 2% difference you mentioned in terms of more shots affecting SV%. What kind of swing could that have? Because it sounds small until you realize that a SV% of .900 is technically 90%. A 2% swing to 92% makes you a Vezina contender. Or is it .898 - .902?

 

[Hockey Outsider]

Maybe I'm overcomplicating things, but couldn't a % of [goals saved/total goals allowed] or [goals saved/total goals allowed + goals saved] correct for that a little more?

That's a good idea, thanks. Not a huge change but the top 10 list becomes:

1 Ken Dryden 0.028
2 Dominik Hasek 0.022
3 Patrick Roy 0.016
4 Bernie Parent 0.014
5 Jacques Plante 0.013
6 Johnny Bower 0.012
7 Tony Esposito 0.011
8 Pete Peeters 0.010
9 Billy Smith 0.010
10 Glenn Hall 0.010

This list isn't necessarily showing the greatest goalies ever. It shows who saved more than the average goalie, on a per shot basis, over the course of their entire careers.

Goalies could get more or less shots and allow more or less goals on average in different eras.

I tried to account for that as best I could by adjusting for the number of shots a goalie would have faced on an average team. For example this year Luongo faced about 25% more shots than average (per game). So I lowered his scored 25% to account for the fact that he had far more opportunities to accumulate Goals Saved. I did this for each goalie each year.

Also... the potential 2% difference you mentioned in terms of more shots affecting SV%. What kind of swing could that have? Because it sounds small until you realize that a SV% of .900 is technically 90%. A 2% swing to 92% makes you a Vezina contender. Or is it .898 - .902?

Excellent point, I will try to clarify this.

When I said that there's a 2% impact on save percentage by shots per game, I mean that if we tried to predict save percentage solely as a function of shots per game, we'd get a correlation of 2%. This amount is very small so the largest component of save percentage must be determined by other factors (ie actual goaltender performance).

Conversely, the correlation between a goalie's win% and team stats they have (virtually) no control over, shots per game and goals per game, is 57% for 2004. So no matter how well or poorly a goalie plays, nearly 60% of their win% is determined by factors beyond their control. Furthremore, goals per game and shots allowed both have a statistically significant impact on win% at the 99% confidence level. This makes it a poor statistic, and especially poor when we can see that GAA and (especially) save percentage are far less team-dependent

 

[DownFromNJ]

True to an extent but starting goalies face 1,500-2,000 shots per year. This is a large enough sample size (ie we can find a confidence interval of a few tenths of a percent) to judge their performance. For modern seasons we can refer to Alan Ryder's shot quality. For other years we will have to either disregard shot quality, or address it as an intangible.

Goalies may face that many shots per year, but again not all shots are alike. Look the incredibly small differences between a great goalie's Sv% (.930) and an above average goalie's Sv% (.915).

Quality of shots can definately drop or raise someone's Sv% by enough to make a major difference. Also, some scorekeepers are notorious for giving a team more shots than the goalie actually had to save (I heard here that Boston used to give a lot of phantom shots).

 

[PecaFan]

In the '72 Summit Series, the offside face-off dots were still in the middle of the ice.

Hmm, I'll have to check my tapes, I don't remember that. Might have been an internation rules thing.

But regardless, in the NHL, the faceoff dots have been in the same spots since 1951. Total Hockey has a good reference on it.