Goalie analysis (2009-2012): Wins Added

[Bear of Bad News]

Thanks! I have Thomas at +6.7, -1.5 and +8.8 from 2009 to 2011, which is prety close to what you have. Different approaches, but it's good to see that we're in the same ballpark. How do you calculate SNWL?

It's actually very similar to what you do (although not as sophisticated):

SNWL% = (((1-[LG SV%])*[SA])^2 / ([GA]^2 + ((1-[LG SV%])*[SA])^2)

Or another way of saying - it's the pythagorean formula, calculating "goals for" as the number of goals a league-average goalie would allow when facing the same shots as Goalie X, and the "goals against" are the goalie's actual goals against. I remove Goalie X's statistics from the league-average calculation.

Then SNW = SNWL% * (Decisions), and SNL = Decisions - SNW.

 

[Hockey Outsider]

Thanks for the thoughtful comments.

What do you (or others) think are the biggest points of doing this study and publish its results?
I suppose it is supposed to help give a picture of about much influence different goaltenders have over their team's ability to win? Apart from that, isn't this mainly a curiosity stat?

The main purpose is this puts goaltending into the currency of winning, which is ultimately the goal of any team. Save percentage is a good statistic but it can be rather abstract - it's not immediately obvious how many games a goalie with a 93% save percentage would help his team win relative to a goalie with say a 92% save percentage.

Why not just calculate the "combined" save percentage for each goalie, and then the average "combined" save percentage for the league as a whole. Then normalize the results, so that the average goalie gets 1, and the other goalies get results relative to that. Very easy, and one can instantly know exactly how good a goalie performed compated to league average. Let's call it save ratio (I'm not North American and am not familiar with all the official terms used.)

I think you answered your own question later in the post - back in 2009, I also did a calculation of adjusted save percentage (adjusted for the league average). I don't think that either method (Wins Added or adjusted save percentage) is perfect, but they're both informative.

I like the idea of adjusting save percentage to a common benchmark, like 91%. I find it more intuitive - 94% is unheard of, 93% is excellent, 92% is good, 91% is average, etc.

This goes back to my point about presenting & selling your analysis. If I tell everyone that Hasek has a career average adjusted save percentage of 93.0% (compared to a league average of 91.0%), people immediately understand that he was as dominant as his name suggests. If I say his career save ratio is 1.022, many people (myself included) would find that less informative because it's an abstract ratio rather than something intuitive and understandable.

Shouldn't this simple method give you even "fairer" results than the one you use? (Fairer to the goalie, as we eliminate both his skaters' ability to score and his skaters' ability to prevent opponents from shooting. Without having thought carefully about it, I think you don't even have to exclude EN goals too, which if so would be another benefit.)

On one hand, if a goalie plays on a stronger team, it increases the number of games we'd expect him to win per the formula. However, it should also increase the number of games he'd actually win in real life, so he wouldn't be penalized.

On the other hand, if a goalie is lucky to play on a team that consistently scores clutch goals, then maybe his Wins Added are overstated. If a team wins a few close games by luck, the formula would give the goalie credit for his teammate's clutch scoring.

Finally, don't forget that home and away matters a lot, as team's generally win more at home than on the road. If you're not able to account for that in your formulas, you can list number of home games in a column and let the readers make their own adjustments. (There may have been cases where certain goalies played unproportional shares of home games.)

I'm skeptical. I know that teams generally play better and win more at home, but how many goalies are going to have a significant difference between home and away starts? I don't have the data to look into this, but if there are major differences let me know.

If you'll continue to prefer your current method, maybe you also should use pythagoran math for both expected and factual, to avoid mixing apples and oranges (and perhaps also to avoid having to deal with OT wins/losses).

What do you mean expected and factual?

 

[Hockey Outsider]

Excellent work, and I find the results very interesting. You have Halak and Theodore down in the minus section though (2012 analysis), which I assume is a cut and paste/formatting error of some sort.

I might adopt this as my primary statistical measuring "tool" for a goalie's season on the whole. I like that SV% plays a fundamental part, and that games played and special teams influence the results further as one should "expect".

Thank you. Good catch - the numbers I originally posted were correct, I just had them sorted incorrectly. I've update the 2012 post.

There are many different ways to measure goalie performance, but something taking into account save percentage, special teams influence and workload should be fairly accurate.

It's actually very similar to what you do (although not as sophisticated):

SNWL% = (((1-[LG SV%])*[SA])^2 / ([GA]^2 + ((1-[LG SV%])*[SA])^2)

Or another way of saying - it's the pythagorean formula, calculating "goals for" as the number of goals a league-average goalie would allow when facing the same shots as Goalie X, and the "goals against" are the goalie's actual goals against. I remove Goalie X's statistics from the league-average calculation.

Then SNW = SNWL% * (Decisions), and SNL = Decisions - SNW.

Great, thanks. That method makes sense to me. Do you have a list of single season and/or career leaders?

 

[Doomsday Device]

Having Smith at the top makes me question the stats.

Phoenix has quality goaltending year after year. It makes more sense that its the system then that they consistently get new goalies that are best in the league.

Bryzgalov's best even strength save percentage actually came with Gretzky as his coach, and I don't think he and Tippett run their teams all that similarly.

Smith actually had an great season back in 08-09 as well when he had an even strength of save percentage of 0.931 with the Lightning. So he's played well in the past without Tippett's "system" but clearly has some significant consistency issues. I don't think the value of this system is lessened just because his strong performance this year is so unexpected.

 

[plusandminus]

Thanks for the thoughtful comments.

Thanks.

I'm not sure you understood me fully (or I understood you). But if you're happy with your method the way it is, it's OK.

I mentioned the importance of separating home games from away games, and (just like another poster often does when I point out things like these) you automatically seemed to assume I probably was wrong. That is a phenomenom I often encounter on this board when I try to point out less obvious things that apparantly goes against more or less fixed beliefs.

I'm skeptical. I know that teams generally play better and win more at home, but how many goalies are going to have a significant difference between home and away starts? I don't have the data to look into this, but if there are major differences let me know.

Just a few examples from the 2008-09 to 2010-11 ("2008" is in this case 2008-2009):

Seas|Team|Name|hGP|aGP|Diff
2009|NAS|Pekka Rinne|38|20|18
2008|FLO|Tomas Vokoun|36|23|13
2010|MIN|Niklas Backstrom|32|19|13
2010|PHO|Ilya Bryzgalov|40|28|12
2009|MIN|Niklas Backstrom|35|25|10
2010|TBL|Dwayne Roloson|22|12|10
2008|SJS|Brian Boucher|6|16|-10
2009|VAN|Andrew Raycroft|5|16|-11
2008|OTT|Alex Auld|16|27|-11
2009|STL|Ty Conklin|7|19|-12
2010|PHO|Jason LaBarbera|2|15|-13
2010|MIN|Jose Theodore|8|24|-16
2009|NAS|Dan Ellis|6|25|-19

The data shoud show not only starts, but all games a goalie actually played in.
Data is mainly from hockeyreference. (I have spent many hours putting it into my own database, which means I have data for all seasons from 1987-88 to 2010-11 and can do more or less advanced studies on it.)

Do you still think this is not a significant difference?

If you don't trust me and my stats, I can give you a link to hockeyreference:
http://www.hockey-reference.com/players/r/rinnepe01/splits/2010/
And what do we see?

Name|Home/Away|GP|Save%|GAA
Pekka Rinne|Home|38|.921|2.31
Pekka Rinne|Away|20|.889|3.21

Quite a significant difference, wouldn't you say?

We can convert the save_percentage to goals_allowed_percentage, by the simple formula of 1 - save_percentage. At home, he allows 7.9 % of all shots, and on the road he allows 11.1 % of all shots. That's 1.4 times more.


You mentioned presentation. Well, the presentations of official goalie stats may look appealing to North Americans (I cannot comment upon that), but it comes with the price of over simplifying things. You already were aware that situational play "bias" the official stats presented. Now you hopefully are aware that home and away play also does it. (Yet another thing is strength of opposition.)

I think you thus provide mainly just yet another more or less biased study. Following the advices I gave you would (in my opinion) have improved it.


Regarding normalizing save percentage:

This goes back to my point about presenting & selling your analysis. If I tell everyone that Hasek has a career average adjusted save percentage of 93.0% (compared to a league average of 91.0%), people immediately understand that he was as dominant as his name suggests. If I say his career save ratio is 1.022, many people (myself included) would find that less informative because it's an abstract ratio rather than something intuitive and understandable.

In itself, save percentage says nothing about how good the goalie has performed (save percentage wise) compared to other goalies in the league. A save percentage of .890 can be very good or below average depending on era and context. Adding a column showing normalized percentage thus would add information, especially when comparing seasons.

I actually prefer goal_allowance_percentage (which is 1 minus save_percentage), as I find it more telling and better shows the actual difference between having a great goalie and a poor. Let's compare save_percentages of .93 and .86, which means goal_allowance_percentages of .07 and .14. The .93 goalie in that regard is twice as good as .86, as the .93 goalie allows half the number of goals that the .86 goalie allows (given the faced the same amount of shots). The worse goalie allows 14 goals when the better one allows 7. This is to me a telling stat, that immediately gives an indication of how team and skater stats are affected by goalie performance.

I'm not sure how much you have used normalized save percentage, but I'm sure you would find it very useful to include in your goaltending formulas.

The main purpose is this puts goaltending into the currency of winning, which is ultimately the goal of any team. Save percentage is a good statistic but it can be rather abstract - it's not immediately obvious how many games a goalie with a 93% save percentage would help his team win relative to a goalie with say a 92% save percentage.

On one hand, if a goalie plays on a stronger team, it increases the number of games we'd expect him to win per the formula. However, it should also increase the number of games he'd actually win in real life, so he wouldn't be penalized.

I tried to show you a (in my opinion) simplier, but yet maybe even more "fair"/"accurate" (to the goalies) way of doing it. I haven't dug deep into this, but my suggestions would be "fairer" to the goalies. In your presentation, you focus on goalies, and rank goalies, as if individual goalie importance is the key thing you're after. If you want to "rank" goalies, I don't think your method is as good as what I suggested. To me, it at this points looks as if you have done a combined "goalie and skaters performance" study, but present it as being a goalie study.
My method would "isolate" goalie performance, by putting all goalies into an average team. But if one wants to focus on it the way you do - which according to comments in the thread appears to be popular - then it's of course OK to do so.


Regarding "not mixing apples and oranges"...

What do you mean expected and factual?

a. We can use pythagoran win formula to calculate expected amount of wins if having an average goalie.
b. We can also use it to calculate expected amount of wins based on the particular goalie's actual stats.
c. We can also use factual wins ("decisions").
In this case, I would probably prefer to use a and b, or possibly a and a combined b/c.
I haven't dug into this more than shallowly.

The NHL has a very strange way of rewarding draws, by handing out an extra point in games where teams succeeds in having a draw after 60 minutes of play. Best is to always hand out 2 points (or always 3 points, as they do internationally) in games. I personally recalculate points when doing studies based on team performance, for example to 2, 1.5, 0.5, 0 rather than the bizarre 2, 2, 1, 0 system currently in use.
In this particular study of yours, shootouts will be special. Normally, they are often considered sort of a "lottery", but in this study - where the focus is on goaltending - it's a bit different.

 

[Bear of Bad News]

I'm not sure you grasped, or even wanted to grasp, all I wrote.

That's not particularly nice. Is that how you intended it to sound?

 

[Bear of Bad News]

Do you have a list of single season and/or career leaders?

I do - and I think it will be relatively easy to pull out of Access (the tricky part is turning it into HFBoards tables). I'm headed to the mountains now, but will give it a shot this evening.

 

[plusandminus]

That's not particularly nice. Is that how you intended it to sound?

I honestly don't know. I'm not used to communicate in English so maybe it came out as less friendly as I intended it too. I think he was polite and nice towards me, and I'm sorry if I came out differently towards him. I have rewritten it now.

 

[steve141]

Thanks for doing this work, Hockey Outsider.

Just a few examples from the 2008-09 to 2010-11 ("2008" is in this case 2008-2009):

Seas|Team|Name|hGP|aGP|Diff
2009|NAS|Pekka Rinne|38|20|18
2008|FLO|Tomas Vokoun|36|23|13
2010|MIN|Niklas Backstrom|32|19|13
2010|PHO|Ilya Bryzgalov|40|28|12
2009|MIN|Niklas Backstrom|35|25|10
2010|TBL|Dwayne Roloson|22|12|10
2008|SJS|Brian Boucher|6|16|-10
2009|VAN|Andrew Raycroft|5|16|-11
2008|OTT|Alex Auld|16|27|-11
2009|STL|Ty Conklin|7|19|-12
2010|PHO|Jason LaBarbera|2|15|-13
2010|MIN|Jose Theodore|8|24|-16
2009|NAS|Dan Ellis|6|25|-19

Interesting break-down. Since we know what an advantage home-ice can be this would definitely be an interesting aspect to add to the formula.

Another similar factor is playing on back-to-back nights. We know that teams generally perform worse on the second night of back-to-backs, so the expected win rate should be lower on those games.

This is relevant for goalies, since many coaches tend to use the backup goalie on one of the back-to-back games. As an example, this year Jimmy Howard played 13% of his games on the second game of a back-to-back, while his backup Ty Conklin played 33% of his games on the second night. With those statistics in mind it wouldn't be fair to Conklin to expect him to win at the same rate as Jimmy Howard.

 

[Hockey Outsider]

I mentioned the importance of separating home games from away games, and (just like another poster often does when I point out things like these) you automatically seemed to assume I probably was wrong. That is a phenomenom I often encounter on this board when I try to point out less obvious things that apparantly goes against more or less fixed beliefs.

Don't take it personally. I simply said that I was skeptical and I wanted evidence. You gave me evidence, so now I will consider it.

Do you still think this is not a significant difference?

It looks like it can be a significant difference. The general rule of thumb is that teams play about 10% better at home. I'm sure you would agree that a goalie who has 30 home starts and 30 road starts would not be affected by home/away differences (30/60*10% + 30/60*-10% = 0.0%).

In the case of Pekka Rinne in 2010: (38/58*10% + 20/58*-10% = 3.1%). Currently, Rinne has +4.6 Wins Added. If I revise his expected goals for and goals against by 3.1%, he gets credit for +2.8 Wins Added (a decrease of 1.8 wins).

In the second most extreme case (Vokoun in 2009), a similarly rough calculation would increase him from +1.9 to +0.6 (a decrease of 1.3 wins).

If we look at the goalie with the largest away differential (Ellis in 2009), his score improves from -0.7 to +0.3 (an extra 1.0 win).

With the exception of Rinne, it looks like the other adjustments would be 1.3 wins or less. This is where the cost-benefit trade-off comes in - I'm not willing to spend countless hours collecting data and updating my formula for something that will, in the overwhelming majority of cases, impact the final outcome by less than 1 win.

If you feel strongly about this, I can send you my spreadsheets and you can update them to take the home/road split into account. I'd be interested in seeing the impact, though I suspect that it would be less than 1.0 Wins Added in the vast majority of cases.

You mentioned presentation. Well, the presentations of official goalie stats may look appealing to North Americans (I cannot comment upon that), but it comes with the price of over simplifying things. You already were aware that situational play "bias" the official stats presented. Now you hopefully are aware that home and away play also does it. (Yet another thing is strength of opposition.)

I disagree. Let's say there are three options for conveying Hasek's career save percentage:

A. He had a 93.0% adjusted Sv% compared to an adjusted league average of 91.0%

B. He had a ratio of 1.022 compared to a league average of 1.000

C. He had a ratio of -7,239.4 compared to a league average of -7,083.8

In all three cases, the same information is conveyed (namely, Hasek stopped the puck 2.2% more effectively than the league average over the course of his career).

I think any hockey fan (regardless of their nationality) will find that A is more informative than B, and B is more informative than C.

I think you thus provide mainly just yet another more or less biased study. Following the advices I gave you would (in my opinion) have improved it.

No study is ever completely free from bias - there will always be factors that are not possible to quantify. However, this is the first rigorous study I've seen on HFBoards that separates ES, PP and PK shots, so for that reason I think it's a step forward.

To reiterate my previous point - taking home/road splits into account would improve this, but the benefit (more accurate information) appears to be very small and in my mind is not worth the cost (hours of manually searching for and compiling the raw data).

In itself, save percentage says nothing about how good the goalie has performed (save percentage wise) compared to other goalies in the league. A save percentage of .890 can be very good or below average depending on era and context. Adding a column showing normalized percentage thus would add information, especially when comparing seasons.

See my thread about normalized save percentage - I state that every goalie is normalized to a season with a 90.5% league average.

I actually prefer goal_allowance_percentage (which is 1 minus save_percentage), as I find it more telling and better shows the actual difference between having a great goalie and a poor. Let's compare save_percentages of .93 and .86, which means goal_allowance_percentages of .07 and .14. The .93 goalie in that regard is twice as good as .86, as the .93 goalie allows half the number of goals that the .86 goalie allows (given the faced the same amount of shots). The worse goalie allows 14 goals when the better one allows 7. This is to me a telling stat, that immediately gives an indication of how team and skater stats are affected by goalie performance.

Again, it goes back to the presentation of the data. I agree that goal allowance percentage is somewhat more meaningful than save percentage, but it's less intuitive.

I tried to show you a (in my opinion) simplier, but yet maybe even more "fair"/"accurate" (to the goalies) way of doing it. I haven't dug deep into this, but my suggestions would be "fairer" to the goalies. In your presentation, you focus on goalies, and rank goalies, as if individual goalie importance is the key thing you're after. If you want to "rank" goalies, I don't think your method is as good as what I suggested. To me, it at this points looks as if you have done a combined "goalie and skaters performance" study, but present it as being a goalie study.

I probably didn't explain myself clearly. Let's say a goalie plays for an average team that allows 30 shots per game and scores 2.5 goals per game. Ceteris paribus we'd expect the goalie to win 50% of his games.

Let's say the team acquires Stamkos, and instead they score 3.0 goals per game. We'd expect the team to win more, perhaps around 59% of its games.

Based on the way the formula works, we'd expect the goalie to in the second scenario to win more games. In reality he actually would win more games because he plays for a stronger team! In both cases, the formula looks at how many games that goalie won, above and beyond those he was expected to win. A goalie can get a high (or low) score on good or bad teams.

The NHL has a very strange way of rewarding draws, by handing out an extra point in games where teams succeeds in having a draw after 60 minutes of play. Best is to always hand out 2 points (or always 3 points, as they do internationally) in games. I personally recalculate points when doing studies based on team performance, for example to 2, 1.5, 0.5, 0 rather than the bizarre 2, 2, 1, 0 system currently in use.
In this particular study of yours, shootouts will be special. Normally, they are often considered sort of a "lottery", but in this study - where the focus is on goaltending - it's a bit different.

The NHL's decision to make some games worth 2 points and other games worth 3 is ridiculous. I never saw anything wrong with a tie - though if the shootout is here to stay, I'd make each game worth 5 points (5 for a regulation win, 4 for an OT win, 3 for a SO win, 2 for a SO loss, 1 for an OT loss, 0 for a regulation loss). But that's a discussion for another thread.

 

[Hockey Outsider]

Thanks for doing this work, Hockey Outsider.



Interesting break-down. Since we know what an advantage home-ice can be this would definitely be an interesting aspect to add to the formula.

Another similar factor is playing on back-to-back nights. We know that teams generally perform worse on the second night of back-to-backs, so the expected win rate should be lower on those games.

This is relevant for goalies, since many coaches tend to use the backup goalie on one of the back-to-back games. As an example, this year Jimmy Howard played 13% of his games on the second game of a back-to-back, while his backup Ty Conklin played 33% of his games on the second night. With those statistics in mind it wouldn't be fair to Conklin to expect him to win at the same rate as Jimmy Howard.

Thanks for the comments.

That's a good point about back-to-back games. I don't have any numbers about which goalies have played in the second night of back-to-back games (or possibly the 3rd game in 4 nights). I think it could impact the numbers (though I'm not sure of the extent of the impact). Unless someone has data, I think we'll have to treat it as a valid consideration, albeit one that can't be quantified.

 

[DJ Man]

I'd make each game worth 5 points (5 for a regulation win, 4 for an OT win, 3 for a SO win, 2 for a SO loss, 1 for an OT loss, 0 for a regulation loss). But that's a discussion for another thread.

That's 'way too logical.

The whole "points" concept could have been avoided by recording half-wins, but fear of fractions prevailed.

I think that the NHL brass figured that if they made a win worth even three points, some hallowed record for team points would fall too easily. Of course, the extended schedule guaranteed that would happen anyway, sooner or later.

Teams should really be playing for a prize of fixed value. Allowing them to manufacture an extra point out of thin air is just ridiculous.

(Or maybe it's a Canadian tradition: after all, in the CFL you get a point for missing a field goal. In the NHL, you get a point for losing, but not losing too swiftly.)

(Sorry for the digression!)

 

[Dooman]

Edmonton didn't have a goalie in 2012?

 

[Hockey Outsider]

That's 'way too logical.

The whole "points" concept could have been avoided by recording half-wins, but fear of fractions prevailed.

I think that the NHL brass figured that if they made a win worth even three points, some hallowed record for team points would fall too easily. Of course, the extended schedule guaranteed that would happen anyway, sooner or later.

Teams should really be playing for a prize of fixed value. Allowing them to manufacture an extra point out of thin air is just ridiculous.

(Or maybe it's a Canadian tradition: after all, in the CFL you get a point for missing a field goal. In the NHL, you get a point for losing, but not losing too swiftly.)

(Sorry for the digression!)

I'd be in favour of either a five-point system like I described above, or a three-point system (three for a regulation win, two for an OT/SO win, one for an OT/SO loss, and none for a regulation loss).

I know that a lot of team records would fall, but I don't think that team records are directly comparable anyway. During the 1990's only the best teams would reach 100 points in a season - on average there were about three per year. Since the lockout, there have been around nine teams per year to reach triple digits. The standards for excellent (and good, average, bad, etc) have all changed anyway.

Edmonton didn't have a goalie in 2012?

I only posted results for goalies with 45+ decisions, so none of them met that threshold.

Dubnyk had +0.4 Wins Added. He had a good 92.7% even strength save percentage (tied for 11th among goalies 29 with 40+ decisions), but a poor 85.4% powerplay save percentage (tied for 5th worst).

Khabibulin had a league worst-5.8 Wins Added. He was one of only three goalies (who had 30+ decisions) who had win percentage below 40% - the other two sub-40% goalies played for Columbus. This is much worse than expected on a team that was merely below average offensively (19th in goals for) and defensively (19th in shots against).

 

[Dooman]

So by your metric, Edmonton had the worst goaltending 3 years in a row.

How freaking lovely. lmao. And people wonder how we earned those 3 #1 picks?