Goalie Research: Wins Added

Ogopogo* · Jul 12, 2008

Outsider, as always I love your effort and passion for the game.

In looking at your formula it appears that the key distinction is the difference in save % between the subject and the league average. If I am understanding correctly, this doesn't take into account the quality of defense in front of the goalie. Save % is as much a team statistic as it is an individual statistic so not accounting for the great defensive team that a player like Roy was behind in Montreal will skew these results.

Correct me if I'm wrong in my understanding of your formula.

Bear of Bad News · Jul 12, 2008

Ogopogo said:
Correct me if I'm wrong in my understanding of your formula.

You're correct. You may enjoy the Alan Ryder paper found here:
http://hockeyanalytics.com/Research_files/Player_Contribution_System.pdf

Hockey Outsider · Jul 14, 2008

Ogopogo said:
If I am understanding correctly, this doesn't take into account the quality of defense in front of the goalie. Save % is as much a team statistic as it is an individual statistic so not accounting for the great defensive team that a player like Roy was behind in Montreal will skew these results.

That's correct. A goalie playing behind a poor defense (in the sense that they allow high-quality, dangerous shots) will have his performance understated.

overpass said:
The extra point for a shootout win is another matter. If you want to have a strict measure of regular season value, the most correct method would be to measure the goalie's shootout contribution, right? With your method, I suppose that would be to measure the goals scored by his team in the shootout, find the performance of the average goalie in the shootout, and compare actual shootout record to expected shootout record. I don't know if the Pythagorean formula applies in the shootout.

Personally, I don't mind the stat leaving out shootout contributions, as they have no value come playoff time. However, as your stat that measures value and is tied to wins, it seems like the shootout should be included, as it leads to wins.

Good point. I consider the numbers to primarily be backwards-looking (evaluating actual performance) rather than forward-looking (projecting what would happen ie in the playoffs). I'll need to do some further modifications to account for the shootouts in 2006-2008. Thanks for the suggestion.

Hand of Gaudreau · Jul 15, 2008

Hockey Outsider said:
How the Formula Works

* Warning this is very technical and boring

Step 1: calculate expected win percentage for a given team-year

Calculation: There is a very strong correlation (about 94%) between the Pythagorean formula, and actual win percentage. Thus, my estimate for how many games a goalie would have won is based on that formula. To calculate it, you need the teamâ€™s goals for per game [A], and the teamâ€™s shots against per game . To calculate what an average goalie would have done you need to look at the league average save percentage [C]. Plug it in to this formula: A^2 / [(A^2) + [(B^2) * (1-C)]]^2. That shows the expected win percentage based on how a goalie with a league-average save percentage would have performed, given the teamâ€™s level of offense and defense.

Example: In 1998, Dominik Hasek played for the Buffalo Sabres. His team scored an average of 2.52 goals per game [A], and allowed 30.6 shots against . The league average save percentage was 90.6% [C]. Plugging these numbers into the formula, a statistically average goalie is expected to win 43.4% of their decisions.

Step 2: calculate the number of games the goalie was expected to win

Calculation: take expected win percentage from step 1 and multiply it by minutes played, divided by sixty.

Example: In 1998, Hasek was expected to win 43.4% of his decisions. He played 4,220 / 60 = 70.3 games so he was expected to win 30.7 games.

Step 3: compare expected wins to actual wins

Calculation: subtract expected wins from actual wins

Example: in 1998, Hasek won 39.5 games but a statistically average goalie only would have won 30.7 games, had they played 70.3 games for the 1998 Sabres. Therefore Hasek singlehandedly won 8.8 games for the Sabres. (That might not sound like a lot, but, to put that into perspective, Buffalo finished 6th in the conference with 89 points. If Hasek was replaced with an average goalie, Buffalo would have lost the ~18 points that Hasek singlehandedly earned them, and they would have fallen to 10th in the conference, well out of the playoffs.

Step 4: adjust to an 82-game schedule

Calculation: take result of Step 3 * 82 games/ length of NHL schedule

Example: in 1998 the schedule was 82 games so no adjustments are necessary

Im not terribly brilliant would anybody be willing to work out how useful Kiprusoff was in the 05-06 season when he won the vezina?

Hockey Outsider · Jul 15, 2008

IggyTheKid said:
Im not terribly brilliant would anybody be willing to work out how useful Kiprusoff was in the 05-06 season when he won the vezina?

I have Kiprusoff ranked 1st in the league in 2006 with a very impressive +12.2 wins added. He finishes ahead of Brodeur, Vokoun, Luongo and Gerber.

I also have Kiprusoff ranked 2nd in 2004.

Bear of Bad News · Jul 15, 2008

I imagine that for purposes of calculating the league-average save percentage, you remove the goaltender in question's shots and saves from the emulsion. Can you confirm?

You had brought me around to this way of reasoning in the past, so I'm 99% sure that you're still doing it. But I didn't see it mentioned specifically.

Hockey Outsider · Jul 15, 2008

Doctor No said:
I imagine that for purposes of calculating the league-average save percentage, you remove the goaltender in question's shots and saves from the emulsion. Can you confirm?

You had brought me around to this way of reasoning in the past, so I'm 99% sure that you're still doing it. But I didn't see it mentioned specifically.

I should have removed the goalie's shots from the league average, but I didn't. (I forgot). I'm adding it to the list of adjustments to make (probably better to do them all at the same time).

Based on some quick calcs, I don't think it will affect a goalie's Wins Added by more than 0.2 wins per season. So I wouldn't expect any material changes, but the adjustment is definitely fair game.

Bear of Bad News · Jul 15, 2008

Yeah, especially with the pool as large as it is, it certainly won't make much of a difference. Thanks! :handclap:

Hockey Outsider · Jul 15, 2008

Updated version with the following changes:

- When calculating league average sv%, I remove the goalie's shots faced and goals allowed
- I use the exponent of 1.9 when calculating expected wins (optimized to somewhere between 1.9 and 1.91, but rounded for simplicity)
- I calculated expected wins based on actual number of goalie decisions * win% (as opposed to estimating how many decisions they "should have" based on ice time)
- I included 2008 stats
- For 2006-2008, the shootout has been excluded entirely due to a lack of data. All shootout wins and losses have been treated as ties. That still gives me a win percentage of around 52% (due to overtime wins/losses I think) so I've adjusted actual wins downwards by approx 4%. Note that I'm still not completely confident about the 2006-2008 numbers-- they still look too high to me, but they're much more reasonable than what I had before.

Despite all these changes the actual result haven't changed much:

Goalie|Decisions|Wins Added
Patrick Roy | 942 | 74.3
Dominik Hasek | 728 | 67.9
Martin Brodeur | 951 | 60.5
Curtis Joseph | 903 | 31.8
Ed Belfour | 954 | 28.5
Roberto Luongo | 458 | 27.1
Jean-Sebastien Giguere | 379 | 25.9
Daren Puppa | 411 | 21.5
Marty Turco | 333 | 20.4
Manny Legace | 272 | 19.4

Basically no change from before. Luongo is knocked down a few spots because he's benefited from shootout wins during the past few years (this adjustment was not taken into account on the chart on page 1).

Here's how they rank if the best goalie each year is awarded 10 pts, down to 1 point for the 10th best goalie:

Hard to overstate Roy's consistent greatness. His yearly rankings (starting in 1988) are 4th, 1st, 2nd, 3rd, 4th, NR, 3rd, 7th, NR, 3rd, 5th, 8th, 5th, 5th, 7th and 5th.

According to the system, Giguere, Backstrom and Brodeur were the deserving Vezina finalists in 2008. Nabokov (2nd in Vezina voting and 1st team all-star) is ranked 6th.

Big#D · Jul 16, 2008

Hockey Outsider said:
According to the system, Giguere, Backstrom and Brodeur were the deserving Vezina finalists in 2008. Nabokov (2nd in Vezina voting and 1st team all-star) is ranked 6th.

Could you show those results (top 10 say) for 2008 and maybe the top 3-5 for the last few years. I'd like to see how they compare to the Vezina voting.

Did you put 2008 in order (i.e. - if your calculations were used for the Vezina, would Giguere have won in 2008)?

Hockey Outsider · Jul 16, 2008

Hereâ€™s a ten year recap. Sometimes the results are very similar to actual Vezina voting, sometimes theyâ€™re way off. I want to emphasize that these lists are simply the outputs of the formula and are not necessarily my opinion.

1998

WA closely corresponds to the actual Vezina voting. The five Vezina finalists all finish in the top six as per WA.

1999

WA agrees that Hasek was the deserving Vezina winner but the rest of the finalists are poor selections. This is a classic example of GMs picking goalies with the most wins without considering the context. The Vezina finalists had their wins inflated due to playing on excellent offensive teams (Josephâ€™s Leafs were 1st in goal-scoring; Brodeurâ€™s Devils were 2nd; Tugnuttâ€™s Senators were 6th) and/or excellent defensive teams (Brodeurâ€™s Devils were 3rd in shots allowed; Tugnuttâ€™s Senators were 4th). Yes, they won a lot of games, but they were expected to, given the quality of the teams they played for.

2000

For the third straight year, WA agrees with the Vezina winner. The voting results are extremely close to the WA formula results. Even Turek (protected by the devastating MacInnis/Pronger duo) exceeds expectations, finishing 6th according to WA.

2001

Wow. Iâ€™m as surprised about Legace and Fernandez as you are. I know that backupâ€™s arenâ€™t supposed to win the Vezina but Legace went 24-5-5 on a Detroit team that was surprisingly weak defensively. Fernandezâ€™s record (19-17-4) looks average to the untrained eye; however, the Wild were dead last in offense (30th) and below-average (19th) in defense so he greatly exceeded expectations.

2002

Theodore turns in one of the greatest-ever regular season performances by a goalie. Montreal was ranked 27th in shots allowed and 19th in goals forâ€¦ an average goalie would have won 43% of his decisions but Theodore 55% of his. Roy (the 1st-team all-star) and Nabokov narrowly miss the Wins Added top five list.

2003

Giguere was virtually unknown back during the 2003 regular season despite leading the terrible Ducks to the playoffs (one of three goalies to appear in 60+ games with a save percentage of 92.0%). Obviously he made a name for himself that spring. The 2003 Senators were such a great team (2nd in goals for; 3rd in shots allowed) that WA views Lalimeâ€™s 39-20-7 as merely â€œmeeting expectationsâ€.

2004

These two lists are very similarâ€¦ except the GMs voted Brodeur as the Vezina winner while the system has him rated average. Brodeur gets credit for winning 59% of his games, but WA estimates that an average NHL starter also would have won around 59% of their games on New Jersey. Although Luongoâ€™s Panthers failed to make the playoffs, no goalie in history could have led them there (unless they stopped 94%+ of the shots they faced over 70 startsâ€¦ a feat so difficult no goalie has ever even approached that level of dominance).

2006

Quite similar. All five Vezina finalists have very strong scores according to WA.

2007

For the second straight year, WA agrees with the top two Vezina finalists. All five Vezina nominees have good WA scores. Surprisingly MA Fleury gets zero Vezina votes â€“ his 40-16-9 record is exactly the type of conventional statistic that the GMs usually take at face value. (Fleury, actually, is an interesting case. His save percentage is very mediocre but heâ€™s still 3rd in the league per WA â€“ evidence of clutch goaltending? Or did he benefit from Crosbyâ€™s clutch scoring?)

2008

Even with Brodeur, this might be the most obscure group of Vezina nominees since in two decades. I know very little about Backstrom, Thomas or Ellis but they all put up outstanding save percentages and won a lot of games on mediocre (or worse) teams. Brodeur wouldnâ€™t have been my choice for the Vezina but heâ€™s still a legitimate winner. (Heâ€™s only about 1 win behind Giguere). A lot of people criticized Giguere because he was protected by Pronger, Niedermayer and the Zero Line. However you need to look at both sides of the rink â€“ Anaheim had the 3rd worst offense in the league. Anaheimâ€™s poor offense more than offset their strong defense and the Ducks were expected to win about 47% of their games; Giguere went 35-17-6.

overpass · Jul 16, 2008

Again, very interesting stuff, Hockey Outsider. Thanks for posting this.

Who do you have as the best goalie(s) from 2000-2004? I'm always interested in who the best player or the best goalie in the world is at any given time. I expect Wins Added would see Roy as the best in the late 80s and early 90s, Hasek as the best in the rest of the 90s, and Brodeur as the best since the lockout. Conventional wisdom would have Brodeur as the best goalie from 2000-2004, but as he scores poorly in this system, who does Wins Added see as the best goalie over this period?

Hockey Outsider · Jul 16, 2008

2000-2004 leaders:

A great group of goalies, but nobody is too far ahead of the pack.

Noldo · Jul 17, 2008

This has been excellent read. Very interesting Hockey Outsider.

Is the formula something that could be tracked over the course of the season or does it require so large sample size (or much work) that it is better done after the season has ended?

Howe Elbows 9 · Jul 17, 2008

I'm a Sharks fan, so I would be interested in hearing your opinion about Nabokov's numbers. The narrow misses from the top 5 in 2002 and 2008 combined with third place in 2004 seem promising. Has he been consistently top 10 or has he had any bad years?

Judging from W% and SV%, 2003 and 2006 weren't too kind on him...

Hockey Outsider · Jul 17, 2008

Noldo said:
Is the formula something that could be tracked over the course of the season or does it require so large sample size (or much work) that it is better done after the season has ended?

It would probably take me about 1 hr to do this calculation mid-season... so I won't provide weekly updates but perhaps I can show the standings at the 20, 40 and 60 game marks.

Shlomo said:
I'm a Sharks fan, so I would be interested in hearing your opinion about Nabokov's numbers. The narrow misses from the top 5 in 2002 and 2008 combined with third place in 2004 seem promising. Has he been consistently top 10 or has he had any bad years?

2000: backup with 0 wins added in 11 games
2001: 6th, with 5 wins added
2002: 8th, with 2 wins added
2003: 5th worst, with -4 wins added
2004: rebounds well; 3rd with 5 wins added
2006: worst in the league with -7 wins added
2007: basically average (-1 win added)
2008: 6th, with 5 wins added

He looks inconsistent. When he's on his game he's one of the best goalies in the league (in four of his seven years as a starter, he's in the top 8), but he had two awful years.

In 2003 he was expected to win 49% of his decisions but he was 19-28-8 (90.6% save percentage is only a bit below the league average... did he give up a lot of poorly-timed goalies, was he bad in close games, or was he just unlucky)? In 2006, he was expected to win 60% of his decisions but went 16-19-7 (a terrible 88.5 sv% seems to be the main cause of this).

Shlomo said:
Judging from W% and SV%, 2003 and 2006 weren't too kind on him...

Goalies aren't explicitly evaluated based on their own save percentage... but that's usually a big reason why they outperform expectations. Example: if a goalie wins 65% of his starts on a team with middle-of-the-pack offense and defense, it's probably because he has a great save percentage. There are some exceptions though.

foame · Jul 17, 2008

Hockey Outsider said:
Interestingly, six of the top ten marks have come from 2006 and 2007, probably due to the fact that starting goalies are given an increasingly larger number of starts each year:

Goalie|Season|Wins Added
Martin Brodeur|2007|+15.1
Roberto Luongo|2007|+14.3
Miikka Kiprusoff|2006|+12.2
Curtis Joseph|1994|+11.5
Dominik Hasek|1997|+11.5
Martin Brodeur|2006|+10.8
Tomas Vokoun|2006|+10.6
Ed Belfour|1993|+10.2
Roberto Luongo|2006|+9.9
Patrick Roy|1989|+8.9

Can u post top 20 pre-lockout?

Hockey Outsider · Jul 17, 2008

foame said:
Can u post top 20 pre-lockout?

Sure. This is 1988-2004 (ie all the data I have pre-lockout):

Biggest changes post-lockout:
- Brodeur has jumped from 4th and joined Roy and Hasek to form the big three
- Luongo and Giguere have emerged as elite goalies

Lard_Lad · Jul 17, 2008

What does the other end of the table look like - who are the "Wins Subtracted" leaders for a single season and from 88-present?

Hockey Outsider · Jul 18, 2008

Lard_Lad said:
What does the other end of the table look like - who are the "Wins Subtracted" leaders for a single season and from 88-present?

Careers, 1988-present

Goalie|Decisions|Wins Subtracted
Marc Denis | 314 | -12
Kevin Weekes | 292 | -12
Damian Rhodes | 296 | -12.1
Craig Billington | 265 | -12.3
Steve Passmore | 79 | -12.6
Kirk McLean | 607 | -13
Pat Jablonski | 107 | -13.1
Patrick Lalime | 377 | -13.8
Grant Fuhr | 589 | -14
Peter Sidorkiewicz | 237 | -14.8
Glenn Healy | 417 | -16.4
Tim Cheveldae | 343 | -16.7
Dan Cloutier | 317 | -17.7
Don Beaupre | 388 | -18.2
Jeff Hackett | 471 | -18.3
Rick Tabaracci | 254 | -19.1
Tommy Salo | 512 | -19.3
Garth Snow | 327 | -19.8
Jamie McLennan | 238 | -20
Chris Terreri | 372 | -20.8

Generally these are career backup goalies but there are a few exceptions. Salo seems much lower than he should be (IMO) but his solid years in Edmonton can't offset poor play during the rest of his career. Keep in mind that Fuhr's numbers are only from 1988 onwards; his best years are not taken into account. Lalime is a classic example of a goalie that gets overrated because he won a lot of games on an elite team; he won far fewer games than he was expected to while on the Senators.

Single seasons, 1988-present

Goalie|Year|Wins Subtracted
Craig Billington | 1994 | -5.6
Damian Rhodes | 1997 | -5.7
Marc-Andre Fleury | 2006 | -5.7
Kay Whitmore | 1995 | -5.7
Pete Peeters | 1990 | -5.8
Chris Terreri | 1996 | -6.2
Mike Dunham | 2004 | -6.2
Olaf Kolzig | 1995 | -6.5
Don Beaupre | 1995 | -6.5
Tommy Soderstrom | 1994 | -6.6
Evgeni Nabokov | 2006 | -6.7
Kirk McLean | 1988 | -6.7
Jeff Hackett | 1993 | -6.8
Antero Niittymaki | 2007 | -6.9
Kelly Hrudey | 1998 | -7.1
Tim Cheveldae | 1995 | -7.4
Ken Wregget | 1988 | -7.4
Dan Cloutier | 2000 | -7.5
Peter Sidorkiewicz | 1993 | -8.1
Patrick Lalime | 2004 | -8.2

There are some pretty good goalies on this list. Peeters was way past his prime by 1990. The young Fleury rebounded nicely (-6 wins in 2006, then +7 and +3 the next two years). Nabokov, as I wrote about earlier today, is inconsistent but is a legitimate Vezina candidate when he's on. Don't take Kolzig's appearance too seriously -- he only played 14 games that year but his 2-8-2 record got prorated to something nasty because it was the lockout year.

Dan Cloutier posted a 9-30-3 record for the 2000 Lightning. That was a bad team, but he was still expected to win around 43% of his decisions (vs 25% in reality).

Peter Sidorkiewicz played on the expansion Senators in 1993. The numbers say that the Sens were below average (but decent) defensively, but keep in mind this doesn't take shot quality into account. Based on watching the Sens I have little doubt that they were, no better than the second-worst defensive team in the league. I'm not saying that Sidorkiewicz was a great goalie, but after 64 games on an expansion team that allowed countless breakaways and odd-man rushes, he's better than his Wins Added indicates.

The 2004 Senators were the best offensive team in the league, and were 3rd best defensively. Lalime was expected to win 67% of his starts and went just 25-23-7. (I don't think a 67% win expectancy is unrealistic, either, considering that career backup goalie Martin Prusek won 70% of his decisions, going 16-6-3).

Lard_Lad · Jul 18, 2008

Pretty good results, I think if you posted a "worst goalies of the last 20 years" poll you'd get quite a bit of overlap with that. In the single-season one, I see a lot of over-the-hill guys getting time based on reputation and young goalies that were still learning. Both cases where strictly merit-based playing time would have seen them playing less. Definitely seems like a worthwhile metric.

Goalie Research: Wins Added

Ogopogo*

Guest

Your Third or Fourth Favorite HFBoards Admin

Registered User

Gaudreaubey Baker

Registered User

Your Third or Fourth Favorite HFBoards Admin

Registered User

Your Third or Fourth Favorite HFBoards Admin

Registered User

__________________

Registered User

Registered User

Registered User

Registered User

Registered User

Registered User

Registered User

Registered User

Registered User

Registered User

Registered User

Ad

Ad

Ad