Excellent New Goalie Stat (Save Percentage Above League Average)

[zeke]

Thought this was good enough and relevant enough for more than just the stats forum.

IMO, this should immediately replace raw sv% in all goalie conversations.

One of the fundamental problems with stats is that league norms change over the years, and even from year to year. So its usually a good idea to try and adjust stats based on that year's league averages.

In baseball, they use OPS+ to adjust OPS to league average (where a 100ops+ means league average).

Hockey-reference has decided to do the same with SV%.....so we now have SV%+....a stat that adjusts sv% to the league average sv% every year.

http://www.hockey-reference.com/leagues/NHL_2013_goalies.html#stats::15

This is an immediate upgrade on sv% in all contexts, so go ahead and use it as often as possible.


For Leafs fans, for example, this can tell us that Reimer's club-record .924sv% last year (115sv%+) was actually not quite as impressive as Belfour's .922sv% (117sv%+) in '03.

Click to read more...

 

[HarrisonFord]

So 1.15SV+ = 15% above average?

 

[Bear of Bad News]

I've been doing exactly this for quite some time now. For instance:

http://hockeygoalies.org/bio/nhl/toronto.html
(scroll down to where save percentage has been calculated)

Z-Score tells how many standard deviations above (or below) league average a goaltender's performance was during the season.

GD tells how many goals a goaltender prevented during the season, above and beyond the league average (looking at the link above, it appears that H-R is doing this now, calling it GSAA).

GAR is the same as above, but comparing against a "replacement-level" goaltender (since GD would assign a value of zero to a league-average goaltender, but there's definite value in being average).

SNW% gives what the goaltender's winning percentage would be if he were on a team that (a) faces a league-average number of shots, and (b) scores a league-average number of goals. (SNW and SNL are then a goaltender's support-neutral wins and losses).

For more recent seasons, I also calculate a goaltender's game-to-game variance (as a measure of consistency) as well as the strength of schedule faced by the goaltender.

I describe these all in a bit more detail here: http://hockeygoalies.org/stats/glossary.html

 

[Ohashi_Jouzu*]

Yeah, I noticed that the other day. On the history board it's common practice to do the leg work one's self and use this in conversation (goalies are routinely compared across eras over there, so you pretty much have to), but it's nice to have an online stats database doing the grunt work for us now - for everyone at once.

 

[billybudd]

So basically, save percentage over the mean, if I"m understanding?

Seems reasonable.

 

[Garbage Goal]

That's actually the first I've heard of it. Yeah, I like that stat.

 

[HarrisonFord]

I've been doing exactly this for quite some time now. For instance:

http://hockeygoalies.org/bio/nhl/toronto.html

(scroll down to where save percentage has been calculated)

Z-Score tells how many standard deviations above (or below) league average a goaltender's performance was during the season.

GD tells how many goals a goaltender prevented during the season, above and beyond the league average.

GAR is the same as above, but comparing against a "replacement-level" goaltender (since GD would assign a value of zero to a league-average goaltender, but there's definite value in being average).

SNW% gives what the goaltender's winning percentage would be if he were on a team that (a) faces a league-average number of shots, and (b) scores a league-average number of goals. (SNW and SNL are then a goaltender's support-neutral wins and losses).

For more recent seasons, I also calculate a goaltender's game-to-game variance (as a measure of consistency) as well as the strength of schedule faced by the goaltender.

I describe these all in a bit more detail here: http://hockeygoalies.org/stats/glossary.html

By just glancing at the data, I think there's a pretty obvious trend of either all or none of the regular goalies being above/below 0. It would be interesting if there was some sort of way to measure how much of a goaltenders SV% was based on their own performance, and how much was based upon the defensive system playing in front of them.

 

[HarrisonFord]

So basically, save percentage over the mean, if I"m understanding?

Seems reasonable.

Honestly it makes sense. Measuring exceptional performance is comparing the yearly performance of a player compared to his peers. Which is part of the Gretzky is the greatest player ever argument. If you look at his performance compared to the mean, it's pretty staggering.

 

[Bear of Bad News]

By just glancing at the data, I think there's a pretty obvious trend of either all or none of the regular goalies being above/below 0. It would be interesting if there was some sort of way to measure how much of a goaltenders SV% was based on their own performance, and how much was based upon the defensive system playing in front of them.

That's true in many cases (although certainly not all), and it's something that we continue to investigate.

One improvement (that I don't present on my site) is normalizing a goaltender's save percentage for the number of shots faced in each manpower situation (even strength, power play, shorthanded). That helps to correct for goaltenders who face an inordinate amount of shorthanded situations.

 

[Grant]

I wish there was also an option to only show the starting goaltenders, or be able to eliminate all the goalies who played under a certain number of games. Because looking at the list, those that are above average is dominated by starting goaltenders (makes sense). Would be interested to see how much better/worse a goalie was compared to the other starting goaltenders around the league.

 

[Bear of Bad News]

One thing I note is that H-R's GSAA is slightly different than my comparable calculation (GD, or goal differential).

My guess is that for this calculation, H-R includes the goaltender in question's shots and saves in the league average calculation. I don't do that - my opinions is that if you have an exceptional goaltender, you want to compare him against everyone else (not everyone else including him).

But both ways can make sense, and reasonable folks can disagree.

 

[Michael Farkas]

Yeah, I was going to say, Taco has been providing this type of information and more on the HoH board for some time now.

With respect to H-R's work on this and stat guys everywhere, it's nothing new under the sun. It doesn't account for differences in modern goaltending and modern defensive tactics - not that anyone claimed it to be doing that. It does serve to help contextualize the numbers to the era a little bit. A .920 save pct. back in the day would win you a Vezina, now it's about average for a starter. I guess it helps in that calibration. Basically, echoing what O-J said.

If people are really into this kind of stuff, like I said (and now that I've scrolled up, like he said), Taco does a lot of math stuff that is way over my head: http://hfboards.mandatory.com/showpost.php?p=72335485&postcount=3

 

[HarrisonFord]

That's true in a lot of cases, and it's something that we continue to investigate.

One improvement (that I don't present on my site) is normalizing a goaltender's save percentage for the number of shots faced in each manpower situation (even strength, power play, shorthanded). That helps to correct for goaltenders who face an inordinate amount of shorthanded situations.

Yup. Might be worth trying to look at the historical SV% of different coaches over their careers as well, and trying to find a way to eliminate the system bias. Shorthanded bias is a good one to eliminate as well. I'd assume that the most accurate indicator of goaltender performance would come at even strength

 

[Bear of Bad News]

Yup. Might be worth trying to look at the historical SV% of different coaches over their careers as well, and trying to find a way to eliminate the system bias. Shorthanded bias is a good one to eliminate as well. I'd assume that the most accurate indicator of goaltender performance would come at even strength

Coaches are something that I use in my predictive model - it doesn't add a lot of lift, but it's definitely an effect.

 

[HarrisonFord]

Coaches are something that I use in my predictive model - it doesn't add a lot of lift, but it's definitely an effect.

True. Just like with everything, you'll have different coaches having a varying effect on it. Coaches that stick out to me in the game today are Tippett and MacLean.

Good on you for undertaking this, it seems like an interesting study. And probably an exhaustive one too on first glance to an outsider it probably seems so easy to accurately goaltender performance, but there's so many contributing factors to SV% outside of JUST the goalies abilities

 

[Bear of Bad News]

If people are really into this kind of stuff, like I said (and now that I've scrolled up, like he said), Taco does a lot of math stuff that is way over my head: http://hfboards.mandatory.com/showpost.php?p=72335485&postcount=3

Thanks! As for the last part, that just means that I need to explain it better.

Two of the barriers for any "advanced" statistic are (1) making it understandable and (2) making it accessible. And the term "advanced" doesn't help, either, because it scares a lot of people off.

Raw save percentage used to be considered an advanced statistic, and people were much more comfortable with wins/losses, goals-against average, and shutouts to measure a goaltender's performance. Over time, it's become part of the common parlance. Part of that is that people now have a good sense of the save percentage scale, what a "great" save percentage is, what a "lousy" save percentage is, and the levels in between.

I'm at the point where I'm comfortable working with the save percentage translations (GD, GAR, SNW%, Z-Score), but I'm still learning the nuances of the new metrics I've put together (for consistency and strength of opponent). I'm writing a short paper on the latter, so that should hopefully help people feel more comfortable with it.

 

[Hire Sather]

I'll watch the games

 

[Cashville]

Frankly, I think you need to go a step further and make it ES SV%+, but yeah it's good.

 

[Bear of Bad News]

Bringing a few things to the table to help show why save percentage translations are important.

Here's a list of all Vezina Trophy winners:
http://hockeygoalies.org/awards/vezina.html

Over time, you can see the raw (unadjusted) save percentages fluctuate, as periods of offense dominated, and periods of defense dominated. Did all of the goaltenders in the 1980s suck? Maybe, but unlikely.

If you look at the adjusted metrics (Z-Score, Goal Differential, and Goals Above Replacement), you can still see some outliers - this is purportedly the list of each year's best goalie, after all - but the pattern evens out a bit.

It's also easier to pick out the truly remarkable performances. Note Dominik Hasek's 93% (raw) save percentage in 1993-94 - if you look at his z-score, you can see that his performance was 4.6 standard deviations above the league average. What "z score" measures is the likelihood that an average goaltender would put together a season like the one in question (since we've all seen average goaltenders put together stretches of great play). A z-score of 4.6 means that an average goaltender would reproduce Hasek's 1993-94 season about once every 475,000 years.

If you click through (on the above page), you can see the individual goaltenders' careers evolve (under REGULAR SEASON STATISTICS or POSTSEASON STATISTICS). It helps to compare (for instance) Patrick Roy's career with Dominik Hasek's career with Martin Brodeur's career (and you can see that each was remarkable in his own way).

 

[13th Floor]

I'll watch the games

You must be busy.

 

[Bear of Bad News]

I'll watch the games

This is a common (and flippant) rejoinder, and I'm sure that you're quietly smiling to yourself about how you just "stuck it to the statheads".

However, "watching the games" is just as useless as "putting your head in a book" for properly evaluating players (both are necessary).

You should read Daniel Kahneman's book "Thinking Fast and Slow", which largely deals with narrative bias. Our brains are exceptionally biased, and we generally use whatever facts support our preconceived notions about players (and ignore anything that disagrees with those notions). It doesn't help that sports shows replay "highlight" goals and saves, which places a disproportionate share of those events into our memory stores.

We also don't have the ability to watch (and store in our brains) sample sizes for all goaltenders at once, let alone across multiple seasons.

Bottom line: for player evaluation, anyone who tells you that statistics are useless is just as wrong as anyone who tells you that watching the games is useless. You need *both* to properly evaluate.

 

[5 Minute Major]

I wish there was a stat kept for quality scoring chance save percentage. Maybe there is and I don't know that?

 

[RPGrizzly]

I've been doing exactly this for quite some time now. For instance:

http://hockeygoalies.org/bio/nhl/toronto.html
(scroll down to where save percentage has been calculated)

Z-Score tells how many standard deviations above (or below) league average a goaltender's performance was during the season.

GD tells how many goals a goaltender prevented during the season, above and beyond the league average (looking at the link above, it appears that H-R is doing this now, calling it GSAA).

GAR is the same as above, but comparing against a "replacement-level" goaltender (since GD would assign a value of zero to a league-average goaltender, but there's definite value in being average).

SNW% gives what the goaltender's winning percentage would be if he were on a team that (a) faces a league-average number of shots, and (b) scores a league-average number of goals. (SNW and SNL are then a goaltender's support-neutral wins and losses).

For more recent seasons, I also calculate a goaltender's game-to-game variance (as a measure of consistency) as well as the strength of schedule faced by the goaltender.

I describe these all in a bit more detail here: http://hockeygoalies.org/stats/glossary.html

Wow, thanks for this. I've always thought there was a lot more to the typical goalie stats you see, I had no idea the data was already out there! Great work, I'll definitely be spending some time on your website.

I don't have any problems with the SV%+ stat on H-R. It's pretty interesting.

 

[Bear of Bad News]

I wish there was a stat kept for quality scoring chance save percentage. Maybe there is and I don't know that?

The problem with that (from a technical perspective) is that "quality scoring chance" isn't well-defined. Do you know how some teams always lead the league in hits (or blocked shots) by a large margin? It's because a "hit" in Arena A is different from a "hit" in Arena B.

If the definition could be standardized, it'd be very useful (and it would be great to see).

 

[Bear of Bad News]

Wow, thanks for this. I've always thought there was a lot more to the typical goalie stats you see, I had no idea the data was already out there! Great work, I'll definitely be spending some time on your website.

I don't have any problems with the SV%+ stat on H-R. It's pretty interesting.

Thanks - appreciated! Let me know if you have questions; like I said, part of my goal is to get more people thinking about this sort of thing (since I believe that it leads to a more effective understanding of goaltenders).