Excellent New Goalie Stat (Save Percentage Above League Average)

Discussion in 'By The Numbers' started by zeke, Oct 10, 2013.

View Users: View Users
  1. zeke

    zeke #freeVladdy

    Joined:
    Mar 14, 2005
    Messages:
    47,595
    Likes Received:
    5,827
    Trophy Points:
    201
    Thought this was good enough and relevant enough for more than just the stats forum.

    IMO, this should immediately replace raw sv% in all goalie conversations.

    One of the fundamental problems with stats is that league norms change over the years, and even from year to year. So its usually a good idea to try and adjust stats based on that year's league averages.

    In baseball, they use OPS+ to adjust OPS to league average (where a 100ops+ means league average).

    Hockey-reference has decided to do the same with SV%.....so we now have SV%+....a stat that adjusts sv% to the league average sv% every year.

    http://www.hockey-reference.com/leagues/NHL_2013_goalies.html#stats::15

    This is an immediate upgrade on sv% in all contexts, so go ahead and use it as often as possible.


    For Leafs fans, for example, this can tell us that Reimer's club-record .924sv% last year (115sv%+) was actually not quite as impressive as Belfour's .922sv% (117sv%+) in '03.
     
  2. HarrisonFord

    HarrisonFord Mindawakebodyasleep

    Joined:
    Jul 20, 2011
    Messages:
    21,006
    Likes Received:
    457
    Trophy Points:
    124
    Location:
    Toronto
    So 1.15SV+ = 15% above average?
     
  3. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,106
    Likes Received:
    1,389
    Trophy Points:
    139
    Location:
    Windsor
    I've been doing exactly this for quite some time now. For instance:

    http://hockeygoalies.org/bio/nhl/toronto.html
    (scroll down to where save percentage has been calculated)

    Z-Score tells how many standard deviations above (or below) league average a goaltender's performance was during the season.

    GD tells how many goals a goaltender prevented during the season, above and beyond the league average (looking at the link above, it appears that H-R is doing this now, calling it GSAA).

    GAR is the same as above, but comparing against a "replacement-level" goaltender (since GD would assign a value of zero to a league-average goaltender, but there's definite value in being average).

    SNW% gives what the goaltender's winning percentage would be if he were on a team that (a) faces a league-average number of shots, and (b) scores a league-average number of goals. (SNW and SNL are then a goaltender's support-neutral wins and losses).

    For more recent seasons, I also calculate a goaltender's game-to-game variance (as a measure of consistency) as well as the strength of schedule faced by the goaltender.

    I describe these all in a bit more detail here: http://hockeygoalies.org/stats/glossary.html
     
  4. Ohashi_Jouzu*

    Ohashi_Jouzu* Registered User

    Joined:
    Apr 2, 2007
    Messages:
    30,332
    Likes Received:
    5
    Trophy Points:
    0
    Location:
    Halifax
    Yeah, I noticed that the other day. On the history board it's common practice to do the leg work one's self and use this in conversation (goalies are routinely compared across eras over there, so you pretty much have to), but it's nice to have an online stats database doing the grunt work for us now - for everyone at once.
     
  5. billybudd

    billybudd 1for the other thumb

    Joined:
    Feb 1, 2012
    Messages:
    20,949
    Likes Received:
    750
    Trophy Points:
    139
    So basically, save percentage over the mean, if I"m understanding?

    Seems reasonable.
     
  6. Garbage Goal

    Garbage Goal Registered User

    Joined:
    Apr 1, 2009
    Messages:
    20,690
    Likes Received:
    1,456
    Trophy Points:
    139
    That's actually the first I've heard of it. Yeah, I like that stat.
     
  7. HarrisonFord

    HarrisonFord Mindawakebodyasleep

    Joined:
    Jul 20, 2011
    Messages:
    21,006
    Likes Received:
    457
    Trophy Points:
    124
    Location:
    Toronto
    By just glancing at the data, I think there's a pretty obvious trend of either all or none of the regular goalies being above/below 0. It would be interesting if there was some sort of way to measure how much of a goaltenders SV% was based on their own performance, and how much was based upon the defensive system playing in front of them.
     
  8. HarrisonFord

    HarrisonFord Mindawakebodyasleep

    Joined:
    Jul 20, 2011
    Messages:
    21,006
    Likes Received:
    457
    Trophy Points:
    124
    Location:
    Toronto
    Honestly it makes sense. Measuring exceptional performance is comparing the yearly performance of a player compared to his peers. Which is part of the Gretzky is the greatest player ever argument. If you look at his performance compared to the mean, it's pretty staggering.
     
  9. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,106
    Likes Received:
    1,389
    Trophy Points:
    139
    Location:
    Windsor
    That's true in many cases (although certainly not all), and it's something that we continue to investigate.

    One improvement (that I don't present on my site) is normalizing a goaltender's save percentage for the number of shots faced in each manpower situation (even strength, power play, shorthanded). That helps to correct for goaltenders who face an inordinate amount of shorthanded situations.
     
  10. Grant

    Grant LL Genius

    Joined:
    Jan 16, 2012
    Messages:
    14,193
    Likes Received:
    0
    Trophy Points:
    84
    Location:
    London
    I wish there was also an option to only show the starting goaltenders, or be able to eliminate all the goalies who played under a certain number of games. Because looking at the list, those that are above average is dominated by starting goaltenders (makes sense). Would be interested to see how much better/worse a goalie was compared to the other starting goaltenders around the league.
     
  11. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,106
    Likes Received:
    1,389
    Trophy Points:
    139
    Location:
    Windsor
    One thing I note is that H-R's GSAA is slightly different than my comparable calculation (GD, or goal differential).

    My guess is that for this calculation, H-R includes the goaltender in question's shots and saves in the league average calculation. I don't do that - my opinions is that if you have an exceptional goaltender, you want to compare him against everyone else (not everyone else including him).

    But both ways can make sense, and reasonable folks can disagree.
     
  12. Mike Farkas

    Mike Farkas Grace Personified

    Joined:
    Jun 28, 2006
    Messages:
    9,017
    Likes Received:
    1,919
    Trophy Points:
    156
    Location:
    PA
    Yeah, I was going to say, Taco has been providing this type of information and more on the HoH board for some time now.

    With respect to H-R's work on this and stat guys everywhere, it's nothing new under the sun. It doesn't account for differences in modern goaltending and modern defensive tactics - not that anyone claimed it to be doing that. It does serve to help contextualize the numbers to the era a little bit. A .920 save pct. back in the day would win you a Vezina, now it's about average for a starter. I guess it helps in that calibration. Basically, echoing what O-J said.

    If people are really into this kind of stuff, like I said (and now that I've scrolled up, like he said), Taco does a lot of math stuff that is way over my head: http://hfboards.mandatory.com/showpost.php?p=72335485&postcount=3
     
  13. HarrisonFord

    HarrisonFord Mindawakebodyasleep

    Joined:
    Jul 20, 2011
    Messages:
    21,006
    Likes Received:
    457
    Trophy Points:
    124
    Location:
    Toronto
    Yup. Might be worth trying to look at the historical SV% of different coaches over their careers as well, and trying to find a way to eliminate the system bias. Shorthanded bias is a good one to eliminate as well. I'd assume that the most accurate indicator of goaltender performance would come at even strength
     
  14. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,106
    Likes Received:
    1,389
    Trophy Points:
    139
    Location:
    Windsor
    Coaches are something that I use in my predictive model - it doesn't add a lot of lift, but it's definitely an effect.
     
  15. HarrisonFord

    HarrisonFord Mindawakebodyasleep

    Joined:
    Jul 20, 2011
    Messages:
    21,006
    Likes Received:
    457
    Trophy Points:
    124
    Location:
    Toronto
    True. Just like with everything, you'll have different coaches having a varying effect on it. Coaches that stick out to me in the game today are Tippett and MacLean.

    Good on you for undertaking this, it seems like an interesting study. And probably an exhaustive one too :laugh: on first glance to an outsider it probably seems so easy to accurately goaltender performance, but there's so many contributing factors to SV% outside of JUST the goalies abilities
     
  16. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,106
    Likes Received:
    1,389
    Trophy Points:
    139
    Location:
    Windsor
    Thanks! As for the last part, that just means that I need to explain it better.

    Two of the barriers for any "advanced" statistic are (1) making it understandable and (2) making it accessible. And the term "advanced" doesn't help, either, because it scares a lot of people off.

    Raw save percentage used to be considered an advanced statistic, and people were much more comfortable with wins/losses, goals-against average, and shutouts to measure a goaltender's performance. Over time, it's become part of the common parlance. Part of that is that people now have a good sense of the save percentage scale, what a "great" save percentage is, what a "lousy" save percentage is, and the levels in between.

    I'm at the point where I'm comfortable working with the save percentage translations (GD, GAR, SNW%, Z-Score), but I'm still learning the nuances of the new metrics I've put together (for consistency and strength of opponent). I'm writing a short paper on the latter, so that should hopefully help people feel more comfortable with it.
     
  17. Hire Sather

    Hire Sather Don't punt seasons

    Joined:
    Oct 4, 2002
    Messages:
    28,451
    Likes Received:
    786
    Trophy Points:
    214
    Location:
    Connecticut
    I'll watch the games
     
  18. Cashville

    Cashville RIP Lindback

    Joined:
    Apr 12, 2011
    Messages:
    6,488
    Likes Received:
    185
    Trophy Points:
    81
    Gender:
    Male
    Location:
    San Francisco
    Frankly, I think you need to go a step further and make it ES SV%+, but yeah it's good.
     
  19. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,106
    Likes Received:
    1,389
    Trophy Points:
    139
    Location:
    Windsor
    Bringing a few things to the table to help show why save percentage translations are important.

    Here's a list of all Vezina Trophy winners:
    http://hockeygoalies.org/awards/vezina.html

    Over time, you can see the raw (unadjusted) save percentages fluctuate, as periods of offense dominated, and periods of defense dominated. Did all of the goaltenders in the 1980s suck? Maybe, but unlikely.

    If you look at the adjusted metrics (Z-Score, Goal Differential, and Goals Above Replacement), you can still see some outliers - this is purportedly the list of each year's best goalie, after all - but the pattern evens out a bit.

    It's also easier to pick out the truly remarkable performances. Note Dominik Hasek's 93% (raw) save percentage in 1993-94 - if you look at his z-score, you can see that his performance was 4.6 standard deviations above the league average. What "z score" measures is the likelihood that an average goaltender would put together a season like the one in question (since we've all seen average goaltenders put together stretches of great play). A z-score of 4.6 means that an average goaltender would reproduce Hasek's 1993-94 season about once every 475,000 years. :amazed:

    If you click through (on the above page), you can see the individual goaltenders' careers evolve (under REGULAR SEASON STATISTICS or POSTSEASON STATISTICS). It helps to compare (for instance) Patrick Roy's career with Dominik Hasek's career with Martin Brodeur's career (and you can see that each was remarkable in his own way).
     
  20. 13th Floor

    13th Floor Registered User

    Joined:
    Oct 10, 2008
    Messages:
    9,084
    Likes Received:
    1,125
    Trophy Points:
    140
    You must be busy.
     
  21. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,106
    Likes Received:
    1,389
    Trophy Points:
    139
    Location:
    Windsor
    This is a common (and flippant) rejoinder, and I'm sure that you're quietly smiling to yourself about how you just "stuck it to the statheads".

    However, "watching the games" is just as useless as "putting your head in a book" for properly evaluating players (both are necessary).

    You should read Daniel Kahneman's book "Thinking Fast and Slow", which largely deals with narrative bias. Our brains are exceptionally biased, and we generally use whatever facts support our preconceived notions about players (and ignore anything that disagrees with those notions). It doesn't help that sports shows replay "highlight" goals and saves, which places a disproportionate share of those events into our memory stores.

    We also don't have the ability to watch (and store in our brains) sample sizes for all goaltenders at once, let alone across multiple seasons.

    Bottom line: for player evaluation, anyone who tells you that statistics are useless is just as wrong as anyone who tells you that watching the games is useless. You need *both* to properly evaluate.
     
  22. 5 Minute Major

    5 Minute Major Registered User

    Joined:
    Dec 4, 2010
    Messages:
    4,475
    Likes Received:
    732
    Trophy Points:
    110
    Location:
    Binghamton, NY
    I wish there was a stat kept for quality scoring chance save percentage. Maybe there is and I don't know that?
     
  23. RPGrizzly

    RPGrizzly Registered User

    Joined:
    Jan 5, 2006
    Messages:
    399
    Likes Received:
    0
    Trophy Points:
    0
    Wow, thanks for this. I've always thought there was a lot more to the typical goalie stats you see, I had no idea the data was already out there! Great work, I'll definitely be spending some time on your website.

    I don't have any problems with the SV%+ stat on H-R. It's pretty interesting.
     
  24. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,106
    Likes Received:
    1,389
    Trophy Points:
    139
    Location:
    Windsor
    The problem with that (from a technical perspective) is that "quality scoring chance" isn't well-defined. Do you know how some teams always lead the league in hits (or blocked shots) by a large margin? It's because a "hit" in Arena A is different from a "hit" in Arena B.

    If the definition could be standardized, it'd be very useful (and it would be great to see).
     
  25. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,106
    Likes Received:
    1,389
    Trophy Points:
    139
    Location:
    Windsor
    Thanks - appreciated! Let me know if you have questions; like I said, part of my goal is to get more people thinking about this sort of thing (since I believe that it leads to a more effective understanding of goaltenders).
     

Share This Page

monitoring_string = "358c248ada348a047a4b9bb27a146148"