NHL goalies better vs high shot volumes

[Mathletic]

http://www.sportsnet.ca/hockey/nhl/nhl-goalies-better-with-high-shot-volumes/

One of the most important aspects of playing goal is the mental preparation that it requires. If you have never played the position, it is difficult to understand or relate to a goaltender while watching him in a game. There are plenty of mental hurdles to overcome from recovering from bad goals to staying in the right headspace despite a light workload.

...

I found that as a goaltender’s workload increased, so did his save percentage. But the results are counterintuitive—we generally fawn over goaltenders with the high save count with first-star recognition. When reviewing the 156 games with 20 shots or fewer, goaltenders registered a sub-.913 save percentage 92 times. When facing fewer than 20 shots, it is essential to not allow more than one goal to maintain anything above .900.

...

Click to read more...

 

[Doctor No]

http://www.sportsnet.ca/hockey/nhl/nhl-goalies-better-with-high-shot-volumes/

One of the most important aspects of playing goal is the mental preparation that it requires. If you have never played the position, it is difficult to understand or relate to a goaltender while watching him in a game. There are plenty of mental hurdles to overcome from recovering from bad goals to staying in the right headspace despite a light workload.

...

I found that as a goaltender’s workload increased, so did his save percentage. But the results are counterintuitive—we generally fawn over goaltenders with the high save count with first-star recognition. When reviewing the 156 games with 20 shots or fewer, goaltenders registered a sub-.913 save percentage 92 times. When facing fewer than 20 shots, it is essential to not allow more than one goal to maintain anything above .900.

...

It's an interesting conclusion - I'll have to dig into it some more when I get time.

One immediate concern I have is the elimination of non-full games; if goaltenders truly get shellacked in a high-volume game (and are pulled), then they are removed from the sample. But of course, so would the goaltender who allows 2 goals on 4 shots in the first period. My impression is that the former occurs far more than the latter (which would flatten out the author's conclusion), but I haven't run the data.

The counter-theory (first presented to my knowledge in Klein and Reif's Hockey Compendium (1st edition)) is the "Lucy in the Chocolate Factory" example. As her workload increased, she made more and more errors. Later research suggested that this wasn't necessarily the case, but no one (until potentially now) has been able to do anything beyond a reasonable doubt in one direction or the other.

The answer (to me) boils down to the proportion of "hard" and "easy" shots in the different-volume games. It would stand to reason that the overworked goaltender saw more rebound shots (since you need a shot to have a rebound), and it would stand to reason that the underworked goaltender saw more breakaways (since the stronger team is already entrenched in the weaker team's zone, it's hard to get breakways). Which outweighs the other?

I didn't have this available to me the last time that I looked at this, but I could use the framework that I developed to determine above-average, average, and below-average games, and look at how they line up on a shots faced per 60 minutes basis. So if goaltenders (on average) were 0.3 standard deviations below average when facing 20 shots per game, but 0.6 standard deviations *above* average when facing 40 shots per game, it would support the hypothesis.

I describe that methodology more here:
http://hfboards.mandatory.com/showthread.php?t=1392449

Side note: like Kelly Hrudey, Jim Craig used to break down a game into five-minute intervals. I started doing it after reading "Boys of Winter". I find it helpful, at least.

 

[Doctor No]

One other item that could be affecting this in the direction seen by the article - shot over-/under-counting.

If shots missing the net are counted as shots, then by definition they can't be goals, and that would lead to higher-shot games with higher save percentages.

And conversely, with legitimate saves that aren't counted, it would lead to lower-shot games with lower save percentages.

 

[MadLuke]

When reviewing the 156 games with 20 shots or fewer, goaltenders registered a sub-.913 save percentage 92 times. When facing fewer than 20 shots, it is essential to not allow more than one goal to maintain anything above .900.
...

Not sure if a understand correctly here, but does he look about how game they had a .913 save% during the game itself ?

if so, I found this very flawed.

take 100 low shot games for a goaltender(s) and compare this high sample size total save percentage to 100 high shot.

going for individual game save percentage and looking for a under/above count of them can be flawed imo, we could to strange stuff like saying over .990 is achieved more often in very low shot game thus goaltender are better in low shot situation, etc...

 

[Mathletic]

Not sure if a understand correctly here, but does he look about how game they had a .913 save% during the game itself ?

if so, I found this very flawed.

take 100 low shot games for a goaltender(s) and compare this high sample size total save percentage to 100 high shot.

going for individual game save percentage and looking for a under/above count of them can be flawed imo, we could to strange stuff like saying over .990 is achieved more often in very low shot game thus goaltender are better in low shot situation, etc...

I agree. There's some stuff that should be given a bit more thought in his research. This being an example.

Also, like Chalupa Batman did in his research, I think it would be better to control for each goalie. Not take the 20, 30, 40, 50 and 60 shots per game as categories.

 

[Hammer Time]

Score effects, as usual, could be a confounding factor. The lower your sv%, the more likely the other team is already winning by a few goals and doesn't need to shoot as much anymore.

 

[Doctor No]

When I last looked for an overall effect, what I (seemed to have) found was there were "good good team goalies" and "good bad team goalies".

That is to say, some goaltenders do better with a higher workload, and some do better with a lower workload. My bet is that it had to do with shot qualities faced in each situation, although what I saw tended to persist from tea to tea.

Having said that, at that level the sample sizes are small enough that it may just be random noise.

 

[Doctor No]

I agree about score effects - which is another casting of shot quality (teams behind probably take a large volume of lower-quality shots in desperation).

 

[hatterson]

One immediate concern I have is the elimination of non-full games; if goaltenders truly get shellacked in a high-volume game (and are pulled), then they are removed from the sample. But of course, so would the goaltender who allows 2 goals on 4 shots in the first period. My impression is that the former occurs far more than the latter (which would flatten out the author's conclusion), but I haven't run the data.

I actually missed that first time through, but I agree that's a significant issue.

What we really care about (IMO) when discussing work load, is shots faced per minute of playtime. Although certainly the concept of wearing a goalie down and making him tired over a long game of shelling him with shots may hold some merit (You'd be a far better authority on that point than I), I think (based on opinion, not data) that the effect from wearing down is less than the effect of eliminating shorter high shot volume starts.

If a team is getting dominated (which is generally the case in extremely high shot volume games) and the goalie has let in 2-3 goals he'll often get pulled regardless of if the goals were his fault or not. It's a strategy to either shake up the team or maintain goalie confidence.

Rarely will you see a goalie give up 5+ goals and remain in for the entire game which means that if the opposing team is in the 40+ shot range, you're naturally going to have a ton of .900 and above performances simply because the ones that weren't up to that level got pulled earlier in the game.

It's much more likely for a goalie with an .800 save percentage to stay in a game when he faces 10 shots (2 goals against) than if he faces 40 shots (8 goals against) so you're disproportionately removing bad performances from high shot games versus low shot games.

 

[Doctor No]

I pulled this together somewhat quickly. This includes every NHL minute played in the 2013 regular season (even where a goalie has been pulled).

S/60 | Minutes | Saves | Shots | Sv % | Expected Saves | Over/Under Expected
0|4|0|0||0|
1|0|0|0||0|
2|0|0|0||0|
3|0|0|0||0|
4|0|0|0||0|
5|0|0|0||0|
6|0|0|0||0|
7|17|2|2|1.000|2|12%
8|16|1|2|0.500|2|-46%
9|39|5|6|0.833|5|-9%
10|30|5|5|1.000|5|9%
11|57|9|11|0.818|10|-12%
12|99|18|20|0.900|18|0%
13|695|137|151|0.907|138|-1%
14|206|43|48|0.896|44|-2%
15|515|111|130|0.854|119|-7%
16|452|104|120|0.867|110|-5%
17|1170|292|332|0.880|301|-3%
18|1566|422|472|0.894|429|-2%
19|2281|657|725|0.906|660|-1%
20|2370|723|794|0.911|723|0%
21|3103|973|1091|0.892|993|-2%
22|3293|1087|1210|0.898|1101|-1%
23|3676|1267|1412|0.897|1287|-2%
24|4117|1492|1652|0.903|1507|-1%
25|4391|1658|1829|0.907|1662|0%
26|5731|2240|2483|0.902|2262|-1%
27|4859|1977|2186|0.904|1996|-1%
28|5297|2259|2471|0.914|2258|0%
29|4838|2112|2331|0.906|2127|-1%
30|5101|2327|2548|0.913|2326|0%
31|4234|2022|2193|0.922|2004|1%
32|3720|1797|1983|0.906|1811|-1%
33|4812|2436|2649|0.920|2422|1%
34|3498|1839|1983|0.927|1814|1%
35|3023|1632|1764|0.925|1612|1%
36|1762|970|1058|0.917|970|0%
37|2587|1460|1596|0.915|1461|0%
38|2282|1328|1445|0.919|1318|1%
39|1523|896|990|0.905|905|-1%
40|1722|1079|1148|0.940|1051|3%
41|812|513|554|0.926|507|1%
42|983|638|687|0.929|630|1%
43|574|385|411|0.937|376|2%
44|343|234|251|0.932|230|2%
45|259|188|195|0.964|179|5%
46|226|153|173|0.884|158|-3%
47|179|134|140|0.957|129|4%
48|90|63|72|0.875|65|-3%
49|79|62|64|0.969|59|5%
50|73|57|61|0.934|56|1%
51|119|88|101|0.871|93|-5%
52|0|0|0||0|
53|0|0|0||0|
54|20|14|18|0.778|16|-14%
55|36|32|33|0.970|30|7%
56|59|50|55|0.909|50|0%
57|0|0|0||0|
58|16|8|15|0.533|14|-41%
59|0|0|0||0|
60|20|17|20|0.850|18|-6%
61|0|0|0||0|
62|0|0|0||0|
63|0|0|0||0|
64|10|7|11|0.636|10|-30%

I didn't group anything, sample sizes be damned - I present, you decide. The data come from my site (http://hockeygoalies.org) and any errors in the underlying data will affect these results.

First few columns should be self-evident (if I've labeled properly).

The expected number of saves is adjusted for the strength of the opponent; I use this in the variance calculations on my site, for instance:

http://hockeygoalies.org/bio/roy.html

Under "REGULAR SEASON STATISTICS" (click the header to open the table), the columns VAR, BAVG, AVG, and AAV).

Here, if a goaltender consistently faces high-shooting percentage opponents, we'd want to adjust for that. Anyhow, this column adjusts for that.

Then, the last column shows the percentage difference between the actual saves made and the expected saves made.

There's a pattern there (similar to the original post). If I had to bet money, I'd attribute the pattern to score effects (mostly) and shot count biases (slightly).

 

[Doctor No]

What we really care about (IMO) when discussing work load, is shots faced per minute of playtime. Although certainly the concept of wearing a goalie down and making him tired over a long game of shelling him with shots may hold some merit (You'd be a far better authority on that point than I), I think (based on opinion, not data) that the effect from wearing down is less than the effect of eliminating shorter high shot volume starts.

I'd agree with this premise; not universally, but for the majority.

Some of it is mental - which is the point of the original post - when we're playing a team who we don't even belong in the same rink with, I can oftentimes play out of my mind and keep the score close. However, there comes a point in the game where the dream fades and you realize that it's not going to happen. Maybe you're down 5-2 halfway through the game and they get that sixth goal. Anyhow, it's easier to let up mentally at that point (I'm not proud of it, but it is what it is).

Typically, though, it's about keeping those anaerobic stretches as short and infrequent as possible. If you're constantly behind the play, then the mental and physical energy can overwhelm.

 

[hatterson]

I pulled this together somewhat quickly. This includes every NHL minute played in the 2013 regular season (even where a goalie has been pulled).

<snip>

Here's a quick chart of the area of that with >1000 minutes played (to hopefully eliminate most of the random noise), I've also included the expected save percentage in there.

There appears to be a slight upward trend in the actual data that isn't present in the expected.

The two trendlines are automatically generated in excel with the r^2 value.

However as you said, score effects may play a part. In addition, is this data ES only, or including PP/SH?

 

[Doctor No]

However as you said, score effects may play a part. In addition, is this data ES only, or including PP/SH?

This includes everything (and separating them will likely be a key step in the understanding of this).

 

[AfroThunder396]

http://www.sportsnet.ca/hockey/nhl/nhl-goalies-better-with-high-shot-volumes/

One of the most important aspects of playing goal is the mental preparation that it requires. If you have never played the position, it is difficult to understand or relate to a goaltender while watching him in a game. There are plenty of mental hurdles to overcome from recovering from bad goals to staying in the right headspace despite a light workload.

...

I found that as a goaltender’s workload increased, so did his save percentage. But the results are counterintuitive—we generally fawn over goaltenders with the high save count with first-star recognition. When reviewing the 156 games with 20 shots or fewer, goaltenders registered a sub-.913 save percentage 92 times. When facing fewer than 20 shots, it is essential to not allow more than one goal to maintain anything above .900.

...

This has been my position for a long time. I've been working on a simple survey of goaltender SV% by season and I've found so far that goaltender tend to record their best SV% seasons in seasons where they face the most shots. So far only 4/20 show a negative trend as shots increase, the rest show a positive correlation.

I'll post the full data when I'm done with it.

 

[Doctor No]

This has been my position for a long time. I've been working on a simple survey of goaltender SV% by season and I've found so far that goaltender tend to record their best SV% seasons in seasons where they face the most shots. So far only 4/20 show a negative trend as shots increase, the rest show a positive correlation.

To clarify: in seasons where they face the most shots, or in seasons where they face the most shots per 60 minutes?

I'd expect the former even in the absence of this thread - if a goaltender's playing better, then the coach will play him more, resulting in an outcome where his highest save percentage seasons correlate with his highest overall volume seasons.

Also, what correlation are you measuring when you show your 4/20 numbers?

 

[Sun God Nika]

I dont have any stats or any knowledge to prove anything, but I can say from watching nhl games for the pas 7 years I always see the goalies that face 35-50 shots having monster games and save percentages, then goalies who face 20 shots always have weak games, I guess its because you are always being tested making you sharper,

I played goalie in High School and i would typically face 25-35 shots per game but the shot quantity never threw me out of it, but it definitely is much more different in the nhl, im sure.

I think its definitely more then a myth

 

[MN_Gopher]

It would be interesting to see stats in tie games vs leads.

Last 5 min. Team A that is down. Get everything to the net. O zone faceoffs are good, jam away. Increase shot volume for the winning goalie.

Last 5 min. Team B that is up. Keep the clock running. Play in their zone. Get a good cycle going or dump and change, stay fresh. Fewer shot volume.

The goalie that had a bad break early in the game sees fewer shots in the final minutes. If he does it is an odd man rush. The goalie that did well sees more shots, many are bread basket muffins but lead to a faceoff.

Passing defense in the NFL. Some of the best teams have the worst pass D. They are up early and often forcing the other team to abandon the run.

 

[AfroThunder396]

To clarify: in seasons where they face the most shots, or in seasons where they face the most shots per 60 minutes?

I'd expect the former even in the absence of this thread - if a goaltender's playing better, then the coach will play him more, resulting in an outcome where his highest save percentage seasons correlate with his highest overall volume seasons.

Also, what correlation are you measuring when you show your 4/20 numbers?

Gross volume of shots. I'm just making simple scatter plots and applying best fit trend lines. Nothing fancy, nor am I meaning for it to be particularly damning evidence.

I can't prove causation, but personally I think it's the other way around - high volume of shots artificially inflates SV% denominator and gives the illusion of great play which may or may not be true.

I'm not really sure how one could prove this chicken-and-egg argument. But I've seen a lot of people use the "but he put up good numbers behind a bad defense" argument. I'd argue that HELPS the goalie, facing a large volume of shots. It gives the illusion that he's standing on his head when there's no way to verify that's the case.

Seeing a goalie make 32/34 saves a night sounds nice but you can't quantify the quality of those shots so it's impossible to determine if the stats are misleading or an accurate portrayal.

 

[Doctor No]

Well - if it's the accumulated volume of shots over a season that causes the better results (and not the volume on a per-game basis), then we should see the pattern get more extreme as the season wears along. That is to say, there may not be much of a difference at November 1st, but by March 1st, the full pattern is evident.

On the other hand, if it's that the better results lead to a larger column) of shots over a season (since coaches will use the player more often in their better season, then we should see the pattern get less extreme as the season wears along (and here's why).

One of the reasons why the sophomore slump exists in pro sports, for what it's worth, are sample size effects. Binomial variations over (say) a goaltender's first 500 shots play a huge role in their early career. Take a young goaltender who plays two standard deviations over their head in their first 500 shots, and then they suffer a "sophomore slump" when they perform at their true underlying ability in year two.

The young goaltender who plays two standard deviations under water over their first 500 shots? They're probably back in the AHL, and the law of large numbers doesn't get to take effect.

My theory for what you're seeing conforms to that - if a goaltender does great at the beginning of a season (either due to an improvement in ability or random fluctuation), then the coach will use them more, and they'll work their way back to form (but still be above average over their season). In the meantime, they played more games than would have been expected at the beginning.

(All of this assumes that the results that you suggest are true, are true, although I have no real reason to doubt those results).

 

[Master_Of_Districts]

Gross volume of shots. I'm just making simple scatter plots and applying best fit trend lines. Nothing fancy, nor am I meaning for it to be particularly damning evidence.

I can't prove causation, but personally I think it's the other way around - high volume of shots artificially inflates SV% denominator and gives the illusion of great play which may or may not be true.

I'm not really sure how one could prove this chicken-and-egg argument. But I've seen a lot of people use the "but he put up good numbers behind a bad defense" argument. I'd argue that HELPS the goalie, facing a large volume of shots. It gives the illusion that he's standing on his head when there's no way to verify that's the case.

Seeing a goalie make 32/34 saves a night sounds nice but you can't quantify the quality of those shots so it's impossible to determine if the stats are misleading or an accurate portrayal.

If that's true, then teams that give up more shots should have higher save percentages, on average, no?

 

[hatterson]

If that's true, then teams that give up more shots should have higher save percentages, on average, no?

Decided to run the numbers real quick. Did individual seasons from 07-08 through 11-12 and limited it to 5 on 5 only (to eliminate PP/SH ice time fluctuations).

Plotting SA/60 against sv% and the highest R^2 value we get is a woeful .0967 The other R^2 values for seasons were .0012, .0535, .0361, and .013 so it's pretty clear that the relationship is basically non-existent. It's worth noting that in all cases the slope was positive, although that could easily be from score effects or poor shot counting.

I also plotted all 5 seasons together.

Again, trendline and R^2 value are excel generated so if they're wrong take it up with them But really a slope of .0035 and an R^2 of .0375 basically means there's no meaningful relationship on a large scale level

 

[Master_Of_Districts]

Decided to run the numbers real quick. Did individual seasons from 07-08 through 11-12 and limited it to 5 on 5 only (to eliminate PP/SH ice time fluctuations).

Plotting SA/60 against sv% and the highest R^2 value we get is a woeful .0967 The other R^2 values for seasons were .0012, .0535, .0361, and .013 so it's pretty clear that the relationship is basically non-existent. It's worth noting that in all cases the slope was positive, although that could easily be from score effects or poor shot counting.

I also plotted all 5 seasons together.

Again, trendline and R^2 value are excel generated so if they're wrong take it up with them But really a slope of .0035 and an R^2 of .0375 basically means there's no meaningful relationship on a large scale level

Thanks.

And all of that is consistent with data from prior to 2007-08 as well.

So even if a relationship exists at the individual game level - which would seem to be primarily owing to score effects, in any event - there's no evidence for a relationship on a more general level.

Which is to say, there's no evidence that a team's ability to prevent or limit shots has any effect on a goalie's save percentage on aggregate.

 

[Doctor No]

Plotting SA/60 against sv% and the highest R^2 value we get is a woeful .0967 The other R^2 values for seasons were .0012, .0535, .0361, and .013 so it's pretty clear that the relationship is basically non-existent. It's worth noting that in all cases the slope was positive, although that could easily be from score effects or poor shot counting.

Very interesting to see (and I agree with the conclusion, although ultimately it's something to put on the testing list).

 

[AfroThunder396]

If that's true, then teams that give up more shots should have higher save percentages, on average, no?

You would think so, but again, there are some other factors involved. Is the quality of shots scaling accurately with the quantity of shots? Some coaches will choose to give the other team high shot volumes because they're confident they will be low percentage shots from the outside. Other teams have killer defensive zone coverage, but suck in transition, where shot quality is inherently higher. Some teams have excellent shot quality AND shot quantity control. If you looked at the league as a whole, I'd be willing to bet that it averages out to little to no reaction, however on a case-by-case basis we'd certainly see trends emerge.

I don't believe it's a coincidence that some teams like Florida and Phoenix consistently see different goaltenders produce sparkling save percentages year after year. I also don't think it's a coincidence that a guy like Marty Brodeur, who had elite shot prevention in front of him his whole career, saw his save percentage skyrocket immediately after the '05 lockout when Stevens and Niedermayer left and the league began emphasizing offense.

Anecdotal, but with enough effort I'm confident we'd see some pretty clear trends among the statistical white noise.

 

[Master_Of_Districts]

You would think so, but again, there are some other factors involved. Is the quality of shots scaling accurately with the quantity of shots? Some coaches will choose to give the other team high shot volumes because they're confident they will be low percentage shots from the outside. Other teams have killer defensive zone coverage, but suck in transition, where shot quality is inherently higher. Some teams have excellent shot quality AND shot quantity control. If you looked at the league as a whole, I'd be willing to bet that it averages out to little to no reaction, however on a case-by-case basis we'd certainly see trends emerge.

Right - there are many factors which influence save percentage at the team level.

But you stated that a "high volume of shots artificially inflates SV% denominator and gives the illusion of great play which may or may not be true."

If that's the case, then there should be a trade-off between shot quality allowed and shot quantity allowed, which would manifest in the form of a positive relationship between save percentage and shot volume against at the team level.

But that doesn't seem to be the case.

I don't believe it's a coincidence that some teams like Florida and Phoenix consistently see different goaltenders produce sparkling save percentages year after year.

Even if that's so - and it certainly doesn't seem to be in the case of Florida - citing specific examples doesn't establish anything in the absence of any relationship in the general population.

For example, I could just as easily cite examples of teams that excelled at both shot prevention and save percentage, like the 2011-12 St.Louis Blues, the 2010-14 L.A. Kings, the 2003-06 Calgary Flames, the 1998-2004 Dallas Stars, and so on and so forth.

I also don't think it's a coincidence that a guy like Marty Brodeur, who had elite shot prevention in front of him his whole career, saw his save percentage skyrocket immediately after the '05 lockout when Stevens and Niedermayer left and the league began emphasizing offense.

In the absence of any relationship in the general population, it may well be nothing more than a simple coincidence.