Estimating the effect of the numbers of shots faced on save percentage

[matnor]

The discussion in the Jim Carey thread about shots against and save percentage got me interested, is it the case that goalies that face a lot of shots on average have higher save percentage than goalies that save few shots? Conventional wisdom suggests that this might be the case. Quite often do we see goalies that face tons of shots play extraordinarily well (see Halak in this playoffs) whereas goalies who have to wait a long time between shots have trouble keeping the focus. This could also be one explanation for why Brodeur's save percentage is rather underwhelming for his status. New Jersey generally allow few shots on net which would make it harder for Brodeur to get a high save percentage.

To test this idea I use data from the Hockey Summary Project for this regular season. All in all this provides data for 2654 games played by goalies. Looking at the raw data I indeed find support for the hypothesis. On average one extra shot increases the save percentage with 0.43 percentage points. However, since goalies that are being pulled on average face few shots and also have bad save percentage this is not surprising. If we only look at goalies who play the entire game the estimated effect drop significantly, one extra shot now increases the save percentage with, on average, 0.20 percentage points. That means a goalie that faces 30 shots every game is expected to have a one percentage point better save percentage than a goalie that faces 25 shots every game. This is a huge effect!

One concern is that goalies who face a lot of shots play on teams that allows a lot of easy shots. Luckily we can actually control for that. By estimating the effect of allowing a lot of shots relative to the average amount of shots each goalie face we get rid of any team-specific effect (for those of you with statistical knowledge this is equivalent to a fixed-effect estimation). It turns out that this doesn't change the estimate at all, it is basically the same as above and still strongly statistically significant. The graph below shows the estimated effect (the line) as well as the average save percentage for each number of shots against.

Given the results above we can actually calculate "adjusted save percentage", that is adjusting for the average shots faced by the goalies. I have used 30 shots against as the benchmark so, in the case of Ryan Miller, the adjusted save percentage is 93.16+(30-31.49)*0.20=92.86. I only use the games in which the goalies played the entire game which explains why the numbers don't exactly correspond to the regular ones. These numbers should not be taken literally since there are a number of factors that could cause the estimations above to be biased but I think it's interesting nonetheless. It could provide one explanation for Chicago's goaltending woes. They allow so few shots per game that it's hard for the goalies to play well, especially for Huet who faces less than 24 shots per game. Any comments on potential problems with these estimations welcome!

Name|GP|Save%|Avg. shots against|Adj. Save%
Ryan Miller | 65 | 93.16 | 31.49 | 92.86
Tuukka Rask | 39 | 93.02 | 29.38 | 93.14
Miikka Kiprusoff | 66 | 93.00 | 28.56 | 93.29
Jimmy Howard | 53 | 92.95 | 30.25 | 92.90
Tomas Vokoun | 58 | 92.92 | 34.09 | 92.09
Evgeni Nabokov | 66 | 92.83 | 31.29 | 92.57
Henrik Lundqvist | 67 | 92.76 | 30.09 | 92.74
Jaroslav Halak | 41 | 92.74 | 32.24 | 92.28
Tim Thomas | 37 | 92.70 | 30.38 | 92.63
Antero Niittymaki | 38 | 92.69 | 31.32 | 92.42
Roberto Luongo | 60 | 92.48 | 29.70 | 92.54
Ilya Bryzgalov | 50 | 92.47 | 30.00 | 92.47
Martin Brodeur | 70 | 92.32 | 26.99 | 92.93
Jonas Hiller | 52 | 92.23 | 33.65 | 91.49
Marty Turco | 48 | 92.15 | 31.83 | 91.78
Cam Ward | 41 | 92.13 | 31.61 | 91.80
Chris Mason | 56 | 92.05 | 28.98 | 92.26
Craig Anderson | 68 | 92.03 | 31.93 | 91.64
Carey Price | 37 | 91.85 | 31.84 | 91.48
Johan Hedberg | 40 | 91.84 | 31.58 | 91.53
Antti Niemi | 33 | 91.84 | 26.36 | 92.58
Ondrej Pavelec | 35 | 91.66 | 34.94 | 90.66
Pekka Rinne | 51 | 91.64 | 28.86 | 91.87
Marc-Andre Fleury | 58 | 91.61 | 27.95 | 92.03
Jose Theodore | 39 | 91.61 | 31.46 | 91.31
Brian Elliott | 45 | 91.51 | 28.00 | 91.91
Dwayne Roloson | 45 | 91.46 | 32.80 | 90.90
Steve Mason | 48 | 91.43 | 31.13 | 91.20
Jonathan Quick | 68 | 91.05 | 27.44 | 91.57
Mike Smith | 33 | 91.00 | 31.67 | 90.67
Jean-Sebastien Giguere | 31 | 90.92 | 30.90 | 90.74
Niklas Backstrom | 54 | 90.86 | 28.37 | 91.19
Cristobal Huet | 42 | 90.77 | 23.74 | 92.04
Jeff Deslauriers | 41 | 90.58 | 32.88 | 90.00
Jonas Gustavsson | 36 | 90.58 | 29.78 | 90.62

Click to read more...

 

[Canadiens1958]

Interesting Effort

Interesting effort.

What happens when you breakdown specific goalies per game based on the number of shots they face.

Example Goalie A w games 0 - 19 shots faced, SV% =
x games 20 - 29 shots faced, SV% =
y games 30 - 39 shots faced, SV% =
z games 40 + shots faced, SV% =

Would be interesting to see similar breakdowns by era. Example SV% in games from 1960, 1970, 1980,1990,2000,2010 where the goalie faced shots at each level listed above.

 

[matnor]

Interesting effort.

What happens when you breakdown specific goalies per game based on the number of shots they face.

Example Goalie A w games 0 - 19 shots faced, SV% =
x games 20 - 29 shots faced, SV% =
y games 30 - 39 shots faced, SV% =
z games 40 + shots faced, SV% =

Would be interesting to see similar breakdowns by era. Example SV% in games from 1960, 1970, 1980,1990,2000,2010 where the goalie faced shots at each level listed above.

No. of Shots|Number of Games|Save%
0-19|94|88.16
20-29|982|90.64
30-39|939|92.28
40-|50|94.00

The effect is truly huge which almost makes me think there is something mechanical causing this but I can't really think of what that could be. As for using historical data I'll have a look at all of the data from the HSP which would at least get back to the 93-94 season.

 

[pitseleh]

Shot recording bias (differences from arena to arena) might explain some of the difference. The bias won't (obviously) affect the number of goals scored but it will affect the number of saves a goalie makes.

There's likely a small shot difficulty effect too - I think shot totals and shot difficulty are generally (but not always) inversely correlated.

Here's an interesting analysis that looks at whether a goalie gets cold after not taking shots:

http://www.behindthenethockey.com/2009/12/3/1110764/do-goalies-need-to-face-shots-in

 

[matnor]

Shot recording bias (differences from arena to arena) might explain some of the difference. The bias won't (obviously) affect the number of goals scored but it will affect the number of saves a goalie makes.

There's likely a small shot difficulty effect too - I think shot totals and shot difficulty are generally (but not always) inversely correlated.

Since I control for specific goalie effect I don't think the first point is a big concern even though it there could be such an effect. However, I think your second point is very valid.

 

[matnor]

Here's an interesting analysis that looks at whether a goalie gets cold after not taking shots:

http://www.behindthenethockey.com/2009/12/3/1110764/do-goalies-need-to-face-shots-in

Very interesting, seems to go in a different direction than what the results above suggests.

 

[overpass]

Teams that are trailing tend to take more shots at a lower percentage, while teams that are leading tend to take fewer shots at a higher percentage, especially in the 3rd period. See discussion on this here.

As a result, winning teams often have fewer shots than losing teams in a particular game. But good teams tend to outshoot bad teams over the season, because outshooting is both a cause of winning and an effect of losing.

Game level: In the 2009-10 regular season, the outshooting team won 580 games and lost 588.

Season level: In the 2009-10 regular season, the top 12 teams in points outshot their opponents. 12 of the bottom 13 teams in points were outshot by their opponents.

Clearly the game level effect and season level effect are different. As a result, I would be careful about adjusting seasonal numbers using effects that you have found on a game level.

The most correct way to do this might be to separate the 3rd period numbers from the 1st and 2nd period numbers. Treat (1st and 2nd periods) all the same. Separate 3rd periods into periods that the goalie starts 1. Tied, 2. Leading, 3. Trailing, and look at the save percentage numbers there. I don't know if this is feasible for you, but it's what makes sense to me.

 

[matnor]

Teams that are trailing tend to take more shots at a lower percentage, while teams that are leading tend to take fewer shots at a higher percentage, especially in the 3rd period. See discussion on this here.

As a result, winning teams often have fewer shots than losing teams in a particular game. But good teams tend to outshoot bad teams over the season, because outshooting is both a cause of winning and an effect of losing.

Game level: In the 2009-10 regular season, the outshooting team won 580 games and lost 588.

Season level: In the 2009-10 regular season, the top 12 teams in points outshot their opponents. 12 of the bottom 13 teams in points were outshot by their opponents.

Clearly the game level effect and season level effect are different. As a result, I would be careful about adjusting seasonal numbers using effects that you have found on a game level.

The most correct way to do this might be to separate the 3rd period numbers from the 1st and 2nd period numbers. Treat (1st and 2nd periods) all the same. Separate 3rd periods into periods that the goalie starts 1. Tied, 2. Leading, 3. Trailing, and look at the save percentage numbers there. I don't know if this is feasible for you, but it's what makes sense to me.

Agreed, the adjusted numbers are definately not something to take very seriously. I do have the numbers by period and very preliminary results suggest that the effect hold for each period separately, I'll try to post the results later.

 

[Canadiens1958]

Useless Ice

No. of Shots|Number of Games|Save%
0-19|94|88.16
20-29|982|90.64
30-39|939|92.28
40-|50|94.00

The effect is truly huge which almost makes me think there is something mechanical causing this but I can't really think of what that could be. As for using historical data I'll have a look at all of the data from the HSP which would at least get back to the 93-94 season.

Not sure if mechanical is the right term. Shooting from "useless ice" is more of a factor.

The slot is recognized as the prime scoring area because it provides the shooter with the greatest number of efficient options. Likewise the goalie is faced with the greatest number of shot stopping challenges. Reduce it to the shoot-out scenario and look at the SV% for shoot-outs over the course of a season.

"Useless ice". The further one gets from the slot, especially towards the corners or the offensive zone perimeters,the efficient scoring options are reduced. Often the goalie just has to let the puck hit him in a fashion that it does not deflect into the net.

I know that the teams keep there own stats that breakdown shooting efficiency from various parts of the offensive zone for skaters and SV% for goalies. They track the different types of shots, glove side, stick side, etc. Getting them to share the data or how it is interpreted is a different issue.

Your effort is a very positive start.