How variable is goaltender performance compared to what probability would expect?

[Steerpike]

How variable is goaltender performance compared to what probability would lead us to expect?

You hear all the time about how a goalie was "a wall" one night, how they "stole a game" or was in some other way unstoppable. On the other hand, you also hear quite often about goalies that are "off their game" in some way or another.

Thus we should expect that the variability in goals scored against a goaltender would be greater than the variability expected by probability. We can estimate how often a goalie ought to give up a certain number of goals using a Poisson distribution. This takes the rate of goals against (say 2.0 for Rask) and gives you a probability that he will get a shutout, let in 1 goal, let in 2 goals,....ect.

Lets look at Rask's expected distribution vs his actual distribution.

Rask 2.0 GA/60, 58 games

The expected games in which he gives:
0 goals is 7.84945
1 goals is 15.6989
2 goals is 15.6989
3 goals is 10.4659
4 goals is 5.23296
5 goals is 2.09319
6 goals is 0.697729
7 goals is 0.199351
8 goals is 0.0498378

In reality he gave up:
0 goals in 7
1 goal in 19
2 goals in 12
3 goals in 12
4 goals in 5
5 goals in 2
6 goals in 1



I'm going to do more goalies, it's just a little tedious.

You could also look at this from a shots/save percentage angle but I'm not sure how yet and this is interesting on its own I think.

Click to read more...

 

[Steerpike]

Quick: 2.1 GA/60 ; 49 games played

expected games at:

0 goals is 6.00037
1 goals is 12.6008
2 goals is 13.2308
3 goals is 9.26156
4 goals is 4.86232
5 goals is 2.04217
6 goals is 0.714761
7 goals is 0.214428
8 goals is 0.0562874

actual:

0 goals in 6
1 in 12
2 in 12
3 in 13 one with 20 min one with 36 min
4 in 5
5 in 1

 

[Steerpike]

Bishop: 2.2 GA/60; 60 GP (excluding 3 games he played for <6 minutes???)

expected games for:

0 goals is 6.64819
1 goals is 14.626
2 goals is 16.0886
3 goals is 11.7983
4 goals is 6.48908
5 goals is 2.85519
6 goals is 1.0469
7 goals is 0.329027
8 goals is 0.0904825

actual games:

0 in 5
1 in 15 one played for 40 min
2 in 18
3 in 13 one played for 21.8 min
4 in 4 one played for 35.8 min
5 in 5

---------------------------------------------------------

Craig Anderson: 2.8 GA/60; 51 GP

expected games for:

0 goals is 3.10131
1 goals is 8.68368
2 goals is 12.1571
3 goals is 11.3467
4 goals is 7.94267
5 goals is 4.4479
6 goals is 2.07568
7 goals is 0.830274
8 goals is 0.290596
9 goals is 0.0904076

actual:

0 in 5 one 20 min game
1 in 6
2 in 11 two brutal games in which got pulled in 4 minutes
3 in 12 one 44 min game
4 in 12 one 30 min game
5 in 5 one 43 min game
6 in 1
7 in 1


I really wish I could exclude partial games as it gets complicated but outlier ****-storm games are a substantial part of what we are interested in here.

-------------------------------------------------------------------------------------------------------

Varlamov GA/60 2.4; 63 GP

expected games for:

0 goals is 5.71523
1 goals is 13.7166
2 goals is 16.4599
3 goals is 13.1679
4 goals is 7.90074
5 goals is 3.79235
6 goals is 1.51694
7 goals is 0.520094
8 goals is 0.156028

actual:

0 in 3 one 20 min game
1 in 11
2 in 29 one 12.7 min game one 20 min game
3 in 10
4 in 6
5 in 3 one 30 minute game
6 in 0
7 in 0
8 in 1

 

[Steerpike]

What should you take from this? Goalies don't exactly have to be incredible or terrible to have shutouts or blowouts. Probability dictates that if they give up an average of 2.4 goals per game and play 63 games they ought to have about 6 shutouts, and they probably will have a few games where they let in 6 or 7. It's not that they were fantastic or terrible those nights, they just get lucky or unlucky.


The fact that goalies get pulled when they appear to be having a terrible game really throws a wrench in my ability to analyze blowouts. Based on how often goalies get pulled though I would guess that there are some nights where the goalie isn't really seeing it right, but I would guess that far too much fault is put on a goalies shoulders in these situations. Obviously though if a goalie wants to be pulled to escape embarrassment that still makes sense.

 

[Steerpike]

In honor of goalies getting pulled, lets look at the carrer numbers of Patrick Roy to see if we can accurately predict how many shutouts he had and guess how many times he should have had a disaster game.

He played in two extremely different eras so lets split up his Montreal carrer and his Avs carrer

Montreal: 2.77 GA/60 551 GP:

The number of games we expect at:

0 goals is 34.5268
1 goals is 95.6391
2 goals is 132.46
3 goals is 122.305
4 goals is 84.6962
5 goals is 46.9217
6 goals is 21.6622
7 goals is 8.57203
8 goals is 2.96807
9 goals is 0.913505
10 goals is 0.253041

It looks like he should have thrown 35ish shutouts and had quite a few atrocious games just due to probability.

In reality he had just 29 shutouts in that span.

For Colorado he had a 2.27 GA/60 and played 478 games.

The number of games we expect at:

0 goals is 49.3832
1 goals is 112.1
2 goals is 127.233
3 goals is 96.2733
4 goals is 54.6351
5 goals is 24.8043
6 goals is 9.3843
7 goals is 3.0432
8 goals is 0.863507
9 goals is 0.217796

In reality he had just 37 shutouts. On the Colorado board I heard it floated that Roy got really nervous when he was getting a shutout and let in some bad goals at the ends of games because of it. Perhaps that effect is quite mathematically significant.

 

[Steerpike]

Now lets do Brodeur and see if he's afraid of laying an egg (not sure if that expression works or not)

2.24 GAA 1259 GP

The number of games we expect at:

0 goals is 134.031
1 goals is 300.23
2 goals is 336.258
3 goals is 251.072
4 goals is 140.601
5 goals is 62.989
6 goals is 23.5159
7 goals is 7.52509
8 goals is 2.10703
9 goals is 0.524415


So we expect 134 shutouts. In reality he's gotten 124.

That's much closer.

I think there might be some systematic error though due to the other team pulling their goaltender at the end of a game. When you have a shutout going you are most likely up at the end of a game and I would guess that last minute goals lead to fewer shutouts than the model predicts.

I know the Avs saved themselves from getting shut out several times this year by pulling their goalie.

 

[Steerpike]

Thought you were going to get out of here without a summarizing spreadsheet? Think again.

Here's how an 82 game season would look for various GAA rates.

Each row is a different GAA and in each column you can see how many games you would expect your team to score the number of goals at the top of the column. It shows why soccer sucks.

What's really fascinating is that if you score three goals per game on average, you expect to score 3 goals in just as many games as you expect to score 2 goals. And that works with ever integer. If your expectation is n goals then it's just as likely that you score n-1 goals as it is that you score n goals!


This is of course due to the fact that (x^x)*exp(x)/(x!)=(x^(x-1))*exp(x)/((x-1)!) so the probabilities come out the same. Wonky.

 

[Doctor No]

I'm enjoying this quite a bit - thanks for taking the time to formulate it and to post it.

I approach the same topic from a different direction here:
http://hfboards.mandatory.com/showthread.php?t=1392449