All-in-one hockey value stats

[mindmasher]

Even with all that said, I still think GVT has serious limitations. I'm just arguing that the goaltender position, typically speaking, is the most important single player on your team.

 

[GuineaPig]

GVT tells us that 6 out of the top 10 NHL players are goaltenders. Do you believe this to be true?

It isn't that 6 out of the top 10 are the best players, just that 6 out of the 10 are the most valuable according to GVT.

0 out of 10 are defenders. Do you believe this to be true?

That's a different kettle of fish. I think everyone can agree that GVT doesn't accurately (or sufficiently) account for defensive play.

 

[wgknestrick]

Well Taco, it's funny you bring this up. I think there are a lot of smart people - Dejardins and Dellow for instance - who have argued repeatedly that unless you are getting an elite level goalie, save your money because performance variations amongst middle tier goalies is negligible.

In other words, goaltenders are very important, but tend to be very close in ability. Therefore unless you have a bead on a Tim Thomas type, you should be spending small dollars on a couple of capable NHL starters who simply don't have many teams to play for.

In another way, goaltenders can also be a considerable source of lost points on a team if you deign to start a guy who is performing considerably below replacement level.

So yes, ultimately, goaltenders have little value in that there are a lot of good cheap options. You see some GM's following this principle closely. Detroit, Chicago, Avs, Philly, Phx, etc.

Well this statement (which I think we both believe to be true) goes along with my argument against GVT. You also note that those teams you've mentioned all seem to have good D corps (CHI, DET, PHI-when Pronger was there) or good D systems (PHX).

Standard deviation of GVT among goaltenders was over 10 last year. This either indicates there is a huge divide between goaltending talent or that GVT is improperly valuing goaltenders. Forwards and D men GVT show a standard deviation of around 5-4 respectively.

I still think GVT is probably the best show out there currently, but this one flaw is pretty large IMO. It only needs tweaked to "fix" it.

Here is the GVT for last year only, but I've found the results to be similar year after year

https://docs.google.com/spreadsheet/ccc?key=0Am3OYanYYtQQdDVuV2t2Mmx4eUpMc0VBdGR2N0xUWkE

Just examine defenders to goaltenders to see how seemingly different the populations are. This is not to be expected for an adjusted metric.

 

[Hammer Time]

I'll play devil's advocate (since I'm widely known as the biggest "goalie homer" on the board).

Suppose that goaltenders have much more influence on the game than either forwards and defensemen (and let's suppose that it isn't even close). That's different than what's being considered here, which is "how much more value does Goaltender X bring to a team than a replacement-level player?".

If we pictured a world where goaltenders had the most influence on the game, but where all goaltenders were identical, then Goaltender X would have no value (because you could go down to the goalie store and get another one just like him).

To summarize: the question here isn't whether or not a goaltender has value. The question is whether or not he has more valuable than a team's next option.

The problem is with the definition of replacement level - Ryder's Player Contribution (I'd hazard a guess that GVT has a similar definition) defines replacement level as the level that a minor-league player would perform at if called up.

So what these stats are actually measuring is "how many wins did the Bruins gain with Tim Thomas in net, compared to if they had Anton Khudobin in net" rather than comparing Thomas to Rask (which is what you are suggesting).

This is why the likes of Antti Niemi appear higher than, say, Ryan Suter on a GVT list. Niemi may not be an elite goalie, but simply by not making rookie mistakes, he's already contributing a lot of "value over replacement level" to his team.

 

[wgknestrick]

The problem is with the definition of replacement level - Ryder's Player Contribution (I'd hazard a guess that GVT has a similar definition) defines replacement level as the level that a minor-league player would perform at if called up.

So what these stats are actually measuring is "how many wins did the Bruins gain with Tim Thomas in net, compared to if they had Anton Khudobin in net" rather than comparing Thomas to Rask (which is what you are suggesting).

This is why the likes of Antti Niemi appear higher than, say, Ryan Suter on a GVT list. Niemi may not be an elite goalie, but simply by not making rookie mistakes, he's already contributing a lot of "value over replacement level" to his team.

For GVT, it is always "how many goals" did x gain.

The VT stands for Versus Threshold, and you are absolutely right on that.

 

[Bear of Bad News]

So what these stats are actually measuring is "how many wins did the Bruins gain with Tim Thomas in net, compared to if they had Anton Khudobin in net" rather than comparing Thomas to Rask (which is what you are suggesting).

No, I meant Khudobin in this case (although I didn't phrase it as eloquently as I could have).

 

[Czech Your Math]

There's two fundamental assumptions used in GVT to value goalies.

First, it assumes that threshold level is allowing 4% more goals than league average. It's not easy to determine replacement level. I took the top 2N goalies (N = number of teams) and sorted them in terms of SV%. Median 2N is the median of the 2nd N, i.e. the SV% for goalies #45 & #46 (in terms of SV%). Median N+1 is the SV% of the 31st goalie. GVT is 4% more goals than league avg., which I hope I've calculated the same as GVT.

YEAR | Med N+1 | Med 2N | Avg. | GVT
2012 | 91.3% | 90.4% | 91.4% | 91.1%
2011 | 91.4% | 90.7% | 91.3% | 91.0%
2010 | 90.9% | 90.4% | 91.1% | 90.7%
2009 | 91.0% | 90.0% | 90.8% | 90.4%
2008 | 90.8% | 90.0% | 90.9% | 90.5%
2007 | 90.5% | 89.3% | 90.5% | 90.1%
2006 | 90.0% | 89.3% | 90.1% | 89.7%
2004 | 91.0% | 90.5% | 91.1% | 90.7%
2003 | 90.8% | 89.7% | 90.9% | 90.5%
2002 | 90.6% | 90.1% | 90.8% | 90.4%
2001 | 90.1% | 89.3% | 90.3% | 89.9%
2000 | 90.5% | 89.7% | 90.4% | 90.0%
1999 | 90.6% | 89.8% | 90.8% | 90.4%
1998 | 90.5% | 90.0% | 90.6% | 90.2%
1997 | 90.3% | 89.7% | 90.5% | 90.1%
1996 | 90.0% | 88.7% | 89.8% | 89.4%
1995 | 90.2% | 89.0% | 90.1% | 89.7%
1994 | 89.3% | 88.3% | 89.5% | 89.1%
1993 | 88.6% | 87.8% | 88.5% | 88.0%
1992 | 88.6% | 88.1% | 88.8% | 88.4%
1991 | 88.9% | 87.8% | 88.6% | 88.1%
1990 | 88.0% | 87.0% | 88.1% | 87.6%
1989 | 88.0% | 87.1% | 87.9% | 87.4%
1988 | 87.8% | 86.8% | 88.0% | 87.5%
1987 | 88.1% | 87.4% | 88.0% | 87.5%
1986 | 87.5% | 86.1% | 87.4% | 86.9%
1985 | 87.7% | 86.3% | 87.5% | 87.0%
1984 | 87.1% | 86.3% | 87.3% | 86.8%

The replacement level generally falls between median N+1 and median 2N. That seems pretty fair IMO, because teams should be able to acquire one of the better (1st-15th) backups in the league for a reasonable price. If such a backup (or sub-par starter) was actually of great value, one would expect he would at least be performing among the top 30/60 in SV% and/or he would be starting already.

The second assumption is that the difference between the goalie's SV% and league avg. is 75% the responsibility of the goalie. GVT's author mentions a 15% correlation between the SV%s of goalies on the same team, which may seem to justify such an attribution. I'm less certain how accurate this 75% attribution is, but it's not easy to support/disprove this number.

I found the data relating to PDO for the '08-'11 seasons and was recently looking at some of that. Some of what I found relates to this topic. Based on the data from BehindTheNet.com (the accuracy of which I'm uncertain at this time, e.g. shooting%s don't calculate as listed), it seems that opponents' shooting% was correlated by 40-62% from one season to the next (depending on whether using 5v5 or overall data). This may suggest that many goalies have "fluke" seasons. Perhaps the 75% responsibility needs to be substantially reduced and the random factor spread out more amongst the team (skaters), or maybe it just needs to be recognized that goalie has a lot of impact and that there's bound to be a few goalies that have big seasons each year. Besides confirming the reliability of this data (and using data from other sources), it would be helpful to see how individual goalies' opponent shooting%s (1-SV%) correlated from one season to the next.

Skaters' scoring data is compared, on a per-minute basis, to a 75% threshold. This is arbitary also. On an overall (not per-minute) basis, this seems rather high. IOW, 100 points is enough to challenge for the Ross the past couple years, but acquiring a 75 point scorer at a reasonable price seems much less likely than finding a goalie who is not getting full playing time. I tend to believe that this could be an important cause of the disparity between top goalies and top forwards in GVT.

 

[wgknestrick]

That was a great post and I think we are on to something with the 15% skaters attribute toward SV%. The threshold seems accurate and correct.

While I don't have the data to back up this statement (yet), I'll use TBL and TOR as my prime examples over the last couple of years. Still, 15% of a SV% is not much to spread across a team of skaters. 1 player take 85% of the blame, while 15-18 skaters SHARE 15% of it. No QualComp adjustments to the top defenders. Just a blanket, equal share of the results. Goalies should NOT be evaluated against their total SV%, but things (GVT) should be split intelligently accounting for 5v4 SV%, 5v5 SV%, etc.

Teams have much more affect on 5v4 SV% (probably higher than 15%).

Typically GVT has been brutal on the goalies for both of these generally poor defensive teams (no matter who was put in net). GVT places 85% of the blame on the goalie's shoulders for this resulting in "the worst" goal tenders getting -20ish GVTs which is far and away much, much lower than any other skater. If GVT truly was accurately reflecting these goalie's performances, we'd see these teams quickly replace them and "solve" their defensive problems. We all know that isn't the case for the poor TOR fans.

The defenders for these very poor teams do not typically shoulder the blame and post + GVTs despite getting dominated and feeding their goal tenders to the wolves. If we were dealing with just a "poor" goaltender, I'd expect to see backups and other options played for these teams with some success, but that doesn't seem to be the case. I don't think that SV% for a goalie is as isolated in every case as we'd like to assume. There needs to be a team adjustment for their PK ability and rate at which they take penalties (cough, TOR).

There are teams at the extremes that have a huge impact on their goal tenders performance and SV%. BOS at the top and TOR at the bottom. Looking at these teams PK%s is a good place to start. Looking at total SV% alone also has to stop IMO in the GVT calculations to progress it to the next level.

Once this happens, GVT will probably start to gain more popularity.

 

[Czech Your Math]

That was a great post and I think we are on to something with the 15% skaters attribute toward SV%. The threshold seems accurate and correct.

While I don't have the data to back up this statement (yet), I'll use TBL and TOR as my prime examples over the last couple of years. Still, 15% of a SV% is not much to spread across a team of skaters. 1 player take 85% of the blame, while 15-18 skaters SHARE 15% of it. No QualComp adjustments to the top defenders. Just a blanket, equal share of the results. Goalies should NOT be evaluated against their total SV%, but things (GVT) should be split intelligently accounting for 5v4 SV%, 5v5 SV%, etc.

Teams have much more affect on 5v4 SV% (probably higher than 15%).

Typically GVT has been brutal on the goalies for both of these generally poor defensive teams (no matter who was put in net). GVT places 85% of the blame on the goalie's shoulders for this resulting in "the worst" goal tenders getting -20ish GVTs which is far and away much, much lower than any other skater. If GVT truly was accurately reflecting these goalie's performances, we'd see these teams quickly replace them and "solve" their defensive problems. We all know that isn't the case for the poor TOR fans.

The defenders for these very poor teams do not typically shoulder the blame and post + GVTs despite getting dominated and feeding their goal tenders to the wolves. If we were dealing with just a "poor" goaltender, I'd expect to see backups and other options played for these teams with some success, but that doesn't seem to be the case. I don't think that SV% for a goalie is as isolated in every case as we'd like to assume. There needs to be a team adjustment for their PK ability and rate at which they take penalties (cough, TOR).

There are teams at the extremes that have a huge impact on their goal tenders performance and SV%. BOS at the top and TOR at the bottom. Looking at these teams PK%s is a good place to start. Looking at total SV% alone also has to stop IMO in the GVT calculations to progress it to the next level.

Once this happens, GVT will probably start to gain more popularity.

I agree that PP and PK effectiveness should be factored out and attributed first if possible.

As I said in my post, and you seem to agree, it's very possible that goalies are being given too much credit for the difference between their SV% and threshold. However, just to be clear, GVT's author says that goalies on the same team have only a 15% correlation in SV%, but he does not fully attribute the value from the difference between goalie's SV% and threshold (4% more goals than league avg.) to the goalie, not even 85%. He attributes only 75% of this value to the goalie and terms it "Goalie Responsibility."

The question is whether that 75% is the right number and whether the other thresholds (such as 75% of team's avg. per-minute scoring production) are fair and whether other parts of the system are accurate/fair.

 

[wgknestrick]

I agree that PP and PK effectiveness should be factored out and attributed first if possible.

As I said in my post, and you seem to agree, it's very possible that goalies are being given too much credit for the difference between their SV% and threshold. However, just to be clear, GVT's author says that goalies on the same team have only a 15% correlation in SV%, but he does not fully attribute the value from the difference between goalie's SV% and threshold (4% more goals than league avg.) to the goalie, not even 85%. He attributes only 75% of this value to the goalie and terms it "Goalie Responsibility."

The question is whether that 75% is the right number and whether the other thresholds (such as 75% of team's avg. per-minute scoring production) are fair and whether other parts of the system are accurate/fair.

I think we are starting to come to the conclusion that the adjustment may not be the same % allocation for team/goalie for every team. While 15% (or 25%) may work as an NHL average, there are certain teams at the extremes where that % may need to be adjusted based on PK minutes and success during those minutes. I think we may be able to borrow from some of the great work on expected goals for goalies that use these inputs.

Teams should shoulder more of the scaled SV% credit with 5v4 min/5v5 minutes as the deciding factor or input.

This still comes back to the tough GVT pill to swallow of a single player costing his team 20+ goals over a replacement goalie. I can't see that ever being the case (unless I personally was playing goalie for said team). NHL goalies just aren't "bad" to that order of magnitude outside of MAF in the 2012 playoffs.

I think offensive GVT is just about perfect, and we may be able to refine dGVT and gGVT to match it.

Does anyone have a GVT calculating spreadsheet to start from?

 

[Czech Your Math]

I think we are starting to come to the conclusion that the adjustment may not be the same % allocation for team/goalie for every team. While 15% (or 25%) may work as an NHL average, there are certain teams at the extremes where that % may need to be adjusted based on PK minutes and success during those minutes. I think we may be able to borrow from some of the great work on expected goals for goalies that use these inputs.

Teams should shoulder more of the scaled SV% credit with 5v4 min/5v5 minutes as the deciding factor or input.

This still comes back to the tough GVT pill to swallow of a single player costing his team 20+ goals over a replacement goalie. I can't see that ever being the case (unless I personally was playing goalie for said team). NHL goalies just aren't "bad" to that order of magnitude outside of MAF in the 2012 playoffs.

I think offensive GVT is just about perfect, and we may be able to refine dGVT and gGVT to match it.

Does anyone have a GVT calculating spreadsheet to start from?

There's definitely more refinement that could be done. I think that many goalies at/near the top and at the bottom is an indication that the GR of 75% is probably too high. Maybe it should be 50-60%?

Has anyone done a study on goalies switching teams? That might help factor out the team component of SV% and, along with factoring in other data, could help determine a better number.