Skaters have no impact on their goaltenders performance

[Rengorlex]

Skater performance has no correlation with how well goaltenders perform relative to their xGA. Goals against should therefore not be used as a basis for skaters defensive ability. A Twitter thread by Micah: (I suggest reading it all)

Click to read more...

 

[ijuka]

"Skater performance has no correlation with how well goaltenders perform relative to their xGA. Goals against should therefore not be used as a basis for skaters defensive ability."

This is a faulty conclusion, because a skater's defensive ability can still affect the amount of dangerous shots the goalie faces, which can affect goals against.

 

[Rengorlex]

"Skater performance has no correlation with how well goaltenders perform relative to their xGA. Goals against should therefore not be used as a basis for skaters defensive ability."

This is a faulty conclusion, because a skater's defensive ability can still affect the amount of dangerous shots the goalie faces, which can affect goals against.

Which is reflected in xGA, not (necessarily) GA.

 

[emptyNedder]

Which is reflected in xGA, not (necessarily) GA.

I am not fully versed in the xGA metric. But I have to presume that it is still flawed. From what I do know it is based mostly on location of shot and (perhaps) whether the shot is "clear." The second is likely to be subjective.

Think of the NFL statistic for QB rushes. Something similar may be happening on some but not all high-danger shots. Better defenders might not prevent a high-danger shot but they may make the shooter less comfortable. Other considerations are how clean the pass is to the shooter and if it is cross ice, which means the goalie is likely moving.

I know some analysts (I am pretty sure McCurdy is one) have more specific xGA models that account for such things--though there is still a good deal of subjectivity in when a pass is "clean," etc.

There seems to be evidence that defense--whether individual or team--impacts a goalie's performance. I would expect actual goals and xGA are more common in OT because at 3-on-3 team defense is significantly less effective.

The bottom line is that goals (as opposed to expected goals) scored in a game are almost certainly impacted by better and worse defenders.

 

[Rengorlex]

The bottom line is that goals (as opposed to expected goals) scored in a game are almost certainly impacted by better and worse defenders.

True, but the difference between xga and ga is not.

 

[SniperHF]

Just because you can't mathematically prove something in the early stages of a discipline (advanced hockey analytics) doesn't mean it isn't so.

There are a ton of baseball statisticians who looked real stupid about 15 years ago because they mocked the idea of catching performance affecting pitchers. Almost all of them were flat out wrong in their arrogance.

Especially with a sport like hockey that's substantially harder to quantify, using blanket terms like " no impact" Is asking for it.

 

[morehockeystats]

There is a catch. The goalie performance by Micah is the relation between GA and xGA (not necessarily just GA-xGA). It could very well be that skater has little effect onto it, unless he consistently blocks his own goalie's sight or makes the skater whiff on the shot (then the decisive factors of xGA are out of scope used by Micah)

 

[LEAFANFORLIFE23]

What a ridiculous thing to say

 

[BraveCanadian]

An obviously false premise.

 

[ResilientBeast]

Title is misleading

 

[morehockeystats]

An obviously false premise.

Why?
Do you mean a skater can cause the goalie to the degree the goalie starts saving significantly less wrist shots from the point? Or the other way, help him enough to stop more of them? Because that's what Micah's point is.

 

[abo9]

Why?
Do you mean a skater can cause the goalie to the degree the goalie starts saving significantly less wrist shots from the point? Or the other way, help him enough to stop more of them? Because that's what Micah's point is.

I played goaltender for a number of years.

Based on your example, yes, I believe that skaters can significantly impact a goalie's performance.

ie. a forward who gets to hang onto the puck for an extended period of time, with open passing lanes is much more dangerous than one in the same position on the ice who's pressured and/or doesn't have good passing lanes.

And then, are rebound shots classified? A shot in the slot from rebounds because the D can't clear the slot is much more dangerous than a shot from the slot from a forward thats on a rush and covered by a D.

Overall, I dont think you'll see an impact from single skaters when aggregatinf everyone, but theres five skaters on the ice which, as a group, I believe impact the goalie's performance. Put the same goalie in front an all star D vs a garbage D (say Canes vs Mtl s D) - the same wrist shot from the point will require much more attention when playing behind MTL's D

 

[morehockeystats]

I played goaltender for a number of years.

Based on your example, yes, I believe that skaters can significantly impact a goalie's performance.

ie. a forward who gets to hang onto the puck for an extended period of time, with open passing lanes is much more dangerous than one in the same position on the ice who's pressured and/or doesn't have good passing lanes.

And then, are rebound shots classified? A shot in the slot from rebounds because the D can't clear the slot is much more dangerous than a shot from the slot from a forward thats on a rush and covered by a D.

Overall, I dont think you'll see an impact from single skaters when aggregatinf everyone, but theres five skaters on the ice which, as a group, I believe impact the goalie's performance. Put the same goalie in front an all star D vs a garbage D (say Canes vs Mtl s D) - the same wrist shot from the point will require much more attention when playing behind MTL's D

The pressured forward generates less xGA.

The rebound shots are classified, extra xGA is added for them.

All those cases impact xGA, not the f(GA,xGA) that Micah is measuring as goaltender's performance. He classifies those xGA changes as 'defensive impact of a player'.

Except the last sentence. According to Micah and his findings similar shots don't, after all, differentiate. His findings indicate function f(GA, xGA) of a given goalie doesn't change because of the corps in front.

As I said, there's a perception catch. If the D allowed for 15 high danger chances which resulted in 9 goals vs the defense that allowed 5 high danger chances where the result was 3 goals, the G performance might as well be the same. Micah didn't find a consistent case where similar 5 high danger chances same goalie faced would result in 3 goals with players X,Y,Z on the ice and in 1 goal with players A,B,C.

 

[lomiller1]

I am not fully versed in the xGA metric. But I have to presume that it is still flawed. From what I do know it is based mostly on location of shot and (perhaps) whether the shot is "clear." The second is likely to be subjective.

It accounts for:
Shot type (As Tracked by NHL)
Shot location (as tracked by NHL)
Rebound\tip (based on shot attempt timing)
I seem to recall it can capture some info on Rush vs Cycle but I don't remember how that is extracted.

 

[morehockeystats]

It accounts for:
Shot type (As Tracked by NHL)
Shot location (as tracked by NHL)
Rebound\tip (based on shot attempt timing)
I seem to recall it can capture some info on Rush vs Cycle but I don't remember how that is extracted.

I extract rushes (when I can detect an event very far behind from the shot location within a few second prior to the shot). Also, I track recent giveaways and recent takeaways (especially when across ice west-east), the goal difference (e.g. desperate weaker shots), the on-ice body count and the catching/shooting side combined with the side of the ice (e.g. RRR) to evaluate the danger of the shot.

 

[Ted Hoffman]

I would posit that the assertion is true provided that xGA is being measured properly. Spoiler: it's almost certainly not, unless someone is measuring xGA not just on the (x,y) location of the shot on the ice surface, but is factoring in at least the following:

* The velocity v of the shot​

* The angle (α, θ) of the shot - specifically, the (y,z) spot of the net the puck is shot toward​

* The direction γ, distance d and velocity v' of any pass to the shooter, given the (x,y) location of the pass​

* The length of time t' the shooter handles the puck before shooting​

* Whether the puck is deflected in any way, where (x,y,z) the puck is deflected, and the resulting new trajectory (v, α, θ) of the puck after deflection​

* The goalie's (x,y) position on the ice when the puck is shot​

* The goalie's specific stance (upright, butterfly, double-pad stack, etc.)​

​

If all of those are being measured and have an xGA that can be calculated given all of those measurements, maybe I might buy the argument. But that still requires actual GA relative to xGA to move toward its goalie-specific ratio over some time t, and for that ratio to not change based on anything else.

See where I'm going with this? It requires a metric f***ton of data to get an accurate calculation of xGA, and then the assumption that all other variables that impact a game's outcome have zero impact on GA relative to xGA. Oh, and it also implies that said ratio is the same whether skaters are on the ice or not, or where they are on the ice. [Unless you're factoring that into your xGA, which ... if you're going to incorporate all the stuff I list above, you should probably incorporate that, too. Cause, I didn't mention if the goalie is screened, or has been bumped at some point prior to the shot, or other things that impact a goalie's ability to focus on a shot / see it to be able to stop it. A few other things too, which I'll leave to everyone else to figure out. I'm not doing all your deep thinking for free.]

Short: I think it's a claim that might stand up, provided one can measure xGA at a sufficiently granular level to account for all possible variables that impact the probability p of any shot resulting in a goal. I think all teh analytics folks are a long, long, long way away from accomplishing that, though. In the current world, given how xGA is by and large getting measured? No way in hell is that claim remotely correct.

 

[morehockeystats]

I would posit that the assertion is true provided that xGA is being measured properly. Spoiler: it's almost certainly not, unless someone is measuring xGA not just on the (x,y) location of the shot on the ice surface, but is factoring in at least the following:

* The velocity v of the shot​

* The angle (α, θ) of the shot - specifically, the (y,z) spot of the net the puck is shot toward​

* The direction γ, distance d and velocity v' of any pass to the shooter, given the (x,y) location of the pass​

* The length of time t' the shooter handles the puck before shooting​

* Whether the puck is deflected in any way, where (x,y,z) the puck is deflected, and the resulting new trajectory (v, α, θ) of the puck after deflection​

* The goalie's (x,y) position on the ice when the puck is shot​

* The goalie's specific stance (upright, butterfly, double-pad stack, etc.)​

​

If all of those are being measured and have an xGA that can be calculated given all of those measurements, maybe I might buy the argument. But that still requires actual GA relative to xGA to move toward its goalie-specific ratio over some time t, and for that ratio to not change based on anything else.

See where I'm going with this? It requires a metric f***ton of data to get an accurate calculation of xGA, and then the assumption that all other variables that impact a game's outcome have zero impact on GA relative to xGA. Oh, and it also implies that said ratio is the same whether skaters are on the ice or not, or where they are on the ice. [Unless you're factoring that into your xGA, which ... if you're going to incorporate all the stuff I list above, you should probably incorporate that, too. Cause, I didn't mention if the goalie is screened, or has been bumped at some point prior to the shot, or other things that impact a goalie's ability to focus on a shot / see it to be able to stop it. A few other things too, which I'll leave to everyone else to figure out. I'm not doing all your deep thinking for free.]

Short: I think it's a claim that might stand up, provided one can measure xGA at a sufficiently granular level to account for all possible variables that impact the probability p of any shot resulting in a goal. I think all teh analytics folks are a long, long, long way away from accomplishing that, though. In the current world, given how xGA is by and large getting measured? No way in hell is that claim remotely correct.

Or, all seven bullets above do not correlate significantly with the defensive corps on the ice.
I continue to insist that many people who object Micah's point, actually have a different definition of a "goaltender's performance", but this difference makes the whole argument moot.

 

[Ted Hoffman]

Or, all seven bullets above do not correlate significantly with the defensive corps on the ice.

If you think the first 5 items "do not correlate significantly" with the defensive corps on the ice, ... well, there's a reason why I call much of what people are doing these days teh analytics. And no, I don't mean that in the nicest way possible, either.

Put another way: if you think the first 5 items "do not correlate significantly" then it implies that an offense generates chances that create an xGF regardless of what the defense does - which is to say, defensive strategy is useless and one would get the same results whether the defense was actively trying to prevent the opposition from scoring, playing a passive zone, or everyone was standing over at the bench woofing down hot dogs, nachos and beer watching on cheering the goalie on for all his effort. It should be trivial to see that's incorrect. The actions of the offense aren't invariant to the actions of the defense.

The defense does not impact the last two items I list, nor would I have ever asserted that. [Well, I might say very likely does not while leaving the door open for the possibility that a goalie may tweak his style based on who's in front of him at a given time - though I think any impact might be statistical noise, and certainly would require an extensive volume of data to tease that out.] The last two could [should] have an impact on what the defense does, and likely does have an impact on what the offense does - we hear from time-to-time, especially in the playoffs, about teams trying to make use of an opposing goalie's weaknesses - though how much is open to debate. [I leave that to others to discuss.]

If the problem is Micah's definition of "goaltender's performance" then perhaps Micah needs to explain that more clearly, with whatever qualifiers are necessary. Until then, it appears to me he's done data vomit like most good teh analytics folks do, cranked out a bunch of charts and graphs that cause people to go ooh, aah and lobbed out some numbers, and there's your answer and everyone's just supposed to accept it without question. I suspect somewhere in there, he's missed a simple question: how do I test the thing I want to examine to get a proper answer? Or, maybe more critically, do I have all the data in the level of detail I need to make a proper analysis of what I'm trying to test?

[Further criticism of his claims in the tweets reserved, because ... I have things to go do today. Plus, I want to see if anyone else figures out the conclusions that logically fall out from a couple of the statements made.]

 

[morehockeystats]

If you think the first 5 items "do not correlate significantly" with the defensive corps on the ice, ... well, there's a reason why I call much of what people are doing these days teh analytics. And no, I don't mean that in the nicest way possible, either.

I admit I didn't attribute the "all seven" correctly. I should've stuck to 6 and 7, because the top five actually affect the xGA, not the performance. I got confused by you bundling them all together when they address different things. 6 and 7 do not address the xGA of a shot. The xGA value is the same for more or less any two goalies and is set the moment the puck leaves the stick.

You haven't understood. The defensive strategy is intended to minimize the xGA value. The effect of a skater on the goalie performance would be a different GA by the goalie given the SAME xGA with that skater on and off the ice. You can bundle as many factors into xGA, but it does not affect this statement.

 

[Steerpike]

Wait isn't all this is saying is that the variability in defensive performance is well captured by XGA?

That's not saying defenders have no impact on how many goals go in.... Just that they only really impact XGA and after that it's really on the goalie/random.

 

[Bank Shot]

"Skater performance has no correlation with how well goaltenders perform relative to their xGA. Goals against should therefore not be used as a basis for skaters defensive ability."

This is a faulty conclusion, because a skater's defensive ability can still affect the amount of dangerous shots the goalie faces, which can affect goals against.

Might be a faulty conclusion but I think its fairly obvious that goals against shouldn't be used to measure a players defensive ability.

The whole reason advanced stats guys went to measure shot attempts was to get a bigger sample size than goals/shots. A guy could easily have a skewed goals against because of a few bad goals or bad changes through no fault of his own.

Defensive ability is pretty hard to measure. The best way would be measuring how few slot chances given up when a player is on the ice when weighed against opponents and zone starts. How those factors could be weighted properly is anyone's guess.....

 

[bossram]

Wait isn't all this is saying is that the variability in defensive performance is well captured by XGA?

That's not saying defenders have no impact on how many goals go in.... Just that they only really impact XGA and after that it's really on the goalie/random.

Well yeah, Micah's thread is basically showing that skaters can impact xGA or chances against, but it's up to the goalies to stop those chances. If a skater allows a lot of xGA, they'll be appropriately dinged in their own defensive metrics.

If you have two skaters who have allowed 10 goals against, but one allowed 15 xGA and the other 5 xGA, it doesn't really make sense to call them equivalent defensively.

This is obviously predicated on an xG model accurately capturing the likelihood of certain shots going in. Now, we know the public models don't have access to things like passing data, puck movement or pressure, which leaves a clear gap in the public xG models. But the private models (like SportLogiq) do have this kind of data and, crucially, their results don't seem to be drastically different from the public ones (gleaned from whenever someone divulges some information from the private models on twitter).

 

[Ted Hoffman]

I was going to leave this alone, then decided nah, I want to try it one more time and see if people "get it" finally.

It's accurate to say that actual +/- is a poor (though not worthless) measure of offensive/defensive ability, and expected +/- is a better measure. (Though still not perfect either.) I'm sure someone is ready to say "... and the difference between actual and expected +/- is on the goalies!" No. While goalie play has an impact, it's also impacted by what all the other skaters on the ice are doing that creates that expected value. [There's a multi-million dollar idea here, if you understand what this leads to. No, no one has scratched the surface on it yet.]

It is accurate to say that goalies do not impact the offensive/defensive measures of skaters, provided that we're talking expected values. It is not accurate to say that skaters do not impact the absolute performance of goalies [which should be obvious], and I'd argue that certain actions of skaters can impact the performance of goalies relative to expectation unless you're capturing expected performance at a sufficient level of granularity to account for all the actions and positions of skaters. Which, I can promise current xGF/xGA models do not - and which is the crux of the problem I have with the assertions made above as irrefutable truths.

[An entire discussion of actual vs. expected values is omitted. Spoiler: no matter how much you all try to model away randomness, you'll never do it and have a workable model - and that's the downfall of a f***load of teh analytics work that goes on. Well, that + other stuff, but I only have so many internet characters to use this month.]

The idea that GA shouldn't be used to measure a skater's defensive ability? Hogwash. It's not a great stat, it's a poor measure, but like all statistics it requires proper interpretation to understand when it's useful and when it's not. That's an exercise for individuals, and that can be done properly without someone generating 667GB of data to create models with 374 possible variables that need to run for 18.26 hours to come up with "brilliant" conclusions.

 

[morehockeystats]

I am sure, in the fine tradition of Karl Popper, @Ted Hoffman is able to come up with a decently sized set of skaters and goalies that refute Micah's theory. That only needs to be done once, but it needs to be rigorous.

 

[bossram]

I was going to leave this alone, then decided nah, I want to try it one more time and see if people "get it" finally.

It's accurate to say that actual +/- is a poor (though not worthless) measure of offensive/defensive ability, and expected +/- is a better measure. (Though still not perfect either.) I'm sure someone is ready to say "... and the difference between actual and expected +/- is on the goalies!" No. While goalie play has an impact, it's also impacted by what all the other skaters on the ice are doing that creates that expected value. [There's a multi-million dollar idea here, if you understand what this leads to. No, no one has scratched the surface on it yet.]

It is accurate to say that goalies do not impact the offensive/defensive measures of skaters, provided that we're talking expected values. It is not accurate to say that skaters do not impact the absolute performance of goalies [which should be obvious], and I'd argue that certain actions of skaters can impact the performance of goalies relative to expectation unless you're capturing expected performance at a sufficient level of granularity to account for all the actions and positions of skaters. Which, I can promise current xGF/xGA models do not - and which is the crux of the problem I have with the assertions made above as irrefutable truths.

[An entire discussion of actual vs. expected values is omitted. Spoiler: no matter how much you all try to model away randomness, you'll never do it and have a workable model - and that's the downfall of a f***load of teh analytics work that goes on. Well, that + other stuff, but I only have so many internet characters to use this month.]

The idea that GA shouldn't be used to measure a skater's defensive ability? Hogwash. It's not a great stat, it's a poor measure, but like all statistics it requires proper interpretation to understand when it's useful and when it's not. That's an exercise for individuals, and that can be done properly without someone generating 667GB of data to create models with 374 possible variables that need to run for 18.26 hours to come up with "brilliant" conclusions.

I'm not sure what you're arguing is here is even relevant.

No one reasonably knowledgable in this discussion is saying that skaters don't impact the absolute performance of goaltenders. They impact the goalies performance by allowing varying degrees of chances (xGA). The difference between the expected and actual is determined to be left to the goalies.

Yes, it is predicated on having an accurate xG model. The public ones aren't able to capture everything because the data isn't there. But crucially, the inklings we get from the private xG models, with thousands more inputs, don't really seem very different. I'd say the current models are doing a good enough job.

Yeah, you can use GA to measure a skater's performance if you want. It's just not a particularly useful measure.