All Purpose Analytics and Extended Stats Discussion

Hivemind

We're Touched
Oct 8, 2010
37,008
13,424
Philadelphia
So, if it's really a function of the goalie, and nothing else (summary), then each teams save percentage rankings on the PK (and goals against) should mirror those ES. Right?

Not going to wade too far into this discussion, other than to say that would only be true if the attributes that made a goalie good at even strength were perfectly overlapping with the attributes that make a goalie good on the penalty kill.

Moreover, from a statistical standpoint, PK sv% has not shown itself to be a particularly repeatable skill.
 

Ridley Simon

Registered User
Feb 27, 2002
18,123
9,066
Marin County — SF Bay Area, CA
Not going to wade too far into this discussion, other than to say that would only be true if the attributes that made a goalie good at even strength were perfectly overlapping with the attributes that make a goalie good on the penalty kill.

Moreover, from a statistical standpoint, PK sv% has not shown itself to be a particularly repeatable skill.

I feel like the arguments being placed here is another shot at the perception around "Defensive Hockey" and the lack of statical basis to quantify some of that.

We all know (or should know) that not every player plugged into all these PK situations would perform the same. That seems to be the root of the narrative, simply put.

I can't imagine any hockey professional buying into that either. Even Chayka.

Edit --- I'm putting myself in the Penalty Box (need a a Time Out). I vowed to stay out of these analytics discussions and never ever venture into the Fancy Stat thread. LO and behold I find myself and my comments moved into it after a few hours away from the IPad.

BOOOOOOOOOOO
 
Last edited:

twabby

Registered User
Mar 9, 2010
13,623
14,439
So, if it's really a function of the goalie, and nothing else (summary), then each teams save percentage rankings on the PK (and goals against) should mirror those ES. Right?

In theory, if specific "PK Abilty" doesn't exist, then there shouldn't be a team that is abjectedly better at PK percentages and/or GAA rankings than they are at ES percentages/GAA.

Not sure if that's borne out by the numbers or not.

I don't think the author concluded that PKing is solely a function of the goaltender, just that individual skaters don't have a huge impact on penalty killing compared to their impact at even strength and on the power play. It's never suggested that systems, coaching, and the opposition (and their tactics) don't have a significant impact.
 

CapitalsCupReality

It’s Go Time!!
Feb 27, 2002
64,374
19,069
A hockey professional like an NHL coach or player would probably fall out of their chair laughing at this...."notion".
 

twabby

Registered User
Mar 9, 2010
13,623
14,439
A hockey professional like an NHL coach or player would probably fall out of their chair laughing at this...."notion".

Most hockey professionals also used to think that size and grit were more important than speed and skill, and now those opinions have been completely changed. Ideas evolve as new information is collected. I'd rather look at objective analysis than appeals to authority.

And it's not like the model ignores defensive impact. There is a whole part of the model devoted to even strength defense which concludes players do have significantly different abilities at even strength defense. Here is the data released today:

https://t.co/FJ35D3WiAr
 
Last edited:

Hivemind

We're Touched
Oct 8, 2010
37,008
13,424
Philadelphia
There's a great deal of selection bias inherent to that study. Only players coaches deem trustworthy of PK duty are given any significant PK minutes. The study concludes there's little difference between a top flight PKer and a lower skilled PKer, not that there's little difference between a top flght PKer and a player not trusted on the PK at all.

That being said, there's no real evidence to support ANY conclusions regarding players who are never trusted on the PK in the first place. Simply stating that a player doesn't receive PK minutes and is therefor not a good PKer is not a sufficient answer to the question.
 

Ridley Simon

Registered User
Feb 27, 2002
18,123
9,066
Marin County — SF Bay Area, CA
There's a great deal of selection bias inherent to that study. Only players coaches deem trustworthy of PK duty are given any significant PK minutes. The study concludes there's little difference between a top flight PKer and a lower skilled PKer, not that there's little difference between a top flght PKer and a player not trusted on the PK at all.

That being said, there's no real evidence to support ANY conclusions regarding players who are never trusted on the PK in the first place. Simply stating that a player doesn't receive PK minutes and is therefor not a good PKer is not a sufficient answer to the question.

This is all true. It's akin to Special Teams in football. You don't put your stars out there to do it (for a variety of reasons), and it's similar for PK'ing.

I get that you dont want to pay a player a substantial contract on PK'ing alone, just as you wouldn't with special teams. But that's a far cry from discrediting the abilities of players that excel there (not at you, Hivemind, you haven't done that).
 

CapitalsCupReality

It’s Go Time!!
Feb 27, 2002
64,374
19,069
Most hockey professionals also used to think that size and grit were more important than speed and skill, and now those opinions have been completely changed. Ideas evolve as new information is collected. I'd rather look at objective analysis than appeals to authority.

And it's not like the model ignores defensive impact. There is a whole part of the model devoted to even strength defense which concludes players do have significantly different abilities at even strength defense. Here is the data released today:

https://t.co/FJ35D3WiAr

The league has changed and will continue to change. There was a time when size and grit were more of a premium. However given the choice even today between a big guy with skill and speed and a small guy with skill and speed, who do you think a coach or GM wants? It's going to be the guy with skill AND physical gifts more often than not.

What won't change, that you love to discount because there's no stat column you can look to for understanding, is that coaches use their experience and knowledge to assess players and decide who gets what assignment. A smart man doesn't need to cut off his own finger to prove a knife is sharp. Trotz doesn't need to see Orlov or Schmidt on the PK in the playoffs to believe he has a lot of better options.
 

twabby

Registered User
Mar 9, 2010
13,623
14,439
The league has changed and will continue to change. There was a time when size and grit were more of a premium. However given the choice even today between a big guy with skill and speed and a small guy with skill and speed, who do you think a coach or GM wants? It's going to be the guy with skill AND physical gifts more often than not.

What won't change, that you love to discount because there's no stat column you can look to for understanding, is that coaches use their experience and knowledge to assess players and decide who gets what assignment. A smart man doesn't need to cut off his own finger to prove a knife is sharp. Trotz doesn't need to see Orlov or Schmidt on the PK in the playoffs to believe he has a lot of better options.

I'm not saying that some players aren't better than others at PKing. I'm saying that the difference in individual players' PKing ability is minimal and perhaps negligible compared to other aspects of their game according the model I linked. I don't think it's a foregone conclusion that the Capitals PK would suffer much if Orlov and Schmidt had to replace Alzner and Orpik and everything else stayed the same, for instance. Maybe less than a goal or two on average over the course of an entire season. If you're going to pay millions of dollars for a player, I'd hope that they bring more to the table than just the ability to kill penalties.

There's a great deal of selection bias inherent to that study. Only players coaches deem trustworthy of PK duty are given any significant PK minutes. The study concludes there's little difference between a top flight PKer and a lower skilled PKer, not that there's little difference between a top flght PKer and a player not trusted on the PK at all.

That being said, there's no real evidence to support ANY conclusions regarding players who are never trusted on the PK in the first place. Simply stating that a player doesn't receive PK minutes and is therefor not a good PKer is not a sufficient answer to the question.

I would counter that most teams go through enough injuries that players who previously weren't trusted to PK are forced into that role and that the model captured the times where these previously "untrusted" players did in fact have to kill penalties and still the conclusion was that these untrusted PKers weren't significantly worse than normal PKers. Obviously the sample size would likely be smaller, but it's probably still significant enough to draw this conclusion.
 
Last edited:

twabby

Registered User
Mar 9, 2010
13,623
14,439
It probably is minimal, but that one goal might be the difference in winning or losing as series.

Sure, but unless you know that one goal is going to occur during a specific high-leverage moment during the postseason why not just play the odds and try to maximize the overall impact of your roster in other aspects of the game where players' abilities are much more apparent?
 

Revelation

Registered User
Aug 15, 2016
5,298
2,963
It probably is minimal, but that one goal might be the difference in winning or losing as series.

Does that also apply to goals not scored due to having a less offensively/transitionally capable defenseman in the lineup? Goals allowed due to being unable to sustain offensive pressure/transition at even strength? Or just goals surrendered on the PK.
 

txpd

Registered User
Jan 25, 2003
69,649
14,131
New Bern, NC
Caps lost game 6 to the Pens in OT. In a 1-1 game with Alzner off the ice injured, Orpik took a double minor. Dumb. So dumb. Why dumb? They are two of the 4 penalty kill defensemen. Caps gave up 2ppga's and lost the game and were eliminated from the series.
 

Revelation

Registered User
Aug 15, 2016
5,298
2,963
Caps lost game 6 to the Pens in OT. In a 1-1 game with Alzner off the ice injured, Orpik took a double minor. Dumb. So dumb. Why dumb? They are two of the 4 penalty kill defensemen. Caps gave up 2ppga's and lost the game and were eliminated from the series.

Caps also lost Game 7 to the Rags in OT the year before with both Alzner and Orpik on the ice defending.
 

g00n

Retired Global Mod
Nov 22, 2007
30,430
14,290
I'm not saying that some players aren't better than others at PKing. I'm saying that the difference in individual players' PKing ability is minimal and perhaps negligible compared to other aspects of their game according the model I linked. I don't think it's a foregone conclusion that the Capitals PK would suffer much if Orlov and Schmidt had to replace Alzner and Orpik and everything else stayed the same, for instance. Maybe less than a goal or two on average over the course of an entire season. If you're going to pay millions of dollars for a player, I'd hope that they bring more to the table than just the ability to kill penalties.



I would counter that most teams go through enough injuries that players who previously weren't trusted to PK are forced into that role and that the model captured the times where these previously "untrusted" players did in fact have to kill penalties and still the conclusion was that these untrusted PKers weren't significantly worse than normal PKers. Obviously the sample size would likely be smaller, but it's probably still significant enough to draw this conclusion.


Can we admit that the assumptions being made are selective? As in, data is the rule sometimes but when the rules break down assumptions about things evening out or not mattering are being made?

Also, if we're talking a few more goals per year given up and we're OK with it then why is it so bad when shooting stats are a few percentage points lower for one guy, essentially meaning the same thing (a few less net goals-for per year)? Why is it a big deal in one case but not the other?
 

twabby

Registered User
Mar 9, 2010
13,623
14,439
Can we admit that the assumptions being made are selective? As in, data is the rule sometimes but when the rules break down assumptions about things evening out or not mattering are being made?

Also, if we're talking a few more goals per year given up and we're OK with it then why is it so bad when shooting stats are a few percentage points lower for one guy, essentially meaning the same thing (a few less net goals-for per year)? Why is it a big deal in one case but not the other?

Not sure what you mean in your first paragraph.

In regards to your second paragraph, I was just spitballing numbers for the drop off in PKing from Orpik-Alzner to Orlov-Schmidt. I would imagine the author of the study would only drop the PKing from his model if the goals-above-replacement was much less significant than other aspects such as faceoffs, drawing/not taking penalties, even strength defense and offense, and power play offense. So maybe the drop off from top to bottom-end PKers is closer to like .5 or fewer goals on average over the course of the season.

I've asked the author for a little more detail on why he dropped PKing and am currently awaiting a response. Even if the GAR is small, I'm not sure why he wouldn't just include it in his model for completeness.
 

g00n

Retired Global Mod
Nov 22, 2007
30,430
14,290
Not sure what you mean in your first paragraph.

In regards to your second paragraph, I was just spitballing numbers for the drop off in PKing from Orpik-Alzner to Orlov-Schmidt. I would imagine the author of the study would only drop the PKing from his model if the goals-above-replacement was much less significant than other aspects such as faceoffs, drawing/not taking penalties, even strength defense and offense, and power play offense. So maybe the drop off from top to bottom-end PKers is closer to like .5 or fewer goals on average over the course of the season.

I've asked the author for a little more detail on why he dropped PKing and am currently awaiting a response. Even if the GAR is small, I'm not sure why he wouldn't just include it in his model for completeness.

I mean, as we've talked about before, there are assumptions being made when convenient. Like assuming guys get shuffled around from PK1 to PK2 enough for it to not matter. And that changing trends in playing style over the years means criticism of stat applications is somehow anti-progress. And so forth.
 

twabby

Registered User
Mar 9, 2010
13,623
14,439
I mean, as we've talked about before, there are assumptions being made when convenient. Like assuming guys get shuffled around from PK1 to PK2 enough for it to not matter. And that changing trends in playing style over the years means criticism of stat applications is somehow anti-progress. And so forth.

I'm not going to speak for the author in regards to the PK usage assumption. He included the opposition players/coaches and zone starts in his model so I'll take him at his word, but again I'm not sure why he excluded shorthanded defense altogether because why not just include it for completeness?

It's fine to critique statistical methods and that's why there has been so much progress. Old methods such as Fenwick-close are relics at this point when they used to be seen as the gold standard. I'm sure there will be even more progress in the coming years.

But it's not fair to completely dismiss a model just because it does not 100% explain everything. And in this case, the model suggests that there isn't really discernible skill in PKing amongst most NHL-level players. Just because it seems unintuitive doesn't necessarily mean the model is wrong. The authors did the legwork and made this conclusion based on data so it's worth it to at least consider it as possible until it is shown to be out of whack.
 

CapitalsCupReality

It’s Go Time!!
Feb 27, 2002
64,374
19,069
This is where you lose me.

"And in this case, the model suggests that there isn't really discernible skill in PKing amongst most NHL-level players."

I feel confident any professional coach or player would find this absurdly wrong. More than enough reason to dismiss it.
 

twabby

Registered User
Mar 9, 2010
13,623
14,439
That's missing the point entirely. These new models set out to challenge old schools of thought and clarify the game so of course there is going to be resistance from the status quo. But isn't that true of basically all types of research, knowledge, and schools of thought? Almost every new scientific theory has faced large amounts of resistance before ultimately gaining acceptance, even things that seem incredibly obvious like the Earth being round (unless you're Shaquille O'Neal of course).

Again, I have reservations about the author not including the GAR impact of shorthanded defense but unless someone actually finds a flaw with the methodology I'm not willing to say his assertion is wrong. It's a more extreme version of the view I already held that defensive talent is very tightly packed relative to other facets of the game, so it's something I'm absolutely willing to consider.
 
Last edited:

g00n

Retired Global Mod
Nov 22, 2007
30,430
14,290
I'm not going to speak for the author in regards to the PK usage assumption. He included the opposition players/coaches and zone starts in his model so I'll take him at his word, but again I'm not sure why he excluded shorthanded defense altogether because why not just include it for completeness?

It's fine to critique statistical methods and that's why there has been so much progress. Old methods such as Fenwick-close are relics at this point when they used to be seen as the gold standard. I'm sure there will be even more progress in the coming years.

But it's not fair to completely dismiss a model just because it does not 100% explain everything. And in this case, the model suggests that there isn't really discernible skill in PKing amongst most NHL-level players. Just because it seems unintuitive doesn't necessarily mean the model is wrong. The authors did the legwork and made this conclusion based on data so it's worth it to at least consider it as possible until it is shown to be out of whack.

That's missing the point entirely. These new models set out to challenge old schools of thought and clarify the game so of course there is going to be resistance from the status quo. But isn't that true of basically all types of research, knowledge, and schools of thought? Almost every new scientific theory has faced large amounts of resistance before ultimately gaining acceptance, even things that seem incredibly obvious like the Earth being round (unless you're Shaquille O'Neal of course).

Again, I have reservations about the author not including the GAR impact of shorthanded defense but unless someone actually finds a flaw with the methodology I'm not willing to say his assertion is wrong. It's a more extreme version of the view I already held that defensive talent is very tightly packed relative to other facets of the game, so it's something I'm absolutely willing to consider.

/eyeroll

Well you're essentially admitting that your own bias and preconceived notions are coloring where you're willing to place the benefit of the doubt. It's also just as possible that the flaws here are fatal.

Your continual characterization of concerns about the methodology and conclusions as some kind of luddite obstacle to historical progress is pretty ridiculous grandstanding, twabbs. :laugh:
 

twabby

Registered User
Mar 9, 2010
13,623
14,439
/eyeroll

Well you're essentially admitting that your own bias and preconceived notions are coloring where you're willing to place the benefit of the doubt. It's also just as possible that the flaws here are fatal.

Your continual characterization of concerns about the methodology and conclusions as some kind of luddite obstacle to historical progress is pretty ridiculous grandstanding, twabbs. :laugh:

I'm always going to side with data over gut feel and anecdotal evidence if an idea is in doubt. If that's a bias then I guess I'm guilty. It is possible the flaws in the model are significant but it's been out for several months now and no one has shown any fatal flaw.

I've been hard on Brooks Orpik in the past, but given the data (not just in this model) it seems like he is very comfortably a solid bottom 4 defenseman and I'm willing to say my old opinion on him was wrong (that he was barely an NHLer). So let's not pretend I pick and choose when to use statistical arguments based on my preconceived notions.

The concerns expressed by you and others about this model aren't based on data, an objective analysis, or a rigorous scrutiny of the model, they're based on your preconceived notions about penalty killing and appealing to authority. You and coaches think players have vastly different PKing abilities, therefore the model is flawed. Sorry, that's not good enough.
 

g00n

Retired Global Mod
Nov 22, 2007
30,430
14,290
I'm always going to side with data over gut feel and anecdotal evidence if an idea is in doubt. If that's a bias then I guess I'm guilty. It is possible the flaws in the model are significant but it's been out for several months now and no one has shown any fatal flaw.

I've been hard on Brooks Orpik in the past, but given the data (not just in this model) it seems like he is very comfortably a solid bottom 4 defenseman and I'm willing to say my old opinion on him was wrong (that he was barely an NHLer). So let's not pretend I pick and choose when to use statistical arguments based on my preconceived notions.

The concerns expressed by you and others about this model aren't based on data, an objective analysis, or a rigorous scrutiny of the model, they're based on your preconceived notions about penalty killing and appealing to authority. You and coaches think players have vastly different PKing abilities, therefore the model is flawed. Sorry, that's not good enough.

Like I and others have said before, the objections can't be reduced to dismissable terms like "gut feel" and "anecdotal evidence". Much of what goes on can be accurately described by hockey people as specific skills and techniques that can be done well or poorly. They just can't be put into numbers for you.

You're presenting a very slanted case.
 

twabby

Registered User
Mar 9, 2010
13,623
14,439
Like I and others have said before, the objections can't be reduced to dismissable terms like "gut feel" and "anecdotal evidence". Much of what goes on can be accurately described by hockey people as specific skills and techniques that can be done well or poorly. They just can't be put into numbers for you.

You're presenting a very slanted case.

These skills and techniques that can be done well or poorly according to coaches theoretically can be converted to numbers, it's just very difficult. The expected value of boxing a player out on a point shot vs. allowing that same point shot to be screened is very difficult to calculate but why can't this be quantified given enough data? When coaches say certain skills and techniques are important they are implicitly saying they impact goal-differential, so yes they can be quantified.

There's no doubt in my mind that there are skills and techniques necessary in order to be an effective penalty killer. Otherwise, you or I could kill penalties and get the same results as the pros. But that's not the case being made. The case being made is that amongst NHL skaters, holding all else equal (coaching, goaltending, the opposing players) the impact of individual PKing ability is negligible. Perhaps this is because the skills and techniques necessary to kill penalties are rather routine once you are at the NHL level and that perfecting these skills and techniques offer diminishing returns.
 

g00n

Retired Global Mod
Nov 22, 2007
30,430
14,290
These skills and techniques that can be done well or poorly according to coaches theoretically can be converted to numbers, it's just very difficult. The expected value of boxing a player out on a point shot vs. allowing that same point shot to be screened is very difficult to calculate but why can't this be quantified given enough data? When coaches say certain skills and techniques are important they are implicitly saying they impact goal-differential, so yes they can be quantified.

There's no doubt in my mind that there are skills and techniques necessary in order to be an effective penalty killer. Otherwise, you or I could kill penalties and get the same results as the pros. But that's not the case being made. The case being made is that amongst NHL skaters, holding all else equal (coaching, goaltending, the opposing players) the impact of individual PKing ability is negligible. Perhaps this is because the skills and techniques necessary to kill penalties are rather routine once you are at the NHL level and that perfecting these skills and techniques offer diminishing returns.

JmWlE_s-200x150.gif


Ok, man. Whatever you say.
 

Ad

Upcoming events

Ad

Ad

-->