Philosophy of hockey sabermetrics: Can hockey accurately be measured?

I find a problem with basically all studies here (including my own) in that they require a lot of work and time. There are usually many hours of...
Rating:
4/5,
  1. Mathletic

    Mathletic Registered User

    Joined:
    Feb 28, 2002
    Messages:
    15,769
    Likes Received:
    344
    Trophy Points:
    199
    Location:
    Ste-Foy
    Finally had time to read "The Signal and the Noise" by Nate Silver. Few excerpts I liked from the chapter on baseball.

    On the "fear" that analysts would be taking jobs from scouts:

    Beane told me the A’s scouting budget is now much higher than it has ever been. Moreover, he said it was the A’s fascination with statistical analysis that led them to increase it. As we’ve seen, baseball players do not become free agents until after six full seasons, which is usually not until they’re at least thirty. As Bill James’s analysis of the aging curve revealed, this often leads clubs to overspend on free agents—after all, their best years are usually behind them. But there is a flip side to this:*before*a player is thirty, he can provide tremendous value to his club. Moreover, baseball’s economics are structured such that younger players can often be had for pennies on the dollar

    On objective analysis vs. gut-feel decisions:

    “From our standpoint in Oakland, we’re sort of forced into making objective decisions versus gut-feel decisions. If we in Oakland happen to be right on a gut-feel decision one time, my guess is it would be random. And we’re not in a position to be making random decisions and hope we get lucky. If we’re playing blackjack, and the dealer’s showing a four and we have a six, hitting on the sixteen just doesn’t make sense for us.â€

    On the decision-making process:

    The key to making a good forecast, as we observed in chapter 2, is not in limiting yourself to quantitative information. Rather, it’s having a good process for weighing the information appropriately. This is the essence of Beane’s philosophy: collect as much information as possible, but then be as rigorous and disciplined as possible when analyzing it.

    On categorization:

    When we can’t fit a square peg into a round hole, we’ll usually blame the peg—when sometimes it’s the rigidity of our thinking that accounts for our failure to accommodate it. Our first instinct is to place information into categories—usually a relatively small number of categories since they’ll be easier to keep track of.

    This might work well enough most of the time. But when we have trouble categorizing something, we’ll often overlook it or misjudge it. This is one of the reasons that Beane avoids what he calls “gut-feel†decisions. If he relies too heavily on his first impressions, he’ll let potentially valuable prospects slip through the cracks—and he can’t afford that with a payroll like Oakland’s.
     
  2. Doctor No

    Doctor No Registered User

    Joined:
    Oct 26, 2005
    Messages:
    8,098
    Likes Received:
    1,636
    Trophy Points:
    149
    Home Page:
    When I give modeling presentations, there's a joke that I've probably used to death at this point - we take great care to build these elaborate models that predict human behavior perfectly with great accuracy.

    And then imperfect humans have to come along and ruin our perfect model by acting irrationally.
     
  3. Smokey McCanucks

    Smokey McCanucks Registered User

    Joined:
    Dec 21, 2010
    Messages:
    2,693
    Likes Received:
    0
    Trophy Points:
    0
    I feel like it would take an absolutely immense amount of work to get statistical analysis of hockey to the point where the insight it provides will approach the insight that a knowledgeable and experienced scout would get just from watching and reflecting on a player or team. Of course the benefit is that even if it doesn't approach that level of insight, a collection of stats would be widely available, whether publicly or in-house, you wouldn't be relying on the views and opinions of an individual - you'd be relying on something that's objective and that can be easily disseminated.

    Now, if I'm an executive and I'm going to be drafting a kid, I for sure want to have people I trust and whose opinions I value who have seen him play a lot of games in person and got a feel for his game and his tendencies. It's not going to be stats that are ruling the day there, not ever. There are just too many variables in hockey, you can't account for them all in a statistical model, not to the level that you'd have to be able to reach in order to form a real strong opinion.

    But for sure those statistical models have a very important place, and it's getting much more significant. It's having an effect already, teams are changing their playstyles and changing their points of emphasis based on - just one thing - the idea that possession correlates to winning over the long haul, it doesn't matter whether you think the possession stats are accurate or useful or not, you still can see that that has had a clear effect on the style of play of a lot of teams.

    It's had a huge impact, even when you hear Doughty saying "Corsi's crap," well, a big part of them winning those two Cups was Sutter putting major emphasis on a possession style of play - compare it to when he took the Flames to the Final, or even the first year he was with the Kings, they played a much different style than they do now. Maybe Doughty's right. Maybe he knows it's crap because they have something way better, something that's in Sutter's head - and there's no statistical model that is ever gonna approach the knowledge of a guy that's been in and around the game as long as Sutter. Those models would be for everyone else, who haven't been around that long and don't have that knowledge, or to approximate that knowledge for all the games you just aren't able to watch because you had a game that night or because you want to do a quick overview of some euro leagues you don't get to see or something. That's where the value really lies, to supplement the traditional ways of doing things, not to replace them.
     
  4. Smokey McCanucks

    Smokey McCanucks Registered User

    Joined:
    Dec 21, 2010
    Messages:
    2,693
    Likes Received:
    0
    Trophy Points:
    0
    I think the biggest problem I have with a lot of these statistical models that are being thrown around now - there's too little attempt to account for the roles different players play and how they are used. Things like qualcomp and zone starts are just scratching the surface and it's almost lip-service, it doesn't go nearly as far in depth as it needs to to provide an accurate measure and give context to some of the other stats. Without that context they can end up being very misleading.

    There are a TON of very solid defensive players with very lousy possession stats because they get put out on the ice against opponents' best possession guys and most dangerous scorers. And their role is not to regain and then retain possession; it's to limit the damage those top guys can do and hem them in and then get off the ice.

    It's no different than saying (bear with me), why is the checking line's +/- so lousy, they are supposed to be the checking line, right? Well, they are out there against the biggest scoring threats, of course it is going to look lousy. The only way to know whether it is REALLY lousy, though, is to have some theoretical baseline to compare it to, to estimate what the typical line would do against that level of competition, and that's what you base your assessment off of. You can't just say, well Scuderi or Bolland or Callahan, this guy's supposed to be a good defensive guy but his numbers really suck, maybe we were wrong in that assessment. No, the numbers probably suck for that sort of player because the coach knows they are the best defensive guys, the best guys at playing without the puck, so they go out there when the coach figures the other team's top guys are gonna have the puck and he tries to stem the tide, and it's because he's used that way and that's his role that his numbers look bad next to other guys who have other roles that end up giving them pretty numbers.
     
  5. CanadianSharks

    CanadianSharks Registered User

    Joined:
    Dec 18, 2012
    Messages:
    30,025
    Likes Received:
    1,891
    Trophy Points:
    156
    Pretty sure I've heard that one before. I think economics profs like that one. :laugh:

    I myself of 5 years ago saw myself laughing at that now... :facepalm:
     
  6. CanadianSharks

    CanadianSharks Registered User

    Joined:
    Dec 18, 2012
    Messages:
    30,025
    Likes Received:
    1,891
    Trophy Points:
    156
    I'm of the belief that until an effective metric is devised to rate players, different sorts of "tests" should be run with later round picks. For example, the Habs or Sharks hire more analytics staff with the sole intention of devising a sort of combinational algorithm to find players that typically produce in all stats with a positive correlation to things like goals, assists, etc. Obviously I just summed up what everyone is already trying to do, but I think a team ought to really focus on doing that with 5th, 6th and 7th rounders rather than purely relying on traditional scouting methods (if they aren't already). Even a rebuilding team just stock up on picks in a poor draft year and rely purely on stats for some very late picks.
     
  7. Bomber0104

    Bomber0104 Registered User

    Joined:
    Apr 8, 2007
    Messages:
    11,163
    Likes Received:
    1,358
    Trophy Points:
    169
    Advanced stats should be able to tell me which player is strongest on the puck, and which isn't.

    Advanced stats should be able to tell me which player is the best defensively / playing away from the puck

    Advanced stats should be able to tell me which player is the best offensively / playing with the puck

    Advanced stats is incapable of any of these.

    Advanced stats are not advanced at all.

    They are gimicky permutations of equally useless stats.
     
  8. jwhouk

    jwhouk Former Cheesehead, Always a Preds Fan

    Joined:
    Apr 19, 2004
    Messages:
    5,212
    Likes Received:
    35
    Trophy Points:
    146
    Gender:
    Male
    Occupation:
    Semi-Retired.
    Location:
    Valley of the Sun
    Home Page:
    The purpose of advanced statistics is (or at least should be) to paint a picture of what makes for a successful/not successful hockey team.

    I would argue that statistics that have little correlation with winning and losing have little use for fans, teams, media, or anyone else with a casual understanding of sport.

    Fine, you can tell me a guy's Corsi numbers are above average. What does that mean? Does it have anything to do with scoring goals or wins and losses? Does his "shot differential" mean that he's a floater on a good team, or that he's trying to make plays while on a line with Moe Howard and Doug MacKenzie?

    Show me how it affects winning, and I'll pay attention.
     
  9. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,648
    Likes Received:
    2,677
    Trophy Points:
    156
    Location:
    Windsor
    Awards:
    I moved these two most recent posts from a thread where they were off-topic, to here.

    Enjoy.
     
  10. Ogopogo*

    Ogopogo* Guest

    The real con is convincing so many naïve people that Corsi has some kind of value.
     
  11. Doctor No

    Doctor No Registered User

    Joined:
    Oct 26, 2005
    Messages:
    8,098
    Likes Received:
    1,636
    Trophy Points:
    149
    Home Page:
    Corsi does have "some kind" of value.

    If you're disappointed by it, then perhaps you placed too much stock in it to begin with.

    But then, you're attempting to derail the thread. Please stick to the topic.
     
  12. Rebuilt

    Rebuilt Registered User

    Joined:
    Jun 8, 2014
    Messages:
    8,736
    Likes Received:
    9
    Trophy Points:
    39
    Location:
    Tampa
    Everything and anything has 'value' if you believe it. I dont think the advanced stats have been able to predict any future events, except the roughly the same corsi stats.
     
  13. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,648
    Likes Received:
    2,677
    Trophy Points:
    156
    Location:
    Windsor
    Awards:
    Moved this diversion to a somewhat-appropriate location.
     
  14. theButton

    theButton Oooohh! Ahhhhhh!!

    Joined:
    Feb 18, 2013
    Messages:
    218
    Likes Received:
    0
    Trophy Points:
    0
    Certain aspects of hockey can be accurately measured. Hockey, no.
     
  15. Doctor No

    Doctor No Registered User

    Joined:
    Oct 26, 2005
    Messages:
    8,098
    Likes Received:
    1,636
    Trophy Points:
    149
    Home Page:
    The nicest thing about never being able to achieve perfection is that we can always continue to make progress towards that level. Indefinitely.

    Because hockey will never be able to be perfectly measured, we can always get better at measuring hockey. As a scientist, that's awesome. As someone who loves learning, that's awesome.
     
  16. silverfish

    silverfish wrong as usual

    Joined:
    Jun 24, 2008
    Messages:
    34,643
    Likes Received:
    4,314
    Trophy Points:
    186
    Location:
    under the bridge
    Corsi is not perfect, but it's on to something...

    [​IMG]
     
  17. mrkolice

    mrkolice Registered User

    Joined:
    May 26, 2014
    Messages:
    1,246
    Likes Received:
    173
    Trophy Points:
    81
    Occupation:
    Ho Ho
    Location:
    Ljubljana, Slovenia
    that's true, hockey can't be measured directly, almost never. but even rough indirectly-measured estimates about the mechanics of the game can be very useful. we can discuss the reliability or validity of certain parameters, and rightly so, but the bottom-line question is "are these metrics useful".

    i'm fascinated about the big advance in hockey stats in recent decade or so. i'm also fascinated about the epistemological side of the questions being asked. but remember, if it's useful - don't throw it away, rather develop it further.
     
  18. Voight

    Voight #winning

    Joined:
    Feb 8, 2012
    Messages:
    23,550
    Likes Received:
    2,857
    Trophy Points:
    157
    Location:
    MACV-SOG
    I wonder if anyone would ever come up with a hockey version of WAR.
     
  19. Noldo

    Noldo Registered User

    Joined:
    May 28, 2007
    Messages:
    1,481
    Likes Received:
    69
    Trophy Points:
    101
    With Coyotes choice of GM, the issue got even more prominent.

    Chayka's take here is interesting: calculate the data that shows what experienced hockey minds see, presumably because then you can teach a nobody to track the same thing from video and multiple your effectiveness.

    http://www.sportsnet.ca/hockey/nhl/coyotes-john-chayka-nhls-answer-mark-zuckerberg/
     
  20. 93LEAFS

    93LEAFS Registered User

    Joined:
    Nov 7, 2009
    Messages:
    25,372
    Likes Received:
    8,976
    Trophy Points:
    167
    Gender:
    Male
    Location:
    Toronto
    GVT was a slight attempt at it, and I think Puck shares exist, which is another attempt at it. I don't think a new-age stat will ever gain prominence like WAR has in baseball, because most puck-nerds will acknowledge that luck plays a bigger part of hockey. Another thing is, Baseball is already very statistic based, with every thing being very measurable (only Defence took awhile to measure, and it still difficult), whereas Hockey has a smaller season and has less quantifiable factors that measure success. Even Football, which isn't as fluid as a game as hockey as only recently seen an explosion and in advanced analytics, and my favourite team becoming a guinea pig for it (Browns, and I'm willing to try anything at this point to get a competent QB, whether it be guys with pocket protectors or voodoo priestesses).
     
  21. 93LEAFS

    93LEAFS Registered User

    Joined:
    Nov 7, 2009
    Messages:
    25,372
    Likes Received:
    8,976
    Trophy Points:
    167
    Gender:
    Male
    Location:
    Toronto
    I'm interested to see how Chayka does, while slightly envious considering he's a 2 years younger than me and was on the UWO campus as me (Not Ivy though). But I can think of another Ivy grad who did great things for a sports team (Beeston).
     
  22. trentmccleary

    trentmccleary Registered User

    Joined:
    Mar 2, 2002
    Messages:
    21,471
    Likes Received:
    206
    Trophy Points:
    186
    Location:
    Alfie-Ville
    Home Page:
    Is anybody else concerned that the prominence of Corsi is making it less reliable?

    Like new coaches are are asking players to force more shots from non-scoring areas or UFA's are doing the same to inflate their value? ... Basically creating fraudulent improvements. The Leafs traded down a lot in the past 12 months, but improved their Corsi by 300 (+200sf, -100sa) because they went from an old school coach to a possession coach. This would seem like an example of a fraudulent improvement. Even the league could be showing indications that this is the case; SV%'s increasing while goals remain stagnant or decline (stupid shot inflation?). Perhaps Corsi's downfall will be that it will become difficult to determine the talent from the frauds. Then once Corsi is declines in stature, it will no longer have such a grip on strategy and the measure will be more meaningful.

    Or maybe I'm imagining all of this.
     
  23. Doctor No

    Doctor No Registered User

    Joined:
    Oct 26, 2005
    Messages:
    8,098
    Likes Received:
    1,636
    Trophy Points:
    149
    Home Page:
    I don't know if "concern" is the right word, but I'm aware of it.

    With any metric that purports to track well with success, until the claim is made you have an advantage in that people aren't incentivized to game it.

    It's likely that Corsi tracks well with success because teams that do well in Corsi are also doing other things "right", and Corsi is either a proximate measure, or related to the true effect.

    Once teams hear about the correlation, and try to specifically take advantage of it, then they may be doing it at the expense of the thing that's actually valuable.

    There's also the likelihood at this point that confirmation and narrative biases sneak into the equation.
     
  24. Random Forest

    Random Forest Registered User

    Joined:
    May 12, 2010
    Messages:
    14,199
    Likes Received:
    525
    Trophy Points:
    156
    Let's also not lose sight of the fact that corsi is still a relatively primitive tool that's been around for almost ten years now. I'd be downright shocked if NHL teams didn't have better, proprietary data and measures that they are more concerned with targeting. I highly doubt NHL GMs and coaches are investing all their resources to simply maximize corsi alone.

    The correlation/causation phenomenon referenced above, I would assume, is something that analytically minded managers are keenly aware of (well, they should be keenly aware of). Of course, some managers are not all that bright, so maybe that's not a great assumption. :laugh:
     
  25. Teemu

    Teemu Trick question?

    Joined:
    Dec 3, 2002
    Messages:
    26,673
    Likes Received:
    1,797
    Trophy Points:
    260
    Goodhardt's law: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

    That said, there are many more ways at looking at Corsi than simply team CF.
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
monitoring_string = "358c248ada348a047a4b9bb27a146148"