Thanks for the link. It's an interesting article, and I agree with a lot of it. "Analytics" and "advanced stats" are used way too much in the hockey domain, with imprecise meanings. Most of the stats that are used aren't "advanced" at all, and as the article points out, analysis is a tool to make better decisions, not an end product that has much value.
"Decision science" is a better construct for why one would conduct statistical analysis, and it begins with asking better questions. It also requires a healthy understanding of the limitations of statistics, potential biases, and a health appreciation for complexity. The real leap forward in "decision science" vis-a-vis hockey will be when there is iterative learning that involves forming hypotheses based on good analysis, testing those hypotheses with systems adjustments, roster compositions and combinations, etc. and analyzing the subsequent results through a continuous learning cycle. NHL hockey is a very long ways away from that, but then so is "hockey analytics" at this point.
Careful - it's a toolbox, not just a tool.
Of course, I do agree with a bunch of what you said. Analytics in its most basic form still has a ways to go to get to the level achieved in baseball, basketball, and now even football. I find it hard to believe that it is 'a very long ways away' though - it's more so the willingness to experiment and the actual understanding that is a very long ways away. But now we're seeing Carolina take another step forward. Pittsburgh worked with Sam Ventura as a consultant for a long time before finally hiring him on full-time as a Director. Washington has had Tim Barnes for quite some time and has partnered with several more analytics oriented companies, such as our very own Garret's.
In Feburary 2015, ESPN's Craig Custance wrote that Chicago Blackhawks were one of the top teams in utilizing analytics (though I'm skeptical on its lengths) and that the Bruins, Sabres, Blue Jackets, Oilers, Kings, Wild, Islanders, Blues, Lightning, and Leafs weren't that far behind (with the Jets on the cusp of "believers" and "one foot in"). I'd argue that the Leafs and Lightning have leapfrogged the Hawks and everyone else as the most well-known, while the Oilers did a 180 and Sabres are doing Sabres things - all those other teams have been pretty consistent strong contenders if not winners.
Time to move even further forward into what you're saying, though, along the lines of what petbugs is saying in truly defining the problem and what you are trying to achieve - the, you can finally
begin to determine the best tool for the options you want to evaluate.
On another note, reading more of the
McKinsey article, this Q&A stands out to me:
The Quarterly: What other changes jump out at you over past 10 to 15 years?
Jeff Luhnow: In 2003, there were maybe four to five clubs that had analytics-dedicated people on their payroll, and typically they were in an office down the hall working on recommendations to people who may or may not pay any attention. I think what’s changed today is that every general manager has some background or interest in analytics, and the typical size of the group in the front office is probably somewhere between 12 and 15 full-time people who all have advanced degrees, whether it’s computer science or physics or mathematics or some other discipline. Along with that, there are data departments in organizations. Most organizations now have database folks and data scientists that are on their payroll and that are helping them not only store the information and organize it properly but also evaluate what it means.
There was also a trend in the past of using external companies to house data, like scouting reports or statistics. Most of that has now come in-house. When I was with the Cardinals, we used an outside provider, and when I got to the Astros, they were using an outside provider, but the response time and the customization was lacking. Most important, when you come up with a way of looking at the world and you want the external provider to build the model for you, you don’t want them to share it with the other 29 clubs. It’s difficult to have the confidence that it’s not going to be shared in some way, shape, or form. I think that’s led to most clubs believing that their way of handling data and information is a competitive advantage. It therefore becomes critical to have control over that in-house.
I'll beat the drum again. It's not good enough to have people on staff that can feed the decision-makers reports that may or may not be used. There needs to be an organizational buy-in and a good (not a general, a
good) understanding of the material, and a proper method of actual evaluation that isn't just hopping on and off a hot seat. I don't see why the Winnipeg Jets can't be at the forefront of this.
