News Article: "Fun With Numbers" - Advanced Stats Talk Here

[Cosmix]

I think there is huge room for growth. If you are into military history, an analogy I would use is tanks during WWI. When they first hit the battlefield they were pretty awesome but had limited impact when it came down to it. People with imaginations saw the potential, and really refined both the design and the strategies for use that would maximize their effectiveness. By the time WWII came around two decades later, tanks were instrumental in battles and played a huge role in changing the nature of warfare.

Yes, indeed, again!

 

[Quo]

Hmmm...a lot to consider here.

I am a fan of military history and the Blitzkrieg analogy I get.

These designated stats people are a very new wrinkle in NHL front offices. I didn't take away that Nill or Bowman had designated people or a department collating and analyzing stats. Bowman wants to keep the how's and wherefore's of who analyzes the data and what stats are collected close to the vest. It would be easy for a team like the Sens to have similar duties extended to people already on staff and keep it clandestine.

It is important to stay ahead of the curve for sure, but there's also some risk that that innovation could backfire. Maybe a risk of developing a number fetish? And these teams (Dallas, Chicago...etc), they seem to say that they are big on these stats and want to use them but that use is still judicious. Even Chicago and LA not exempt from the crudely titled "Dumb **** Report" that Yost quoted.

I'd say that the Sens being aware of advanced stats and how they can augment traditional evaluation is enough for now. Better something than nothing and maybe they feel that the truly innovative stuff will not stay a secret long enough to really turn the tables against them.

 

[HavlatMach9]

advanced stats are not interesting to me because they seems amateurish

A proper approach would require identifying aspects and parameters by a hockey mind, then scouting those things which would take a lot of manpower, and a math guy to get ranking results. and it would never see the light of day outside the team

you can't expect to mix stats and expect people to take them seriously if they don't know how the new stats came about

 

[StefanW]

Hmmm...a lot to consider here.

I am a fan of military history and the Blitzkrieg analogy I get.

These designated stats people are a very new wrinkle in NHL front offices. I didn't take away that Nill or Bowman had designated people or a department collating and analyzing stats. Bowman wants to keep the how's and wherefore's of who analyzes the data and what stats are collected close to the vest. It would be easy for a team like the Sens to have similar duties extended to people already on staff and keep it clandestine.

It is important to stay ahead of the curve for sure, but there's also some risk that that innovation could backfire. Maybe a risk of developing a number fetish? And these teams (Dallas, Chicago...etc), they seem to say that they are big on these stats and want to use them but that use is still judicious. Even Chicago and LA not exempt from the crudely titled "Dumb **** Report" that Yost quoted.

I'd say that the Sens being aware of advanced stats and how they can augment traditional evaluation is enough for now. Better something than nothing and maybe they feel that the truly innovative stuff will not stay a secret long enough to really turn the tables against them.

There is no real backfire in the innovation. The first big danger that teams face is underutilizing the numbers, and not milking them to their full potential. With apologies to the Sens management team, having guys with little to no statistical know-how go over a few basic numbers and maybe, on a good day, charts does not take things very far.

If I had to take out my crystal ball, I would say that two big shifts are coming. The first is with SportsVu technology, which will provide some precise information that is difficult to manually collect. The second big shift will happen when more than half the teams in the league set up analytics divisions. Right now, if a team can utilize the numbers well they have a decided advantage over their competition. Once every does this that advantage rapidly disappears, because any improvement in the league is relational. In other words, there are 16 playoff spots and one cup up for grabs every year. If 2 teams are doing analytics well they have a huge advantage. If 29 teams are doing them the advantage is gone, and all that is left is risk associated with falling behind. I wrote a bit about this effect here if you are interested.

I think the Sens still have a few years before they have to take analytics more seriously. We had some high profile analytics hires this summer, and everyone will be watching how they do. If teams listen to them (a huge if), and if there is some value added (a much smaller if) than more teams will catch on. But rosters and attitude don't change overnight.

 

[Quo]

Great stuff Stef. I enjoyed that article (looking better too). I think we agree on a lot.

I'll ask though...who in the Dallas front office has an extensive statistical background? On LA? Chicago? To reference the Bowman article again, it was him, with his business pedigree, that conceived of the Blackhawks whole setup. There's no doubt some overlap with business and stats but I don't know if you could call his background extensive.

Seems a lot of the top guys are right in there trying to figure things out and succeeding enough to find some value in them. They may be hockey guys but they can paste things together decently enough. Tim maybe did the same for us and maybe Dorion is doing it now.

That's not to say that having a dedicated R&D analytic department is a bad idea or that how Bowman, Nill and whoever else are doing it now will not someday be outmoded. Seems like quite a good idea actually. Something with the potential to be revolutionary, though potential yet to be seen. Hence some trepidation to really go whole hog on it just yet. Like you said, rosters and attitudes don't change overnight.

 

[Busboy]

I don't agree with your conclusion.

Analytics is not "meaningless" simply because you have "less" money to spend on UFAs. If you neglect analytics you will not have the same information that your competitors will have who do use analytics. Therefore you will be at a competitive disadvantage. Analytics are not used just for determining which UFAs to sign. They can assist a team in identifying which players to retain and which to let go, which players to draft and which to avoid, which players to play and which to sit on the bench in various situations.

If you speak for the Ottawa Senators, then I think this reflects badly on the Ottawa Senators.

I do not like what I hear from Randy Lee either. It makes Murray seem like a dinosaur as a manager.

What do we care what Murray seems like? His job is to put a winning team together, and he needs to do it on a limited budget.

If money wasn't a factor then we'd have a dedicated analytics team. Randy Lee stated that they see value in it and we know Murray is involved in the discussion in some manner.

I know it's in style for pro analytics crowd to put down any form of criticism but there are still significant limitations to the practice.

For a team with a limited budget, i
I think player development is a better way to spend our money for a number of reasons.

StefanW made the assertion that a budget team couldn't afford not to spend on analytics but I'm sure most would agree that you could make the same assertion about player development.

Murray's core strategy has been to draft and develop. That's where we should be spending our money. Our pro scouts and management are capable of evaluating players. It's not as though they are acting without the use of any statistics, they simply disagree to which degree it is appropriate to spend on advanced statistics analysis.

 

[StefanW]

Great stuff Stef. I enjoyed that article (looking better too). I think we agree on a lot.

I'll ask though...who in the Dallas front office has an extensive statistical background? On LA? Chicago? To reference the Bowman article again, it was him, with his business pedigree, that conceived of the Blackhawks whole setup. There's no doubt some overlap with business and stats but I don't know if you could call his background extensive.

Seems a lot of the top guys are right in there trying to figure things out and succeeding enough to find some value in them. They may be hockey guys but they can paste things together decently enough. Tim maybe did the same for us and maybe Dorion is doing it now.

That's not to say that having a dedicated R&D analytic department is a bad idea or that how Bowman, Nill and whoever else are doing it now will not someday be outmoded. Seems like quite a good idea actually. Something with the potential to be revolutionary, though potential yet to be seen. Hence some trepidation to really go whole hog on it just yet. Like you said, rosters and attitudes don't change overnight.

I absolutely agree with you. I don't want to come across as saying that individuals without a graduate degree in stats or a stats related field have nothing to contribute. Spending your life at the rink counts for a lot, and guys like TM and Dorian have a huge advantage in knowing what to look for.

The point I was trying to articulate is that having a limited stats background can possibly lead to roadblocks. If mapping out player trajectories involves doing a linear regression, and I have seen a few of those on blogs lately, it is really easy to miss checking whether assumptions are met. For example, I was able to run a regression in my honours year, but I did not get into variance inflation factors and things like that until grad school.

I guess my main point is that experience leads you to push analysis in one way, but having a solid stat foundation opens up doors to look at data in entire different ways. Ideally you will have both on a team instead of one or the other. Does this make sense?

 

[StefanW]

StefanW made the assertion that a budget team couldn't afford not to spend on analytics but I'm sure most would agree that you could make the same assertion about player development.

Why pick between development and analytics? You can hire 2 or 3 analysts at about 100k each max. The stats package can go for thousands of dollars, unless the use r or some other free software package. They can direct scouts to collect particular types of player data, which can be used during the draft.

To be 100% honest, if your analysts are not saving you at least 200-300k in bad signings or bad contract extensions per season they aren't doing their jobs well.

 

[Quo]

I absolutely agree with you. I don't want to come across as saying that individuals without a graduate degree in stats or a stats related field have nothing to contribute. Spending your life at the rink counts for a lot, and guys like TM and Dorian have a huge advantage in knowing what to look for.

The point I was trying to articulate is that having a limited stats background can possibly lead to roadblocks. If mapping out player trajectories involves doing a linear regression, and I have seen a few of those on blogs lately, it is really easy to miss checking whether assumptions are met. For example, I was able to run a regression in my honours year, but I did not get into variance inflation factors and things like that until grad school.

I guess my main point is that experience leads you to push analysis in one way, but having a solid stat foundation opens up doors to look at data in entire different ways. Ideally you will have both on a team instead of one or the other. Does this make sense?

Well, terms like "linear regression", "variance inflation" are a little over my head admittedly but yeah, I think I see what you're saying. Just looking for a way to milk that cow for all it's worth while the getting is good. Before that Dubas fellow and his cabal of mathematicians gets the leg up on us. All while not losing what makes traditional analysis so valuable.

The name of your site, "integrating hockey analysis", would seem to be the name of the game and it seems to be happening naturally as thoughtful people on both sides of the advanced stat divide gain some comfort with one another. Emphasis on naturally, because like any natural process there's going to be a lot of pain along the way and it'll take time to get right. Seems inevitable really, and makes the whole hubbub coming from the extremities of the debate look like so much pointless chest pounding.

 

[StefanW]

The name of your site, "integrating hockey analysis", would seem to be the name of the game and it seems to be happening naturally as thoughtful people on both sides of the advanced stat divide gain some comfort with one another. Emphasis on naturally, because like any natural process there's going to be a lot of pain along the way and it'll take time to get right. Seems inevitable really, and makes the whole hubbub coming from the extremities of the debate look like so much pointless chest pounding.

I've seen the same scenario play out in different fields. Before computers were around, a lot of social science analysis was qualitative. Once computers were their to help things along, stats started to take over. There are people who will always stay on the stats side, but a lot of people realize that qualitative and quantitative data both provide different but equally valuable insights. There has been a steady growth in mixed methods analysis, which tries to integrate qualitative and quantitative data, over the past couple of decades.

I just figured I would skip a couple of steps and head straight to where I know I want to be in terms of hockey analysis. If the development of hockey analytics plays out in the same was as the rise of quant in academia we are in for some nastiness as quant grows and tries to completely invalidate any type of analysis that is not rooted in easily quantifiable data. We already see rumblings of this with attacks on leadership, determination, grit, heart, etc, but for now it is more about carving out a place for analytics. Once that place is set things may become really ugly until the inevitable backlash starts.

 

[DrEasy]

I'm going to start with focusing more on standard deviations of the save percentages, which is not an entirely new idea. Putting in a goalie with a lower save percentage good be the correct call if the standard deviation is lower, because it would mean the goalie is more consistent. If you are overmatched and really need a win, putting in a goalie with a higher standard deviation may make sense because that is the type of goalie who could steal a game for you. I also plan to do other work with refining goalie stats that I wont get into yet because I'm not sure if it will work out.

This reminds me of a baseball discussion I had about starting pitchers. If you pick a starter with an average ERA with low variance, he is likely to end with a .500 W/L season, assuming your own offense is close to that average ERA. But you can probably pick a starter with a worse ERA but very high variance, and you're almost as likely to get the .500 W/L season, *even if your own offense is a little worse than average*. So if you have a bad team (where a .500 result is considered an improvement) and a low budget, go for that second pitcher, who will also probably cost you less.

 

[Busboy]

Why pick between development and analytics? You can hire 2 or 3 analysts at about 100k each max. The stats package can go for thousands of dollars, unless the use r or some other free software package. They can direct scouts to collect particular types of player data, which can be used during the draft.

To be 100% honest, if your analysts are not saving you at least 200-300k in bad signings or bad contract extensions per season they aren't doing their jobs well.

Could you explain to me how they are saving you money each season?

I'm going under the assumption that our hockey operations works with a strict budget just like our team. In a perfect world we would be spending an extra 100K on analytics, but for now I think we have to assume that would be money we are not spending elsewhere. We don't have the ability to spend as much off the ice as other teams, to me analytics seems like nice to have instead of must have, but I'd like to hear more from you on how exactly it would function for the organization and save them money.

 

[Micklebot]

Could you explain to me how they are saving you money each season?

I'm going under the assumption that our hockey operations works with a strict budget just like our team. In a perfect world we would be spending an extra 100K on analytics, but for now I think we have to assume that would be money we are not spending elsewhere. We don't have the ability to spend as much off the ice as other teams, to me analytics seems like nice to have instead of must have, but I'd like to hear more from you on how exactly it would function for the organization and save them money.

The point is if your Analytics guru isn't allowing you to get players via trade/UFA that provide better value per dollar, they aren't worth the 100K you're paying them, and nobody would bother having them.

The problem with this line of thinking is that you have to assume that analytics A) is more reliable than traditional pro-scouting as a rule, and B) that the analytics guru is providing sufficient extra value compared to the group based analytics that we currently run with, (no defined specialist, but some of the team having a background in it).

There's also the problem of insuring that when it comes to decision making time that management buys into what the analytics guru is selling (why hire him if you're not going to listen). Add to that, I think 100k is certainly less than what Dubas is getting paid, though you could probably find some stats guys who would work at that salary.

It's certainly easy to beleive that a analytics department would be cost neutral or better, simply do to the scale; even if you spent 1 mil, they'd have 55-70 mil worth of opportunity to improve on. Not only would they help scouts in evaluating player aquisitions, and perhaps streamline who to focus on, but they could also help management in identifying contract strategies (who is a good bet for a long term contract, identifying max threshold before we can find a less expensive replacement ect).

 

[Busboy]

The point is if your Analytics guru isn't allowing you to get players via trade/UFA that provide better value per dollar, they aren't worth the 100K you're paying them, and nobody would bother having them.

The problem with this line of thinking is that you have to assume that analytics A) is more reliable than traditional pro-scouting as a rule, and B) that the analytics guru is providing sufficient extra value compared to the group based analytics that we currently run with, (no defined specialist, but some of the team having a background in it).

There's also the problem of insuring that when it comes to decision making time that management buys into what the analytics guru is selling (why hire him if you're not going to listen). Add to that, I think 100k is certainly less than what Dubas is getting paid, though you could probably find some stats guys who would work at that salary.

It's certainly easy to beleive that a analytics department would be cost neutral or better, simply do to the scale; even if you spent 1 mil, they'd have 55-70 mil worth of opportunity to improve on. Not only would they help scouts in evaluating player aquisitions, and perhaps streamline who to focus on, but they could also help management in identifying contract strategies (who is a good bet for a long term contract, identifying max threshold before we can find a less expensive replacement ect).

This is pretty much the way I view it. We are already spending on pro scouts who ultimately will be more reliable than the advanced stats they don't have easy access to or integration with at this point.

But once we are using advanced stats, I think they have serious limitations for how they can be applied to actual contract negotiations and player acquisitions.

I think the advanced stats have more application for fans than management personally. We don't have highly qualified and experienced scouts evaluating talent for us or watching multiple games per night. We're also not able to talk to people in the know and find out information that might not be obvious.

At the end of the day isn't that what we want to use advanced stats for? To see underlying truths that aren't plainly visible and require further digging and added context?

There are ways of identifying players who are under the radar without the use of statistics. It is simply another way of doing what we already do, or an additional tool to be used.

We're not dinosaurs for not having a dedicated department at he moment. In fact, from what I've seen there have been many strategies employed by the Sens management that seem ahead of the curve.

I see many areas where we could also spend additionally and have obvious benefits.

The importance of good nutrition for recovery purposes and injury prevention? Hire a personal chef like the have in Vancouver.

Increased spending in strength and conditioning and rehabilitation? We lose less time and money to injuries.

Running a pro hockey team is complex, and compromises are a necessity when we realize that nobody has blanquette cheques to pass out. Right now management is compromising in the area of advanced stats, I trust that they're making the right decision and respect that they're taking a bit of a wait and see approach instead of jumping in to what's trendy or in vogue.

 

[Micklebot]

Some sneak peaks at graphs to come from Senstats.com

https://twitter.com/senstats/status/506612709732139008/photo/1

Looks like there will be some interesting content to be discused there.

Taking a look at his site, he already has the data posted in downloadable spreadsheets. The first thing that caught my attention is that we apparently as a team do a far better job than our opponenets at entering the zone with control, 2850 zone entries with control vs 2474 for the opponents with control, (48% of our entries are with control vs 43% of our opponents entries).

This seems counterintuitive for a poor defensive team (more specifically that the opposition's entries with control are so low), thought it's hard to judge without data for other teams when not facing us. It seems at least to me, that the opposition is employing a dump and chase strategy against us, which seems logical as it would put additional pressure on our very young D.

 

[StefanW]

Another new advanced stat site has come into being. Some interesting fancy stats that are outside of what was available up until now.

http://progressivehockey.blogspot.ca/

And within minutes, another one that also has cool graphing capability.
http://war-on-ice.com/

 

[Micklebot]

Interesting read over at Senstats; http://senstats.com/a-season-review/

Very data heavy, and most of it not the variety you see plastered across every other blog so it's a bit of a fresh look at things sens from an Analytics perspective.

It was predictably not kind to the GSN line, Kassian, or Spezza's defensive work.

 

[DrEasy]

Interesting read over at Senstats; http://senstats.com/a-season-review/

Very data heavy, and most of it not the variety you see plastered across every other blog so it's a bit of a fresh look at things sens from an Analytics perspective.

It was predictably not kind to the GSN line, Kassian, or Spezza's defensive work.

Isn't that a reason to worry about the Sens' attention to the stats (or lack thereof) given that the GSN line still got a lot of play and that Kassian remained on the roster till the bitter end?

 

[Busboy]

Isn't that a reason to worry about the Sens' attention to the stats (or lack thereof) given that the GSN line still got a lot of play and that Kassian remained on the roster till the bitter end?

Let the coach do his thing. There's a reason why he was awarded the Jack Adams.

I also don't see how any stats are relevant for Kassian other than PIMs and wins vs. losses.

 

[DrEasy]

Let the coach do his thing. There's a reason why he was awarded the Jack Adams.

I also don't see how any stats are relevant for Kassian other than PIMs and wins vs. losses.

The Jack Adams itself is almost reason to worry, as it could just be dinosaurs rewarding other dinosaurs. Jeter has like 10 Gold Gloves after all.

 

[Busboy]

The Jack Adams itself is almost reason to worry, as it could just be dinosaurs rewarding other dinosaurs. Jeter has like 10 Gold Gloves after all.

I don't follow baseball so I don't get your reference.

Are you saying Paul Maclean is a dinosaur because he's not using advanced stats to determine the Sens lineup? And that he didn't deserve the award based on advanced stats?

 

[Alfieghetti]

I think he's saying that the people that give out the awards are cave bros

 

[Benttheknee]

Not sure if this has been mentioned but the only way to truly advance the stats is to identify what you want and then find a way to collect the data that can give you what you want in every rink. If we only have the data we have now, then the stats revolution is over and we have what we have.

 

[Micklebot]

Isn't that a reason to worry about the Sens' attention to the stats (or lack thereof) given that the GSN line still got a lot of play and that Kassian remained on the roster till the bitter end?

Kassian played 5 mins a night, and his starts became less frequent as the season went on. His use had very little to do with whether or not Ottawa was paying attention to the stats. He was there for intimidation, and his mins were managed knowing full well his limitations (ie not being all that good at the sport of hockey, relative to his peers that is).

Truth is, Kassian and GSN got the mins they did in part because we lacked depth. We didn't have the players to put together a 4th line the coach could trust, and guys like DaCosta, Conacher, and Pageau we not getting it done.

Oddly enough, in the last 20 games or so, the GSN's numbers improved a fair bit. I think that may be in part due to increased depth with Hoffman, Stone, and Hemsky replacing Conacher, Injured Ryan, and what ever spare part was in on a given night (Kassian, Condra, DaCosta, Pageau). While the article dismisses the relevance of deployment and QOC, I think in smaller samples it can have a greater effect, as you will see more variance in the QOC where as long term things even out more.

It's also worth mentioning that Neil only averaged 11:48 a night, and Greening a touch more, so clearly the coach wasn't relying too heavily on that line. Smith is more well like by the coaching staff, and his underlying numbers were also much more favourable.

 

[Micklebot]

Not sure if this has been mentioned but the only way to truly advance the stats is to identify what you want and then find a way to collect the data that can give you what you want in every rink. If we only have the data we have now, then the stats revolution is over and we have what we have.

That's what I find interesting about the senstats site. He manually tracked zone entries and exits for all last years sens games; it's labour intensive, but unique from a lot of the other stuff out there and too me is a lot more informative than just Corsi for example.