Sloan Sports Analytics Conference

[LadyStanley]

KKurzCSN 9:24pm via TweetDeck Interesting piece on hockey analytics here from the Boston Globe. Many followers know I’d side with Burke on this one bostonglobe.com/sports/2014/03…


KKurzCSN 9:28pm via TweetDeck
“Let’s stop being obsessed with trying to capture the entire game and come up with some magic formula to capture the game. B/c we can’t."


fearthefin 9:31pm via Web
@KKurzCSN I agree with that but also rest of the quote: "Let’s learn about discrete areas and focus on diff. areas of the game and nail it."


fearthefin 9:33pm via Web
@KKurzCSN But honestly I couldn't agree more w/ sentiment that trying to find one catch-all statistic in hockey is a fool's errand.

Click to read more...

 

[kmad]

Agree with Burke, stats can't possibly paint the whole picture.

 

[wgknestrick]

Earth is FLAT I tell you

 

[Bear of Bad News]

I think that this is the part to focus on: "Let’s learn about discrete areas and focus on diff. areas of the game and nail it."

 

[Bear of Bad News]

Here's a full-length Boston Globe article on Burke's panel:

http://www.bostonglobe.com/sports/2...t-analytics/30d6XZ9OY7AWUXQtbBXpNN/story.html

This story is awesome (standard caveat: note my avatar):

Burke told one of his favorite yarns regarding former Canuck Trevor Linden. When Burke was in Vancouver, Linden was scheduled to attend the club’s pre-draft testing in 1988. Linden, who was raised in Medicine Hat, Alberta, called Burke to say he couldn’t make it. Linden had to help his uncle conduct Branding Day, where they turn male cattle into steers.

“What’s your job?” Burke asked.

“When they come in the pen, I grab them by the neck and haul them down,” Linden said. “We brand them and cut their [testicles] off.”

“You can skip our test, kid,” Burke replied. The Canucks drafted Linden second overall. He played 1,382 games in the NHL.

 

[Ishdul]

Yeah, I agree with him about the catch-all stat. It's been a big point of issue in baseball that everybody wants to boil everything down to WAR, it doesn't work there and works much less in hockey. But then, I don't think there have been as many attempts to do so.

 

[93LEAFS]

I agree with Burke that one catch all statistic isn't gonna work, but I find him in regards to stats to be purposely antagonistic. To be honest i don't know if its even used in hockey, but i have much more interest in how the Sports VU camera's can be used to quantify what we already see on the ice.

 

[LadyStanley]

http://www.tsn.ca/nba/story/?id=445459
TSN's Cullen article (1 of __) from conference

He has another article coming that's hockey-centric.

 

[achtungbaby]

I agree with Burke that one catch all statistic isn't gonna work, but I find him in regards to stats to be purposely antagonistic. To be honest i don't know if its even used in hockey, but i have much more interest in how the Sports VU camera's can be used to quantify what we already see on the ice.

The thing is, who ever said there would be one catch all stat for hockey? Burke seems to be worried about a problem that doesn't exist.

 

[schuckers]

Just to be clear

1. Fishman is the one that said we need to stop looking for a catch-all statistic. Some would consider CorsiRel a catchall stat.

2. Burke (in public anyway) finds no value in hockey analytics of any flavor on any aspect of the game.

 

[WhereIsIt]

The thing is, who ever said there would be one catch all stat for hockey? Burke seems to be worried about a problem that doesn't exist.

A lot of people get really worked up about possession stats, you have to wonder if they think that's the answer.

 

[achtungbaby]

A lot of people get really worked up about possession stats, you have to wonder if they think that's the answer.

Fair enough. I've found that most people who don't want to take 5 minutes to look at them are the ones who get really worked up about possession stats like Corsi. What's weird for me is that A: Burke keeps getting invited to these things and B: He keeps going to them when he clearly doesn't think they're worth his time. Is it like Pascal's Wager for him and he's just hedging his bets? Is he that much of a atttention ***** that he'll literally go to anything as long as there's a camera or a audio recorder? So many questions.

 

[AlienWorkShop]

I agree with Burke that one catch all statistic isn't gonna work, but I find him in regards to stats to be purposely antagonistic. To be honest i don't know if its even used in hockey, but i have much more interest in how the Sports VU camera's can be used to quantify what we already see on the ice.

It's partly Burke's style, but I don't think he's nearly as "anti-stats" as many articles like to paint him. It's very easy to make Burke out to be the out-of-touch, old school dinosaur, but it's apparent to me at least that he has taken a look at these new stats. He's just not convinced by them yet and I doubt you'll find many (reasonable) statheads who are entirely convinced by them yet either.

 

[Bear of Bad News]

Scott Cullen article on the SSAC:
http://www.tsn.ca/nhl/story/?id=445910

Overall, it's a nice anti-Burke piece. Interesting quotes:

"Part of what made the Basketball and Baseball Analytics panels at Sloan successful is that they didn't waste time discussing whether there is value in analytics."

"The funny thing about using analytics is that, somehow -- likely through the famous scene with the scouts around the table in the movie Moneyball -- there persists the idea that anyone wants to use analytics without actually watching players play."

"It's one thing to tell a colourful story about Trevor Linden that shows his character and another to base hockey decisions on that information. Basically, there are plenty of guys with high character that aren't necessarily great hockey players."

Anyhow, he also talks about the other highlights of the SSAC, and where things could be headed next. Good read.

 

[Epsilon]

Nate Silver called Burke out on this (and the lack of good statistical data for hockey in general) in his introductory "manifesto" today for the FiveThirtyEight relaunch:

http://fivethirtyeight.com/features/what-the-fox-knows/

Indeed, as more human behaviors are being measured, the line between the quantitative and the qualitative has blurred. I admire Brian Burke, who led the U.S. men’s hockey team on an Olympic run in 2010 and who has been an outspoken advocate for gay-rights causes in sports. But Burke said something on the hockey analytics panel at the MIT Sloan Sports Analytics Conference last month that I took issue with. He expressed concern that statistics couldn’t measure a hockey player’s perseverance. For instance, he asked, would one of his forwards retain control of the puck when Zdeno Chara, the Boston Bruins’ intimidating 6’9″ defenseman, was bearing down on him?

The thing is, this is something you could measure. You could watch video of all Bruins games and record how often different forwards kept control of the puck. Soon, the NHL may install motion-tracking cameras in its arenas, as other sports leagues have done, creating a record of each player’s x- and y-coordinates throughout the game and making this data collection process much easier.

I would ask a lot of questions of this data if I had it. For instance:

Is it smart for a player to keep control of the puck when Chara (or a similarly gifted defensemen) has him in his sights? Might the player yield fewer turnovers if he passed the puck instead?

Would measuring a player’s perseverance give us meaningful information beyond what is reflected in “box score” statistics, such as goals, assists and plus-minus?

Do players who persevere under threat match those who are regarded as “tough” or as having lot of “heart” by coaches, scouts and commentators? If not, is the metric flawed, or are the coaches biased?

The quality of hockey statistics is fairly poor compared to those for baseball or basketball, so I can understand Burke’s skepticism. But often, general managers and CEOs and op-ed columnists use the lack of data as an excuse to avoid having to examine their premises.

 

[Mathletic]

Nate Silver called Burke out on this (and the lack of good statistical data for hockey in general) in his introductory "manifesto" today for the FiveThirtyEight relaunch:

http://fivethirtyeight.com/features/what-the-fox-knows/

I was reading "A Conflict of Visions" by Thomas Sowell. It made me think of the debate on statistical analysis in hockey. At some point the author discusses conservative values. He frames it in terms of constrained and unconstrained vision. In his view, a conservative has a constrained vision, very aware of the limits of human nature, the limits of resources, the limits of reason, the limits of experts and so on.

The unconstrained view would be that of Robert Kennedy: There are those who look at things the way they are, and ask why... I dream of things that never were, and ask why not? That would be an unconstrained vision.

Don't want to turn this into a political debate but it made me giggle at times going through the book

... I agree with Silver on everything here btw

 

[soireeculturelle]

I was reading "A Conflict of Visions" by Thomas Sowell. It made me think of the debate on statistical analysis in hockey. At some point the author discusses conservative values. He frames it in terms of constrained and unconstrained vision. In his view, a conservative has a constrained vision, very aware of the limits of human nature, the limits of resources, the limits of reason, the limits of experts and so on.

The unconstrained view would be that of Robert Kennedy: There are those who look at things the way they are, and ask why... I dream of things that never were, and ask why not? That would be an unconstrained vision.

Don't want to turn this into a political debate but it made me giggle at times going through the book

... I agree with Silver on everything here btw

"The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man."

 

[The Mighty Duck Man]

Scott Cullen article on the SSAC:
http://www.tsn.ca/nhl/story/?id=445910

Overall, it's a nice anti-Burke piece. Interesting quotes:

"Part of what made the Basketball and Baseball Analytics panels at Sloan successful is that they didn't waste time discussing whether there is value in analytics."

"The funny thing about using analytics is that, somehow -- likely through the famous scene with the scouts around the table in the movie Moneyball -- there persists the idea that anyone wants to use analytics without actually watching players play."

"It's one thing to tell a colourful story about Trevor Linden that shows his character and another to base hockey decisions on that information. Basically, there are plenty of guys with high character that aren't necessarily great hockey players."

Anyhow, he also talks about the other highlights of the SSAC, and where things could be headed next. Good read.

I don't believe for a second they'd do it in hockey, but funny he should mention Moneyball, as in the book, Billy Beane contemplated just that. Obviously baseball's a lot different and a lot easier to analyze without watching, but he did think about firing his whole scouting staff and conducting an entire draft via a computer.


Anyways, good stuff. Really like that quote about exploring different areas of the game. I'm not sure how much statistical analysis will be able to track in the coming years, but it can't hurt accompanied with some outside-the-box thinking.

 

[wgknestrick]

I don't believe for a second they'd do it in hockey, but funny he should mention Moneyball, as in the book, Billy Beane contemplated just that. Obviously baseball's a lot different and a lot easier to analyze without watching, but he did think about firing his whole scouting staff and conducting an entire draft via a computer.


Anyways, good stuff. Really like that quote about exploring different areas of the game. I'm not sure how much statistical analysis will be able to track in the coming years, but it can't hurt accompanied with some outside-the-box thinking.

IMO, I would take an intelligently designed, computer based drafting model over a team of scouts any day. (this model is not something easily built, however). It is all about your "scouting market". The PC is drafting out of 100% of available players, while your "butt in the seat" scouts are only drafting out of the players they see or hear about AND then for only the games they watch. Now how do you decide which scout's recommended players are better? Who is more convincing in their arguments? I bet it still comes back to stats (and analytics) to make that final decision between a couple of "scouted" players.

Why not make that final decision using the same metrics, but by having ALL of the available players in your final pool? The computer is "all knowing", all seeing and I would take more data over "less" when making a decision (every time). I don't know how any team is able to "scout" the players effectively considering you don't know who will be available when you draft anyway.

 

[The Mighty Duck Man]

IMO, I would take an intelligently designed, computer based drafting model over a team of scouts any day. (this model is not something easily built, however). It is all about your "scouting market". The PC is drafting out of 100% of available players, while your "butt in the seat" scouts are only drafting out of the players they see or hear about AND then for only the games they watch. Now how do you decide which scout's recommended players are better? Who is more convincing in their arguments? I bet it still comes back to stats (and analytics) to make that final decision between a couple of "scouted" players.

Why not make that final decision using the same metrics, but by having ALL of the available players in your final pool? The computer is "all knowing", all seeing and I would take more data over "less" when making a decision (every time). I don't know how any team is able to "scout" the players effectively considering you don't know who will be available when you draft anyway.

In baseball? Absolutely it can work if you have the right model, although you probably have to take a look at the guy, especially with pitchers. In hockey? Next to no chance. Scouting's heavily flawed, but from what I can tell, most readily available stats are taken with a large grain of salt. Way too many moving parts to get a grasp of a hockey player from a computer, especially when you're trying to project development.

 

[schuckers]

In baseball? Absolutely it can work if you have the right model, although you probably have to take a look at the guy, especially with pitchers. In hockey? Next to no chance. Scouting's heavily flawed, but from what I can tell, most readily available stats are taken with a large grain of salt. Way too many moving parts to get a grasp of a hockey player from a computer, especially when you're trying to project development.

The dichotomy here is false. There is no reason why stats has to be either/or. Good management combines and augment information from multiple sources (i.e from stats and from scouting). Done event moderately well, this will produce better outcomes.

WYSIATI.

 

[The Mighty Duck Man]

The dichotomy here is false. There is no reason why stats has to be either/or. Good management combines and augment information from multiple sources (i.e from stats and from scouting). Done event moderately well, this will produce better outcomes.

WYSIATI.

Oh, absolutely. We happened to be discussing the idea of conducting a draft solely using a computer, though. For the record, absolutely you should always use both when you can, although when scouting underage players, any kind of statistic is going to have trouble holding much weight, be they traditional or advanced.

Where baseball's a spot where you could draft a player without ever watching him and probably do alright, hockey's one where you could almost draft one without ever seeing his numbers and possibly come out okay. Production, possession, a lot of those can be heavily skewed in the junior ranks.

 

[Mathletic]

Oh, absolutely. We happened to be discussing the idea of conducting a draft solely using a computer, though. For the record, absolutely you should always use both when you can, although when scouting underage players, any kind of statistic is going to have trouble holding much weight, be they traditional or advanced.

Where baseball's a spot where you could draft a player without ever watching him and probably do alright, hockey's one where you could almost draft one without ever seeing his numbers and possibly come out okay. Production, possession, a lot of those can be heavily skewed in the junior ranks.

Been a couple years since I started working on this for the draft and just to give you an idea. Last year I made my list for the Q draft. Using the 12th overall pick (2/3 of the way down) I ended up with Nicolas Meloche (nominated for rookie of the year and top 3 among 16 year olds), Luc Deschênes whom was later traded for a 1st round pick, Campbell Pickard whom could have made almost any team not named the Drakkar, Shawn Ouellet-St-Amant and William Gignan whom both had very good rookie seasons. You'd be hard pressed to find teams doing so well even with earlier picks. Not to mention that I work with data given by the midget AAA leagues in Québec, Nova Scotia, New Brunswick and Newfoundland. Those data are even more limited than those published by the CHL.

I think a lot of people take for granted that you can't model the draft and prospects when they didn't even give it try. They dismiss the idea from the start. I don't think it's that complicated to be honest with you. Like anything else, is it perfect? No, but can it help? Pretty sure. Can it even do better as a stand-alone? Could be.

 

[sousuffer]

In baseball? Absolutely it can work if you have the right model, although you probably have to take a look at the guy, especially with pitchers. In hockey? Next to no chance. Scouting's heavily flawed, but from what I can tell, most readily available stats are taken with a large grain of salt. Way too many moving parts to get a grasp of a hockey player from a computer, especially when you're trying to project development.

Google "systems biology" and "bioinformatics". We can use statistics to model hundreds of thousands of molecules and other moving parts in cancer whose functions/roles are much more seemingly random, but modeling 10 players in hockey has too many moving parts? Just because most people don't have the desire (and in some cases, intellectual power) to learn the tools necessary to figure out how to do so doesn't mean that we should just assume that the more "human" approach is the correct one. Human nature is that one tends to dismiss approaches that one does not understand.

The "eye test" used to be the way to find molecules involved in disease. Over the course of 20+ years, very few were discovered (leading to even fewer treatments). Since statistical approaches started becoming more commonplace (about 15 years ago), hundreds of new molecules (and data types) were discovered. Granted, there are many false positives that don't tell us anything, but the true positives are the important discoveries. Just because stats will give us some wrong answers doesn't mean the right ones it does give us won't tell us exactly what we need (and in a much better way than the eye test).

 

[The Mighty Duck Man]

Been a couple years since I started working on this for the draft and just to give you an idea. Last year I made my list for the Q draft. Using the 12th overall pick (2/3 of the way down) I ended up with Nicolas Meloche (nominated for rookie of the year and top 3 among 16 year olds), Luc Deschênes whom was later traded for a 1st round pick, Campbell Pickard whom could have made almost any team not named the Drakkar, Shawn Ouellet-St-Amant and William Gignan whom both had very good rookie seasons. You'd be hard pressed to find teams doing so well even with earlier picks. Not to mention that I work with data given by the midget AAA leagues in Québec, Nova Scotia, New Brunswick and Newfoundland. Those data are even more limited than those published by the CHL.

I think a lot of people take for granted that you can't model the draft and prospects when they didn't even give it try. They dismiss the idea from the start. I don't think it's that complicated to be honest with you. Like anything else, is it perfect? No, but can it help? Pretty sure. Can it even do better as a stand-alone? Could be.

I never said anything that it doesn't help, just that it would be foolish to use on it's own. And really, even with any kind of advanced model, it would be. Again, it tells you what happened, but with little context. I've never said to dismiss the idea, just that the idea of relying solely on advanced statistics in hockey is completely foolish, just as relying solely on the eye test would be. Do I think, forced to choose, that one would be better going strictly with the eye test? Yeah, I do. Do I think stats are useless and don't help? Not one bit.

As far as your system, it does sound a bit impressive, but even with that model, you'd be foolish to ignore those players and just go with it. Also, I'll admit I know nothing about those players, but it seems like you have a lot of narrative there. The first guy sounds impressive, the other three sound like you're using your opinion and conjecture to make your point. Isn't this exactly the kind of thing the advanced stats community is against?

Google "systems biology" and "bioinformatics". We can use statistics to model hundreds of thousands of molecules and other moving parts in cancer whose functions/roles are much more seemingly random, but modeling 10 players in hockey has too many moving parts? Just because most people don't have the desire (and in some cases, intellectual power) to learn the tools necessary to figure out how to do so doesn't mean that we should just assume that the more "human" approach is the correct one. Human nature is that one tends to dismiss approaches that one does not understand.

The "eye test" used to be the way to find molecules involved in disease. Over the course of 20+ years, very few were discovered (leading to even fewer treatments). Since statistical approaches started becoming more commonplace (about 15 years ago), hundreds of new molecules (and data types) were discovered. Granted, there are many false positives that don't tell us anything, but the true positives are the important discoveries. Just because stats will give us some wrong answers doesn't mean the right ones it does give us won't tell us exactly what we need (and in a much better way than the eye test).

Ok, seriously, we're comparing it with cancer now? That's where it gets ridiculous. My main point was that statistics out there right now for hockey really don't tell you a lot, and there's no model out there that will give you context about what's going on. More yet, for the fifth time, I never said the eye test is the only way to go or we should never try to find these models, just that on their own, they wouldn't do much. Is that really a stretch?