All Purpose Analytics and Extended Stats Discussion

[NobodyBeatsTheWiz]

You might want to read that a bit closer.



He's only missing 21 games, as in he has a 98.3% sample.

Fair enough, I was using the 24 Leafs games he kept mentioning.

Less than a full season still isn't statisitcally significant to develop any kind of metric or predictor to be relevant beyond that season.

 

[Hivemind]

The 24 Leafs games isn't the Ferrari article, that's the Pension Plan Puppets article.

Really though, this is an issue that SportVU will help solve.

 

[g00n]

Zone Time and Corsi/Fenwick correlate very highly.


http://vhockey.blogspot.com/2008/08/zone-time.html


http://www.pensionplanpuppets.com/2013/9/16/4727746/leafs-attack-time-at-the-halfway-mark

Highly but not entirely. You generally shoot when you're in the zone so shots will mostly follow the zone time stat per common sense.

But that's doesn't mean you can substitute something that correlates in the 80-90% range for individual possession or "driving play", as measured by all shots or attempts of any quality split down to a fraction of a percentage point per 60 minutes or whatever. There is a large enough gap there that we can't say corsi or fenwick equal possession in a universally comparable context, especially for individuals.

Meaning, getting nuts over a 49% corsi vs a 53% when comparing players makes no sense.

I know you say nobody does that, but they do.

 

[ChibiPooky]

Some interesting work is being done around the concept of "shots created" - essentially player's shot attempts + shot attempts on which player made the primary pass. I think that's more valuable in terms of comparing individuals, but there is a ton of work that needs to be done with it before I'd cite it as meaningful.

 

[SimplySensational]

The Washington Capitals announced Saturday that the team has hired Tim Barnes as an analytics consultant.

Barnes' claim to fame is actually under an online pseudonym: Vic Ferrari.
As author of the web site Irreverent Oiler Fans, Barnes established a reputation as one of the fore-runners hockey analytics.

Between 2006 and 2008, "Ferrari" helped establish some of hockey's most important advanced statistical metrics including Corsi, Fenwick and PDO.

http://www.tsn.ca/capitals-hire-barnes-aka-vic-ferrari-as-analytics-consultant-1.104616

Irrelevant. Small sample size.

 

[Marshall]

Fancy-stat-ers rejoice.

 

[Hivemind]

I have been irritated and confused by vhockey being private lately. Not it makes sense. I literally let out a cheer when I heard the news.

 

[ChibiPooky]

I'm roughly 2.5x as pleased as punch. Any additional information they can bring to the table is positive. Better yet, Barnes is a guy who pushes the envelope to see what else is useful. No near-term impact, but that's a good hire for the future. I was worried that they were falling behind the curve there for a while.

 

[Hivemind]

I'm roughly 2.5x as pleased as punch. Any additional information they can bring to the table is positive. Better yet, Barnes is a guy who pushes the envelope to see what else is useful. No near-term impact, but that's a good hire for the future. I was worried that they were falling behind the curve there for a while.

It's amazing what Barnes was able to do when he was doing it just as a hobby. He's been pretty quiet the last couple years, but hopefully he can continue expanding hockey knowledge now that it's, presumably, his full time job.

 

[Burakovsky95*]

Absolutely love the hire. He's basically the father of these stats.

 

[NeilYoung]

Heres a good write up http://www.cbc.ca/sports-content/hockey/opinion/2014/08/in-search-of-vic-ferrari.html

 

[SimplySensational]

I'm roughly 2.5x as pleased as punch. Any additional information they can bring to the table is positive. Better yet, Barnes is a guy who pushes the envelope to see what else is useful. No near-term impact, but that's a good hire for the future. I was worried that they were falling behind the curve there for a while.

If he's showing unflattering stats to the coaching staff, it will have an immediate impact.

Specifically will love to see how winning faceoffs means a damn thing if Beagle is the worst possession player in the lineup.

 

[Langway]

*He is a huge believer in NHL coaches. "They have a hell of a grasp on who is bringing what to their team." He admits some have "blind spots" when it comes to favourites or least favourite players, but "most are smart guys who spend all day working on it."

*"The next big thing (in hockey analytics) will be to define the context of ice time ... quality of competition. Who you play against matters a ton." Barnes said where current hockey executives need help is how to use this to determine salary value, but those execs are far better at "weighing" players than those who rely solely on analytics. "If a 'hockey guy' comes out and says, 'That player was always [hidden],' they tend to be right. If you were to give me five hockey analytics versus five GMs or vice-presidents of hockey operations or five coaches -- five coaches especially -- to weigh a player, I'd take [the latters'] opinion ... A lot of analytics would have very young teams that won't win."

A pretty sophisticated outlook. Strong hire and a good fit under the circumstances.

I still think analytics is most useful when it comes to roster building and finding UFA bargains but other applications certainly can be helpful. They'll have some tough UFA decisions to navigate next summer, particularly if they want and manage to keep Green around.

 

[Hivemind]

A pretty sophisticated outlook. Strong hire and a good fit under the circumstances.

I still think analytics is most useful when it comes to roster building and finding UFA bargains but other applications certainly can be helpful. They'll have some tough UFA decisions to navigate next summer, particularly if they want and manage to keep Green around.

I think it can be really useful for providing coaches with an additional perspective, as well. There was a really good article by Tyler Dellow last year regarding Edmonton's success after offensive zone faceoffs (or the lack thereof), and how it all changed after a specific date (which presumably coincided with a coaching adjustment). Unfortunately it's gone along with the rest of Dellow's stuff. You also have guys like Quennville who buy in when constructing lines and deploying players.

 

[g00n]

A pretty sophisticated outlook. Strong hire and a good fit under the circumstances.

I still think analytics is most useful when it comes to roster building and finding UFA bargains but other applications certainly can be helpful. They'll have some tough UFA decisions to navigate next summer, particularly if they want and manage to keep Green around.

Right. So the guy who came up with these stats is saying the "eye test" is extremely important and analytics by themselves can be misleading without that context. Imagine that.

Think of it this way...an average hypothetical defenseman playing 20 minutes a night could be on the ice for 1/3 of the shots for and against, in theory. Off the top of my head and for simplicity's sake, NHL averages are somewhere around 30 shots a game with shooting percentage being around 10%, maybe less. So that hypothetical defenseman might be on the ice for 10 shots for, 10 shots against, and one goal for each team. Again, purely vanilla and hypothetical.

Let's say that d-man is on the ice for 1 more shot for and one less shot against, an 11/9 split. I believe the CF% would then be 55%, which by fancy stat standards is very good, yes?

To me what this means is, with one shot gained per game and one shot against removed, over the course of a season that player is on the ice for about 8 more goals for, and 8 fewer goals against (1 goal every 10 shots, 10 shots every 10 games, about 80 games played). Not necessarily contributing to those situations, just on the ice for them.

So my question is, does that player become "elite" simply based on that 5% distinction over the average hypothetical guy who breaks even statistically? Because of a one shot swing per night?

To me that's where the "eye test" comes in. If you can see over the course of those 80+ games that Mr. 55% is contributing to the possession and the scoring, and isn't a passenger, then the stat confirms your impression. If however you know from experience that that player has benefitted from the work of others or some cupcake matchups, then that 55% in a vacuum means much less. You can try to tease some of that out in QOC and other relative measurements, but as even the founder of the stat says those areas are vastly underdeveloped and you need the "eye test" to give those stats their final meaning.

So, yeah. Welcome stat dude. Tell us something good. But let's hope BT and his experienced eye has the final say.

 

[SimplySensational]

He talked about you know hockey people, not you.

 

[Dream Big]

I've been putting off becoming expert at analytics but I guess it's time has come. I still like the eye test. Vic aka Tim saying the eye test is still valid makes me happy.

Sense of humour and brains are two characteristics that are high on my like list.

Yee Hah.

 

[MW6]

Right. So the guy who came up with these stats is saying the "eye test" is extremely important and analytics by themselves can be misleading without that context. Imagine that.

Think of it this way...an average hypothetical defenseman playing 20 minutes a night could be on the ice for 1/3 of the shots for and against, in theory. Off the top of my head and for simplicity's sake, NHL averages are somewhere around 30 shots a game with shooting percentage being around 10%, maybe less. So that hypothetical defenseman might be on the ice for 10 shots for, 10 shots against, and one goal for each team. Again, purely vanilla and hypothetical.

Let's say that d-man is on the ice for 1 more shot for and one less shot against, an 11/9 split. I believe the CF% would then be 55%, which by fancy stat standards is very good, yes?

To me what this means is, with one shot gained per game and one shot against removed, over the course of a season that player is on the ice for about 8 more goals for, and 8 fewer goals against (1 goal every 10 shots, 10 shots every 10 games, about 80 games played). Not necessarily contributing to those situations, just on the ice for them.

So my question is, does that player become "elite" simply based on that 5% distinction over the average hypothetical guy who breaks even statistically? Because of a one shot swing per night?

To me that's where the "eye test" comes in. If you can see over the course of those 80+ games that Mr. 55% is contributing to the possession and the scoring, and isn't a passenger, then the stat confirms your impression. If however you know from experience that that player has benefitted from the work of others or some cupcake matchups, then that 55% in a vacuum means much less. You can try to tease some of that out in QOC and other relative measurements, but as even the founder of the stat says those areas are vastly underdeveloped and you need the "eye test" to give those stats their final meaning.

So, yeah. Welcome stat dude. Tell us something good. But let's hope BT and his experienced eye has the final say.

I would think those stats are meaningsless if he plays with the exact same 4 for players during this period. Where it could be more showing is if said player is moved around and how his stats differ? And what happens to his linemates stats with/without him etc..

 

[g00n]

He talked about you know hockey people, not you.

Now now, toodlums. Don't get all fussy just because I don't constantly hate on a bunch of players like you do.

Here's an analogy to consider...

Let's say you've got two golfers competing in the alternate shot portion of a competition like the Ryder Cup. Player A bombs a 300yd drive down the middle, then Player B practically shanks the next shot into a shortside bunker with the green sloping away severely. Player A somehow hits an amazing bunker explosion just over the lip and with enough spin to check up a few inches from the cup. Player B taps in.

From a "shot is a shot" standpoint both players took 2 shots each on that hole. Can we say they were of equal quality? Is that 300yd drive as valuable a skill as that tap in just because they both count as a shot on the scorecard? Does that shanked approach shot carry the same value as that incredible bunker shot?

In hockey this might be like a lazy blueliner flicking the puck in for a dump after a bad shift and putting it on net creating an easy save and change of actual possession instead. The lines change and play goes to the other end, where the new defenseman on the ice makes some outstanding reads and decisions against a tough line. He's able to free the puck long enough for an exit pass and the zone is cleared. The grinding line he's with pins a scoring line for the other team deep for an extended period but doesn't generate a single shot of its own. The other team finally clears and lines change again.

In that kind of situation, which is something teams who roll 3 or 4 lines strive for several times a night, D-man A has a shot going toward his CF and D-man B has done nothing of any note when viewed through typical "advanced" stats. In fact, if D-man B is unlucky enough to be on the ice for some shots-against as a result of D-man A's lazy dump, his stats may end up looking worse than A's.

A shot is a shot is a shot is possession??

I think this problem is one thing "Vic" is getting at in his comments. Some have assumed a large enough sample size accounts for these variations but that's circular reasoning...you are trying to determine which players are better than others while assuming they're all playing essentially the same over a long enough period of time.

I like stats. A lot. But I don't like stat abuse, such as relying too heavily on them for things they don't actually describe. And apparently neither does the creator of these often abused stats. It will be interesting to see how this plays out.





This should probably all be moved to the analytics thread, I'm guessing.

 

[g00n]

I would think those stats are meaningsless if he plays with the exact same 4 for players during this period. Where it could be more showing is if said player is moved around and how his stats differ? And what happens to his linemates stats with/without him etc..

No I don't think that solves the problem. The problem is with the core nature of the stats themselves and the razor thin margin between bad, mediocre and "elite". Extrapolating those stats into with and against variations is just an extension of the original flaws, turning it into more questionable data, or at least questionable CONCLUSIONS drawn from the data.

If anything, for data to be valid you need the variables as controlled as possible. Changing linemates actually complicates matters. More data is needed and more LIKE data is needed. You may be able to look back on an entire season and pinpoint a few good lines that generated a fair amount of shots against specific teams, but there are still too many variables to even say if it was the line combo or something else, and we can't really call it "possession" or make it the be-all-end-all of player evaluation.

Again, this is what the creator of these stats seems to be saying. There's a saying that's something to the effect of "if you only have a hammer then everything looks like a nail". That's the case with those who rely too heavily on these newer stats, in my opinion. Any new invention becomes that hammer. It's like when x-rays were discovered. People were using them for all sorts of things, including as a beauty treatment to eliminate blemishes and so forth. Problem was the customers were being turned into patients due to high levels of radiation. The new "big thing" was not what people wanted it to be. But it did have its limited uses.

I am for those limited uses, and firmly against banging everything with the same hammer.

 

[Hivemind]

I think people are misinterpreting his comments regarding the opinions of coaches and GMs. He's saying consensus tends to exist for a reason, not that the eye test trumps statistical analysis.

 

[g00n]

I think people are misinterpreting his comments regarding the opinions of coaches and GMs. He's saying consensus tends to exist for a reason, not that the eye test trumps statistical analysis.

The quote is pretty clear and that's not what he said.

*"The next big thing (in hockey analytics) will be to define the context of ice time ... quality of competition. Who you play against matters a ton." Barnes said where current hockey executives need help is how to use this to determine salary value, but those execs are far better at "weighing" players than those who rely solely on analytics. "If a 'hockey guy' comes out and says, 'That player was always [hidden],' they tend to be right. If you were to give me five hockey analytics versus five GMs or vice-presidents of hockey operations or five coaches -- five coaches especially -- to weigh a player, I'd take [the latters'] opinion ... A lot of analytics would have very young teams that won't win."

That's a pretty specific and detailed quote that says a lot more than consensus exist for a reason, or that it even exists at all. He's clearly saying trust the coaches first, suits second and analytics guys last. He even says why, and what would happen if you went just by analytics.

 

[Burakovsky95*]

I would think those stats are meaningsless if he plays with the exact same 4 for players during this period. Where it could be more showing is if said player is moved around and how his stats differ? And what happens to his linemates stats with/without him etc..

You are correct.

 

[ChibiPooky]

No I don't think that solves the problem. The problem is with the core nature of the stats themselves and the razor thin margin between bad, mediocre and "elite". Extrapolating those stats into with and against variations is just an extension of the original flaws, turning it into more questionable data, or at least questionable CONCLUSIONS drawn from the data.

If anything, for data to be valid you need the variables as controlled as possible. Changing linemates actually complicates matters. More data is needed and more LIKE data is needed. You may be able to look back on an entire season and pinpoint a few good lines that generated a fair amount of shots against specific teams, but there are still too many variables to even say if it was the line combo or something else, and we can't really call it "possession" or make it the be-all-end-all of player evaluation.

Again, this is what the creator of these stats seems to be saying. There's a saying that's something to the effect of "if you only have a hammer then everything looks like a nail". That's the case with those who rely too heavily on these newer stats, in my opinion. Any new invention becomes that hammer. It's like when x-rays were discovered. People were using them for all sorts of things, including as a beauty treatment to eliminate blemishes and so forth. Problem was the customers were being turned into patients due to high levels of radiation. The new "big thing" was not what people wanted it to be. But it did have its limited uses.

I am for those limited uses, and firmly against banging everything with the same hammer.

I don't necessarily agree that the data has to be as controlled as possible. You just need someone of sufficient expertise and experience to interpret it in order to correctly consider how reliable it is. Obviously the data will never be anywhere near 100% clean in a sport like hockey.

This is also where you're missing the line. Professional statisticians - these guys who work for NHL teams - aren't "hit it with the hammer" people. Message board posters are. The pros know how to make sure their data is good, their conclusions are correct, and what degree of confidence they can offer in their opinions. And they are all innovators, meaning that they inherently recognize that whatever way currently exists may not be the best there is.

I agree with you on the interpretation of Barnes' quote about the eye test. He pretty clearly said that "hockey guys" will be right more frequently than guys who only use stats (how's that for a stat guy way of putting it? ).

Edit: also, it's not about banging everything with the same hammer, but banging everything with as many hammers as possible and then seeing what it looks like. Let's hope Mr. Barnes is a dwarven forgemaster.

 

[g00n]

I don't necessarily agree that the data has to be as controlled as possible. You just need someone of sufficient expertise and experience to interpret it in order to correctly consider how reliable it is. Obviously the data will never be anywhere near 100% clean in a sport like hockey.

This is also where you're missing the line. Professional statisticians - these guys who work for NHL teams - aren't "hit it with the hammer" people. Message board posters are. The pros know how to make sure their data is good, their conclusions are correct, and what degree of confidence they can offer in their opinions. And they are all innovators, meaning that they inherently recognize that whatever way currently exists may not be the best there is.

I agree with you on the interpretation of Barnes' quote about the eye test. He pretty clearly said that "hockey guys" will be right more frequently than guys who only use stats (how's that for a stat guy way of putting it? ).

Edit: also, it's not about banging everything with the same hammer, but banging everything with as many hammers as possible and then seeing what it looks like. Let's hope Mr. Barnes is a dwarven forgemaster.

It's not my opinion that you need to limit variables and have "controls" in experimental situations. That's how science and data work. If you have faulty methodology that doesn't account for variables then the interpretation of the resulting data will always need to be couched with caveats about its accuracy. There are many who don't want to do that, and that's what I mean with the hammer analogy. It may be better than the stick we were using before but it's still not right for everything we're trying to do with it.