Analytics be damned!!

[Tyler Biggs*]

Beating the Habs and Flyers is always a fun start to the season.

 

[zeke]

From what I have read, the hockey analytics community seems to forget that in the baseball analytics community, they don't start with team stats. Team stats are just results for them, not the basis of their analysis. What they do is build projections for team stats based on projected INDIVIDUAL stats. Their projections for each season aren't based on the overall team stats from the previous season - in fact they'd think that was fairly crazy. In baseball, they project for each individual on the team first, and THEN build a team projection from those individual projections.

The hockey guys seem to miss that completely. They for the most part ignore individual projections, and actually try to project total team performance simply based on previous years' total team performance, while only giving lip service to individual roster changes. i.e. they'll mention that they think the additions of player X or Y will improve the team, but they won't actually be able to put any actual number on that individual as to exactly how he will impact the team. They resort to guessing (for the most part), based on a personal weighting of the various individual advanced stats - because in hockey, there is nothing approaching the individual WAR stats that there are in baseball.

But no matter which way I look at it, when I actually analyze the track records of the individual Leaf players on this team and build a team projection based on those individual projections, this Leafs team comes out looking very good. And the analytics community yelling about team SH% and SV% from a 48gm season last year, with a roster that has seen at least 1/3 turnover this year, really doesn't do a good job convincing me otherwise.


Let me try and say it another way. (this is just one of many ways to look at it - and I've tried all sorts of different methods this offseason, and all of them give similar results).

Let's break down the Leafs' roster by career goals per game, and pace their career GPG out to a full season. Given that nobody on the team is older than 31, and only Orr is older than 30, none of them are on the downside so this shouldn't be too controversial.

I'll factor in everyone averaging out to 7gms missed to injury apiece - so I'll pace each of the regular starters out to 75gms each. these leaves 84 forward games and 42 defensemen games leftover, so I'll pace two extra forwards out and one extra dman out to 42 games apiece. I'll include icetime to show that minutes distribution isn't a real concern, either.


Bozak: .23gpg, 19:21 = 17gls
Kessel: .37gpg, 17:48 = 28gls
Lupul: .30gpg, 16:10 = 23gls

Kadri: .26gpg, 15:34 = 20gls
VanRiemsdyl: .27gpg, 15:06 = 20gls
Clarkson: .23gpg, 14:07 = 17gls

Bolland: .22gpg, 16:26 = 17gls
Raymond: .22gpg, 15:16 = 17gls
Kulemin: .21gpg, 15:53 = 16gls

McClement: .12gpg, 15:07 = 9gls
McLaren: .05gpg, 5:19 = 4gls
Orr: .03gpg, 6:09 = 3gls

Ashton: .00gpg, 10:05 = 0gls
Bodie: .06gpg, 8:40 = 3gls



Phaneuf: .18gpg, 25:02 = 14gls
Franson: .09gpg, 15:54 = 9gls

Gardiner: .08gpg, 21:19 = 6gls
Ranger: .07gpg, 21:33 = 5gls

Gunnarsson: .05gpg, 20:33 = 4gls
Fraser: .02gpg, 13:43 = 2gls
Rielly: .00gpg, 00:00 = 0gls



That's a total of 234 goals, or 2.85gpg.

Here's how 2.85gpg would rank the past few years:

12-13: 7th
11-12: 8th
10-11: 12th
09-10: 7th

So, in other words, based on a reasonable projection off of career scoring numbers, we'd expect this Leafs team to be a top offensive team, period.


If we don't like career numbers (even though with our young team career numbers should make us look worse, not better), we can look at a more relevant recent sample - say, the last 2yrs. Here's how it would look based on the past two years:

Bozak: .25gpg, 19:41 = 19gls
Kessel: .44gpg, 19:57 = 33gls
Lupul: .44gpg, 18:07 = 33gls

Kadri: .17gpg, 15:29 = 25gls
VanRiemsdyk: .32gpg, 17:15 = 24gls
Clarkson: .35gpg, 16:50 = 26gls

Bolland: .23gpg, 16:27 = 18gls
Raymond: .20gpg, 15:51 = 15gls
Kulemin: .12gpg, 15:50 = 9gls

McClement: .14gpg, 14:19 = 11gls
McLaren: .07gpg, 5:09 = 5gls
Orr: .04gpg, 6:15 = 3gls

Ashton: .00gpg, 00:00 = 0gls
Bodie: .00gpg, 00:00 = 0gls



Phaneuf: .16gpg, 25:15 = 12gls
Franson: .09gpg, 17:20 = 9gls

Gardiner: .08gpg, 21:19 = 6gls
Gunnar: .04gpg, 21:34 = 3gls

Ranger: .00gpg, 00:00 = 0gls
Fraser: .00gpg, 16:44 = 0gls
Rielly: .00gpg, 00:00 = 0gls


That gives us a total of 251 goals, or 3.06gpg.

That would rank like this the past few years:

12/13: 4th
11/12: 4th
10/11: 5th
09/10: 5th

So if we look at the more relevant recent numbers, then the Leafs look even better, and project as an elite offensive team.

Now even if we drop that offense by ANOTHER 10% (after already adjusting down to 75gms per starter), due to say a shooting percentage regression or mass injuries, then they end up with 226gls, or 2.76gpg. Here's how that would rank the past few years:

12/13: 9th
11/12: 10th
10/11: 14th
09/10: 13th

even in a worst case scenario, our offense is still likely challenging for top-10 in the league.

So basically, there's no realistic scenario I see where this team won't at least above average offensively, likely top-10 offensively, and potentially elite top-5 offensively.

And none of the Analytic stats do a good job of convincing me that I should feel any different about the Leafs offense.


Then, on the flipside, we have the Leafs' goaltending.

Reimer:

12/13: .924
11/12: .900
10/11: .922
Total: .915

Bernier:

12/13: .922
11/12: .909
10/11: .913
09/10: .957
08/09: .864
Total: .914


Even if we say that these two very young goalies will not improve this year with more experience, and even if we say they will not sustain their goaltending from last year and will drop down to career norms, we're still looking at a goaltending tandem that will give us ~.915sv%. A tiny step down from last year's .917sv% mark.

Here's how that would rank the last few years:

12/13: 12th
11/12: 9th
10/11: 9th
09/10: 8th

If we say that these very young goalies have actually improved over their careers, and can come closer to matching what they did last year, then we're looking at .920sv% goaltending, which means we're challenging for top-5 in the league.

If we say they come somwhere in between, then they're coming in around the .917 they put up last year, which is comfortably top-10 in the league.

If we say something crazy like these very young goalies might actually even IMPROVE over what they did last year (crazy, I know), then we're looking at elite goaltending.


So much like the team offense, I can't see any way to reasonably predict anything less than above average goaltending for the leafs, could quite reasonably predict well above average goaltending, and even the possibility of elite goaltending isn't out of the question.

And again, there's nothing in the Analytics which does a good job convincing me that we should expect significant regression in our sv% this year, either.


Now there's always disaster scenarios of team-wide implosion or injuries, but in terms of realistic predictions I have a hard time seeing any convincing data, when actually looking at our roster's performance history, that convinces me that we shouldn't expect this team to challenge for top-10 status in both goal production and goaltending, with average (~15th) for both being the realistic low-end, and top-5 being realistically possible in the optimistic scenario.

Now there is a third factor that I don't deal with here, and that's team defense (basically, in analytics terms, shot attempts against), and that could realistically be bottom 5 again, no question about it, due to our style of play and performance history. BUT, even if it is bottom 5 again, that is still not enough to drag the team down to a finish like the ones Mirtle or Pronman predict.

 

[Gutchecktime]

... that's a damn good post . [In reference to Zeke's post]

Chances anyone bothers to respond to this well-thought out rebuttal - 0%, IMO.

 

[Epictetus]

I can only explain myself, however, this link is what I meant by Clutch... http://www.fangraphs.com/library/misc/clutch/

What is a 'high leverage situation' and what is a 'neutral environment', though?

So you'd rather be compiling numbers than actually watching the sport?

I never said that.

But you can formulate it this way.

If you are an owner and have 200k to spend, are you going to trust a scout with professional experience who relies simply on 'intuition' and subjective\emotional decision-making, or the scout with a more concrete, mathematical foundation for a specific player?

Lol what a joke

Not everyone has the Leafs in the playoffs.

and this year is ESPECIALLY glaring because they are insanely drawing these conclusions from a the tiny half-season sample we had last year. even a full season is a small sample for the advanced stats with a weak correlation to future wins - a half-season sample is even weaker by orders of magnitude.

The small sample size is often brought up. I don't see the same conclusions being drawn from it. Majority of the data encompasses years worth, not 48 games. But even so, this is no different than using the playoffs. You can make the case that the regular season provides a more concrete basis (larger sample size) than the playoffs.

But an "Analytics Guy" is not allowed to draw this fairly simple conclusion, because last year Bozak had a 19.7sh%, an 11.1on-ice sh%, and a 1027 pdo. The "analytics guy" will be forced to claim that Bozak is very likely to suffer significant regression this year from last year's production - despite the fact that that would mean Bozak would "regress" to a level well below his relatively consistent career norms - career norms that were established before last year, even with less "lucky" advanced statistics.

How much regression is this 'analytics guy' assuming will occur for Bozak? I don't think you've stated an actual number.

And there are always cases where a player drops below their career consistent norms.

People who make yearly statistical predictions for players always engage in regression situations -- are you saying the analytical guy's justification is simply incorrect or weaker than traditional means of doing this?

And in fact, when you actually look at all the leafs' career production, weighted for recentness and adjusted for minutes played, it's very hard to argue that many leafs overachieved offensively last year, despite what the advanced stats say. About the only two leafs who played a significant amount of games that could legitimately be said to overachieve were kadri and franson - but even then, those were young guys gettting their first legit offensive opportunities, so their lack of track record is hard to hold against them. And moreover, a number of players, either due to poor performance or injury, clearly underachieved last year to balance those guys out - Lupul, Grabo, gardiner in particular.

I agree with the majority of what is said here, excluding the dig at the Leafs and their regression is, since I don't know what specific advanced stats argument you are arguing against. It seems you are more or less attacking some kind of general view that the Leafs are due for a regression.

I would also add in James Reimer, who was a big reason for the Leafs making the playoffs. He provided the Leafs with solid goaltending, something they've lacked.

What we've ended up with is having so many people with a fairly poor understanding of statistics, who have developed unjustified tunnel vision when it comes to these possession stats - they have literally convinced themselves that last year's half-season sample of possession stats has to be the "base" of their analysis, and are actually completely disregarding contrary facts like the Bozak example I showed above. For them, it doesn't matter that Bozak scored at his usual rate last year - he has to be due regression, because the advanced stats from a half-season sample size say that he was lucky last year.

1) I don't see anyone overlooking the small sample size.

2) Many, many players regress below their career norms. I haven't read the actual argument you are attacking here, or seen how much regression we are talking about, either (you only say "well-below" -- but how much quantified? I think it is also fair to say that Bozak needs both JVR and Kessel to produce. His success is thus correlated to either of them.

3) What is the analytical guys argument for Bozak's regression?

I think you have it a bit backwards here. It's the corsi enthusiasts that are telling people that the traditional idea of assessing teams and players based on, you know, actually observing a hockey game and checking the standings is arcane. It is pretty arrogant.

This is what I don't understand: 'corsi enthusiasts' watch the game and check standings, too. This is not something that needs to be separated. Of course there are people who separate the two, but you are probably speaking of a minority.

Advanced stats people look for ways to improve and evaluate the game. Why? Because they love the game itself.

Hockey is a very dynamic of a game, and has many variables which change from shift to shift, and second to second. A good model should capture all these variables. It is literally exponentially more complex to accurately model a hockey game than it is to model something like a baseball game which has discrete outcomes for every pitch. The traditional NHL stat categories does a terrible job capturing all these things, and now, the advanced stats do a slightly less terrible job. Neither will soon replace the need for serious hockey people to make player/team assessments based on watching hockey.

These variables generally, even out. A bad penalty call one game, and a favorable penalty call next game. You hit the post, your opponent hits the post. Etc.

The point about hockey being more dynamic than baseball is true, though. But that doesn't mean advanced statistics have no place in hockey. I've seen numerous people suggest so.

It's already finding a place within the game. So the 'never replacing' talk is premature.

 

[Duke Silver]

Duke's response (the definition of momentum from the dictionary) references words like "perception" and phrases like "everything seems to go wrong". As I've mentioned a few times (and you and others have conveniently ignored) - just because it seems like it exists doesn't mean that it actually does.

Just because you feel deep down that it exists doesn't mean that it actually does. Sort of like Santa Claus.

From the Oxford Dictionary of Sports Science.

I don't know why you asked for a definition in the first place if your plan was to just disregard it and use your Santa Claus line again. This conversation is clearly going nowhere with that attitude.

 

[Sproo]

This is what I don't understand: 'corsi enthusiasts' watch the game and check standings, too. This is not something that needs to be separated. Of course there are people who separate the two, but you are probably speaking of a minority.

In general, I actually support using statistics to inform your opinions of players, but the people who are defending advanced analytics in this thread are clearly separating the two, or at the very least telling us that evaluating players based on watching games is inferior to statistical analysis, and that kind of thinking is very wrong, IMO. For instance, earlier in this thread you said:

Eh. There are flaws in advanced stats, but I will take it over the 'eye test' and various other visual cues like 'watching the game' that are even more flawed when it comes to player evaluation.

And The Legend said this

The increased use of stats (they really aren't so advanced) in the mainstream makes me happy because it means that the days of the former player-turned commentator who tosses out opinions based on emotion and bias are beginning to come to an end.

Now if I were an NHL team owner and my GM told me that he was going to fire all our 'old-school' scouts who relied on watching games and intuition to evaluate players and replaced them with new 'advanced stats' scouts who are skeptical of watching players and relied mainly on spreadsheets, then I would fire him that moment.

 

[The Legend]

In general, I actually support using statistics to inform your opinions of players, but the people who are defending advanced analytics in this thread are clearly separating the two, or at the very least telling us that evaluating players based on watching games is inferior to statistical analysis, and that kind of thinking is very wrong, IMO. For instance, earlier in this thread you said:



And The Legend said this



Now if I were an NHL team owner and my GM told me that he was going to fire all our 'old-school' scouts who relied on watching games and intuition to evaluate players and replaced them with new 'advanced stats' scouts who are skeptical of watching players and relied mainly on spreadsheets, then I would fire him that moment.

So you took my comment as being "fire all scouts"?

Good times.

 

[Sproo]

So you took my comment as being "fire all scouts"?

Good times.

No. But congratulations on again pulling out parts of someone's post out of context in an attempt to make them look foolish and dismiss them.

If you need me to spell it out for you, I was pointing out that you and Epictetus are saying that watching a hockey game and evaluating players based on that along with your intuition is inferior to advanced stats.

 

[The Legend]

In general, I actually support using statistics to inform your opinions of players, but the people who are defending advanced analytics in this thread are clearly separating the two, or at the very least telling us that evaluating players based on watching games is inferior to statistical analysis, and that kind of thinking is very wrong, IMO. For instance, earlier in this thread you said:



And The Legend said this



Now if I were an NHL team owner and my GM told me that he was going to fire all our 'old-school' scouts who relied on watching games and intuition to evaluate players and replaced them with new 'advanced stats' scouts who are skeptical of watching players and relied mainly on spreadsheets, then I would fire him that moment.

No. But congratulations on again pulling out parts of someone's post out of context in an attempt to make them look foolish and dismiss them.

If you need me to spell it out for you, I was pointing out that you and Epictetus are saying that watching a hockey game and evaluating players based on that along with your intuition is inferior to advanced stats.

You referenced me at the end. I found your conclusion and commented. It's not malicious.

I'm fine with people evaluating players based on watching them. Never said I wasn't. What I am tired of is people (generally ex players who are now employed in the media) providing "analysis" in the form of tired cliches (ie "wanted it more", "had momentum", etc)

 

[Sproo]

I'm fine with people evaluating players based on watching them. Never said I wasn't. What I am tired of is people (generally ex players who are now employed in the media) providing "analysis" in the form of tired cliches (ie "wanted it more", "had momentum", etc)

This I can agree with. Once in a while I find it interesting to get the perspective of former NHL'ers or coaches (even Don Cherry makes an insightful comment a few times a year), but yeah, the tired cliches are indeed nauseating. If only there was a 'mute commentators' button. But that said, would James Mirtle make a better commentator than Craig Simpson? I doubt it...(No offense if you're reading this James! )

 

[mix1home]

In short, you can use your ASS (Advanced StatiStics), but I prefer to use my HEAD (Hockey Eye Analysis + Discussion).

 

[Cor]

Bizarre Jeffler ‏@Jefffler 2m

@mirtle When can I expect the advanced stats to catch up and the Leafs to crash and burn? Still waiting since last season.

 

[Jeffler]

Bizarre Jeffler ‏@Jefffler 2m

@mirtle When can I expect the advanced stats to catch up and the Leafs to crash and burn? Still waiting since last season.

Why would you quote a parody of me in a serious thread?

 

[TieClark]

His last 3 articles have no mention of advanced statistics. One is about age, the other two about fighting. I'm not sure why you are bating him, but it'd be helpful if you had actually provided some evidence (specifically to the claim that he used them in every article). It's okay to disagree with Mirtle, but let's not start attacking him without proper cause.

I'm not talking about things such as reporting facts... reporting fact is reporting facts. I'm talking about opinion pieces

Do you realize you completely missed his point?

Regardless about how wrong you are about other things, he said that he doesn't reference Corsi as frequently as was claimed. Your response insinuated that he was wrong because he references advanced stats in every article.

I hope you realize that Corsi and advanced stats aren't completely synonymous....

I didn't miss anything... I didn't claim he mentioned advanced statistics in every article... what I said was advanced statistics are used to form every opinion he has on the game, which in turn leads to every article he writes to be based on advanced statistics.

 

[Cor]

Why would you quote a parody of me in a serious thread?

No jokes allowed.

 

[James Mirtle]

I'm not talking about things such as reporting facts... reporting fact is reporting facts. I'm talking about opinion pieces

I didn't miss anything... I didn't claim he mentioned advanced statistics in every article... what I said was advanced statistics are used to form every opinion he has on the game, which in turn leads to every article he writes to be based on advanced statistics.

a) In some cases, I find the advanced statistics available influence my opinion as I've seen them be very accurate.

b) It would be completely impossible to form every opinion based on advanced statistics given the limited ones available. There's no way to use them for goaltending, for example, or to evaluate special teams play. There are simply a set of situations where advanced stats can help, as when teams (Toronto, Anaheim) or players (Kadri, Fraser) are massive outliers in things like shooting percentage or PDO, and a regression is likely.

c) In my opinion, it's undeniable the Leafs central weakness is their possession game, and the organization's main focus for improvement should be on improving in that area in order to make the next step. Randy Carlyle has said something similar recently when Bob McKenzie asked about improving possession - and this is why analytics are often referenced for this area of their game. You can win without being a good possession team; it is often simply more difficult and harder to sustain. Corsi/Fenwick, or shot attempt metrics of any kind, if used properly, are a very good indicator of possession.

I play hockey, and have for nearly 20 years. Members of my family have played professionally. I'm from a traditional, hockey crazy city and have attended 50+ games a year my whole life. My opinions on hockey are not all about advanced statistics; the way I would frame it is that, out of all the writers for mainstream outlets, I'm one of the few who considers these numbers at all in my work. I think they add considerable value.

If you want analysis without any of this taken into account, you can pretty much read any other newspaper writer out there, rather than picking on the one guy who uses this new information.

Sorry - I can't respond to everyone in this thread. I'm not having a bet with anyone here - that's not something The Globe would condone - but I've put my reputation on the line with my picks, as always. I don't think the Leafs finishing with eight fewer points than they were on pace to a year ago is that unreasonable, as I think they will score fewer goals and suffer more serious injuries, among other things. I will say this: If Bernier is as good as he's looked so far, there's no question they make the playoffs. Predicting what an inexperienced goaltender will do is very, very difficult, though, and I had him more in line with his career averages.

I'm always very accessible - I answer most people on Twitter and email. Thanks.

 

[Gutchecktime]

I don't think the Leafs finishing with eight fewer points than they were on pace to a year ago is that unreasonable, as I think they will score fewer goals and suffer more serious injuries, among other things.

JVR - Bozak - Kessel
Lupul - Bolland - Raymond
Kulemin - Kadri - Clarkson

You can switch 'em around all you like but this is a damn good top 9. I suspect the Leafs finish Top 10 in Goals For for the 3rd straight season. Hopefully you'll be open to admitting that they might just be good shooters at that point.

 

[zeke]

and suffer more serious injuries

don't think the leafs were particularly lucky with injuries last year.


C Bozak 2gms
W Kessel 0gms
W Lupul 32gms

C Kadri 0gms
W JVR 0gms
W Mac 8gms

C Grabo 0gms
W Kuly 0gms
W Frattin 23gms

C McClement 0gms
W Komarov 6gms
W Orr 4gms


D Phaneuf 0gms
D Gunnar 11gms

D Gardiner 0-36gms?
D Liles ~10gms?

D Franson 0gms
D Fraser 0gms


G Reimer ~10gms?



not especially injured, not especially healthy.

 

[James Mirtle]

I love how these hockey nerds devote themselves to possession statistics yet there is only a moderate-at-best correlation to winning. The best is when a team like the Leafs consistently proves them wrong and instead of accepting that their stats really aren't that good they'll just call the Leafs "Lucky". Its golden LOL

All right... one more and then back to work.

Possession statistics, as a predictive measure, within an 82-game season, are as strong an analytic as there is available. In a 48-game season, they lose much of their value as by the time you have a big enough possession sample size, the season is close to over and any potential regression will not have enough time to occur.

The other huge misconception is that people are using solely possession statistics to make these somewhat negative predictions for the Leafs. If you look at Vollman's work predicting a downturn, he doesn't use possession statistics at all - his five measures are PDO, STI, injuries, overtime/shootouts and one-goal games. This is explicitly explained in my story on this, as well as how some of these stats regress to the mean.

The Leafs have actually not consistently proven anything wrong to this point. Anyone with a solid grasp on this type of analysis would point out immediately that you often need a 70+ game sample size for the effect of the regression to play out. And Toronto could actually become a better possession team this season; I've written how the changes on the blueline could be a real positive there.

But they'd have to go a full 82-game season (at least) winning with low possession and high PDO to really refute the analytics. We need this season to play out before we know for sure.

One point I do agree with that some of you have mentioned here is that there are now a lot of people using analytics online that don't understand all of the context needed to properly apply them. A lot of both the criticism and use of these stats is really dumbing down their principles, and as a result, the analysis is faulty.

 

[James Mirtle]

I suspect the Leafs finish Top 10 in Goals For for the 3rd straight season. Hopefully you'll be open to admitting that they might just be good shooters at that point.

I already think they're good shooters. I don't think they're 11.5% shooters and coming down to even 10% is quite a shift. I had them on the lower end of the top 10 in offence this season.

 

[James Mirtle]

don't think the leafs were particularly lucky with injuries last year... not especially injured, not especially healthy.

Vollman's analysis in Hockey Abstract on this is very in-depth. He compares every team over the previous five years and on that basis the Leafs were pretty healthy. It's likely a more substantial portion of the Leafs cap dollars are spent on players on IR this year.

 

[Garbs]

Why would you quote a parody of me in a serious thread?

The internet is serious business.

 

[HarrisonFord]

James, there's a difference between likelihood and inevitability.

 

[HarrisonFord]

Additionally, the Leafs have currently outshot opponents in 2 of the 3 games so far this year. Obviously 3 games isn't enough games to say that things have changed, but it's encouraging nonetheless

 

[Cap'n Flavour]

James, there's a difference between likelihood and inevitability.

Yes there is, so what's your point?

Additionally, the Leafs have currently outshot opponents in 2 of the 3 games so far this year. Obviously 3 games isn't enough games to say that things have changed, but it's encouraging nonetheless

Great, so extrapolate that out to 82 games and you get nonsense.