Corsi, shot quality, and the Toronto Maple Leafs

[Gutchecktime]

Why do you insist upon assuming that "the analytics guys" all feel the same way about this?

My apologies - I shouldn't lump everyone together. I've yet to hear any analytics guy who predicted doom for the Leafs admit they weren't correct though.

 

[Delicious Dangles*]

For clarity, though, I was using the mathematical definition of .500 (i.e. through summing all wins and all losses), rather than the definition utilized by nhl.com. I think that's the only fair thing to do in a league where one win is awarded for every game played.

So in other words, the only fair way to analyze in a league is to completely ignore how the league operates.

 

[Bear of Bad News]

My apologies - I shouldn't lump everyone together. I've yet to hear any analytics guy who predicted doom for the Leafs admit they weren't correct though.

No worries. We're not all the same, although we may seem that way at times.

 

[Delicious Dangles*]

Per your request:

Using data from 2007-08 to 2010-11, I assessed the predictive validity for the following measures, from one randomly selected half of the season to another randomly selected half (with no crossover among the games selected for each sample). The figures below represent the average r and r^2 values over 1000 data samples for each season.

1st half points percentage - 2nd half points percentage: r = 0.38; r^2=0.15
1st half adjusted fenwick - 2nd half points percentage: r= 0.49; r^2=0.24
1st half underlying numbers - 2nd half points percentage: r=0.53; r^2=0.29

The 'underlying numbers' simply regresses each team's even strength and special teams statistics on a bayesian basis and uses the regressed figures as talent estimates for each statistic in question.

The adjustment to fenwick is based on time spent trailing versus time spent leading.

So like I said - shot based measures have more predictive validity.

First off, those are horrible results all around for claiming predictive value. You can pretty much say they all have none.
Second, where's goal differential?
Third, this was supposed to be from season to season. Not sure where we are getting half seasons from.
Fourth, how do you get 1000 data samples when looking at half seasons, and you only have 4 seasons and 30 teams?
Fifth, how are the half seasons randomized if you're using two with no overlap and two make up one season.
Unless you're comparing two random half seasons within a 4-year period, in which case I say DUH there is little correlation. Teams change so much within a 4-year period that they are barely recognizable.

 

[Delicious Dangles*]

Last year, the Leafs scored 145 goals and allowed 128. That was good for the 8th best goal ratio in the league. This was primarily accomplished by having an all situations PDO of 1032.

This year, the Leafs have scored 187 goals and allowed 195, which places them 19th in the league in goal ratio. Their PDO is still high at 1020, but somewhat lower than last year.

Advanced stats proponents predicted that the Leafs would regress, and they have - they've gone from 8th in the league in goal ratio to 19th. People can and will argue all day about the importance of fenwick or PDO, but I don't see anyone disagreeing with the proposition that goal ratio or goal differential is important.

I thought that looked at 5 on 5 numbers though?

In 5 on 5 goal differential, Leafs last year were 12th with 1.05. This year they are 13th with 1.02.

 

[Master_Of_Districts]

The Leafs are going to make the playoffs for yet another year with among the league's worst in Corsi. The Leafs did not regress as you've been banging your drum about all season.

They did regress. The data I posted evidences that.

And you still seem to be unable to admit you were wrong about them. You're having to resort to - as you admitted - cherry picking the data instead of just saying that they haven't really done what you expected them to.

The data in the post you quoted is not cherry picked. It's from all of 2012-13 and 2013-14 thus far.

 

[Master_Of_Districts]

First off, those are horrible results all around for claiming predictive value. You can pretty much say they all have none.
Second, where's goal differential?
Third, this was supposed to be from season to season. Not sure where we are getting half seasons from.
Fourth, how do you get 1000 data samples when looking at half seasons, and you only have 4 seasons and 30 teams?
Fifth, how are the half seasons randomized if you're using two with no overlap and two make up one season.
Unless you're comparing two random half seasons within a 4-year period, in which case I say DUH there is little correlation. Teams change so much within a 4-year period that they are barely recognizable.

1. Re-read my last post relating to the theoretical upper limit. If you still don't understand, I'll try explaining again.

2. I can re-run the numbers with goal differential. If I'm gracious enough to indulge you, that is.

3. Supposed to be from season-to-season? No. That would be amateurish.

4. 1000 data samples per season. Each sample consists of two randomly selected sets of 40 games.

5. Should be self-evident. And no - that's not what I did. Duh.

 

[Gutchecktime]

They did regress. The data I posted evidences that.

The data in the post you quoted is not cherry picked. It's from all of 2012-13 and 2013-14 thus far.

No, they didn't. They played at a 97 point pace last year and are playing at a 97 pace this year.

The data you posted just shows they have a worse goal differential. I'm not sure why that matters. You were wrong. The people that predicted doom and gloom for the Leafs were wrong.

 

[Master_Of_Districts]

I thought that looked at 5 on 5 numbers though?

In 5 on 5 goal differential, Leafs last year were 12th with 1.05. This year they are 13th with 1.02.

Right.

And I was looking at overall numbers.

Not sure where the confusion lies...?

 

[Master_Of_Districts]

No, they didn't. They played at a 97 point pace last year and are playing at a 97 pace this year.

The data you posted just shows they have a worse goal differential. I'm not sure why that matters. You were wrong. The people that predicted doom and gloom for the Leafs were wrong.

Right.

Their goal differential regressed. Shot metrics are relevant insofar as they affect goal differential. They don't dictate what happens in the shootout, which is entirely random.

Not sure if this is a comprehension issue on your end or...

 

[Master_Of_Districts]

So in other words, the only fair way to analyze in a league is to completely ignore how the league operates.

Ultimately - it's an issue of semantics.

Personally, I think it's incongruous to utilize a definition of .500 that renders 23 out of the league's 30 teams as over .500.

 

[Gutchecktime]

Right.

Their goal differential regressed. Shot metrics are relevant insofar as they affect goal differential. They don't dictate what happens in the shootout, which is entirely random.

Not sure if this is a comprehension issue on your end or...

The Leafs have been just as successful this year as they were last year. If you predicted they wouldn't be successful because of shot metrics, you were wrong.

There's no problems with comprehension at all.

 

[Master_Of_Districts]

The Leafs have been just as successful this year as they were last year. If you predicted they wouldn't be successful because of shot metrics, you were wrong.

There's no problems with comprehension at all.

Their goal ratio has regressed.

That was my point.

 

[Delicious Dangles*]

1. Re-read my last post relating to the theoretical upper limit. If you still don't understand, I'll try explaining again.

2. I can re-run the numbers with goal differential. If I'm gracious enought to indulge you, that is.

3. Supposed to be from season-to-season? No. That would be amateurish.

4. 100 data samples per season. Each sample consists of two randomly selected sets of 40 games.

5. Should be self-evident. And no - that's not what I did. Duh.

1. How can you get an upper limit based on exact, unchanging valuations of talent when those don't exist? That also does not excuse how terrible the predictive value is. It is essentially saying it is terrible, but it is slightly better than correlating two random half-seasons, so lets go all willy-nilly with predictions that are just as likely wrong, and treat them as fact.
2. Since that is what you first said you would do, that would be good. Though of course those numbers will only see the light of day if you think they support your theories.
3. Then why were there predictions about this season based on last season?
How is cutting the time frame in half supposed to be any less amateurish?
4. Yeah. How do you get 100 data samples per season representing 80 games when there are only 30 teams playing 82 games per season. That does not add up.
5. Then explain what you did, in actual and not intentionally misleading terms.

 

[Master_Of_Districts]

1. Re-read my last post relating to the theoretical upper limit. If you still don't understand, I'll try explaining again.

2. I can re-run the numbers with goal differential. If I'm gracious enough to indulge you, that is.

3. Supposed to be from season-to-season? No. That would be amateurish.

4. 1000 data samples per season. Each sample consists of two randomly selected sets of 40 games.

5. Should be self-evident. And no - that's not what I did. Duh.

I'm effectively utilizing the same method that professor Brian Macdonald employed when he performed his own study regarding the predictive validity of various measures of team performance. The only difference is that I examined many different samples of randomly selected sets of games, rather than looking at the correlation between odd numbered games and even numbered games.

I'd suggest reading professor Macdonald's study.

http://www.academia.edu/2483597/An_Expected_Goals_Model_for_Evaluating_NHL_Teams_and_Players

If you read his study and still don't understand the method, then I can't help you, unfortunately.

 

[Bear of Bad News]

If you read his study and still don't understand the method, then I can't help you, unfortunately.

If the goal's to get him to understand, then why wouldn't you want to try and help?

As someone watching this conversation, it seems like the problem is that you're speaking different languages. If a dictionary were made available, wouldn't that help the process?

 

[Gutchecktime]

Their goal ratio has regressed.

That was my point.

This is what I mean about tying yourself in knots.

Until the NHL orders the standings by goal ratio, it doesn't really mean much. The people saying the Leafs would be worse due to shot metrics weren't talking about just goal differential.

 

[Master_Of_Districts]

1. How can you get an upper limit based on exact, unchanging valuations of talent when those don't exist? That also does not excuse how terrible the predictive value is. It is essentially saying it is terrible, but it is slightly better than correlating two random half-seasons, so lets go all willy-nilly with predictions that are just as likely wrong, and treat them as fact.
2. Since that is what you first said you would do, that would be good. Though of course those numbers will only see the light of day if you think they support your theories.
3. Then why were there predictions about this season based on last season?
How is cutting the time frame in half supposed to be any less amateurish?
4. Yeah. How do you get 100 data samples per season representing 80 games when there are only 30 teams playing 82 games per season. That does not add up.
5. Then explain what you did, in actual and not intentionally misleading terms.

1. It's a theoretical upper limit. It goes without saying that the theoretical upper limit will somewhat exceed the practical upper limit. Which actually assists my argument.

In any case, the predictive validity is not terrible. Adjusted fenwick predicts 75% of the non-luck variance in future results. The underlying numbers model predicts 90% of the non-luck variance. Far from terrible. There's tonnes of utility there.

Slightly better than correlating two random half seasons? Hardly. If you think otherwise, you simply don't understand.

2. I've run the numbers in the past, and the predictive was virtually identical to points percentage. And no - I have no issue posting the results.

3. Frankly, I have no idea what you're getting at here. Because predictions were made about this season on the basis of last season, that somehow precludes utlizing a within-season analysis? The chain of reasoning is so bizarre that I'm forced to wonder whether you're simply being wilfully obtuse at this point.

A within-season analysis is obviously preferable as it mitigates the impact of roster turnover. Does that mean that a between-season analysis is useless? No. It's simply not as rigorous.

4. I meant to write 1000. I'll try one more time - for each season from 2007-08 to 2010-11, I randomly selected two independent sets of 40 games. I looked at the correlation between the two data sets for each of the three variables in question, in order to assess the predictive validity of each variable. I repeated the process 1000 times for each season. The figures I posted represent the average values for all four seasons.

 

[Delicious Dangles*]

Right.

And I was looking at overall numbers.

Not sure where the confusion lies...?

Why are overall numbers what now matters when I have been told time and time again that 5 on 5 numbers are the only important ones?

 

[Master_Of_Districts]

Why are overall numbers what now matters when I have been told time and time again that 5 on 5 numbers are the only important ones?

I never made that assertion, so...

 

[Delicious Dangles*]

Right.

Their goal differential regressed. Shot metrics are relevant insofar as they affect goal differential. They don't dictate what happens in the shootout, which is entirely random.

Not sure if this is a comprehension issue on your end or...

If shot metrics are only relevant for the purpose of evaluating affects on goal differential, which you then extrapolate to quality of team, then shouldn't actual goal differential have better predictive value for points than those shot metrics?

Are these shot metrics also only based on 5 on 5? Why are you then not looking at 5 on 5 goal differentials?

 

[Master_Of_Districts]

If the goal's to get him to understand, then why wouldn't you want to try and help?

As someone watching this conversation, it seems like the problem is that you're speaking different languages. If a dictionary were made available, wouldn't that help the process?

I'm genuinely trying - believe me.

But part of me thinks that he doesn't want to understand.

He'd rather insult my integrity and insinuate that I'm fabricating the data.

 

[Delicious Dangles*]

Ultimately - it's an issue of semantics.

Personally, I think it's incongruous to utilize a definition of .500 that renders 23 out of the league's 30 teams as over .500.

Unfortunately, those are variables that you cannot change. That is how the NHL does it, so for the purposes of predicting for the NHL, it must be done the way they say.

 

[Master_Of_Districts]

If shot metrics are only relevant for the purpose of evaluating affects on goal differential, which you then extrapolate to quality of team, then shouldn't actual goal differential have better predictive value for points than those shot metrics?

No - and it doesn't.

Are these shot metrics also only based on 5 on 5? Why are you then not looking at 5 on 5 goal differentials?

There's is nothing that would prevent the same analysis from being applied to even strength data.

If you do that, the same pattern holds as far as predictive validity: shot based metrics outperforming goal differential as a predictor of future goal differential.

 

[BoredBrandonPridham]

If shot metrics are only relevant for the purpose of evaluating affects on goal differential, which you then extrapolate to quality of team, then shouldn't actual goal differential have better predictive value for points than those shot metrics?

Are these shot metrics also only based on 5 on 5? Why are you then not looking at 5 on 5 goal differentials?

My understanding is that goal differential would actually be of better predictive quality than shot differential, but unfortunately there is simply not a large enough sample size to work with. Shots are a larger sample size and seem to do a good job predicting what direction a team's goal differential will trend (and presumably subsequently wins/losses).