Unsustainable 2014-2015

[DL44]

Kings should be doing better as per plenty of stats but there are also very visible indicators of why they are not. Though if some of those simple things were sorted out, you could theoretically expect a hot streak in the back end of the season.

It's the Kings... not expecting my expectation to be theoretical.. Virtual lock.

 

[SnowblindNYR]

I have a serious question. Why is it that like 40 games in people are still saying something is unsustainable? It sustained 40 games, even if it's an outlier, why can't the outlier keep being an outlier for the next 40? I never got that. If there were 1,000 games played in a season I can understand the "unsustainable" talk, but 40 games is not that long.

I think I've brought this up, but has anyone heard of the saying by famed economist John Maynard Keynes "the markets can remain irrational longer than you can remain solvent"?

I know the markets are different as there's a "psychology" of the market in play. In class we've actually looked at the S&P 500 and it in the extremes it seems to not follow a normal distribution. But at the end of the day, I still believe that stats can remain irrational for small sample sizes.

 

[Doctor No]

I'll give it a shot.

What's happened has happened, and that's important, because the 40 games already played do count (This is obvious, but it bears repeating).

Suppose that a player's "true ability" (in something) is X, and that in his first 40 games, he performs at a higher level Y. In an 80-game season, we'd expect his end-of-season performance would be approximated by 40X + 40Y, which would be about halfway between his true ability and his first-half performance.

Over a 1000-game season, his end-of-season expectation would be 960X + 40Y, which (obviously) is much closer to his true performance.

With that said, in most things that are being measured, players' "true abilities" are very clustered, and so while Y might be an outlier, 40X + 40Y would be right back towards the middle of the cluster.

 

[SnowblindNYR]

I'll give it a shot.

What's happened has happened, and that's important, because the 40 games already played do count (This is obvious, but it bears repeating).

Suppose that a player's "true ability" (in something) is X, and that in his first 40 games, he performs at a higher level Y. In an 80-game season, we'd expect his end-of-season performance would be approximated by 40X + 40Y, which would be about halfway between his true ability and his first-half performance.

Over a 1000-game season, his end-of-season expectation would be 960X + 40Y, which (obviously) is much closer to his true performance.

With that said, in most things that are being measured, players' "true abilities" are very clustered, and so while Y might be an outlier, 40X + 40Y would be right back towards the middle of the cluster.

I guess that makes sense, but how do you know that the outlier is only 40 games. 40 games is an artificial cutoff. The player is more likely to regress than he is to continue, but it seems to me that an 80 game outlier is not exactly that much crazier than a 40 game outlier.

 

[Doctor No]

I guess that makes sense, but how do you know that the outlier is only 40 games. 40 games is an artificial cutoff. The player is more likely to regress than he is to continue, but it seems to me that an 80 game outlier is not exactly that much crazier than a 40 game outlier.

Well, 40 games is what's been played, so that's where that comes from in the example.

And the math holds whether or not the performance is an outlier or not. Supposing that the performance over the 40 games was the same as the true ability, then X = Y and 40X+40Y = 40Y+40Y = 80Y.

 

[Beatle17]

I'll give it a shot.

What's happened has happened, and that's important, because the 40 games already played do count (This is obvious, but it bears repeating).

Suppose that a player's "true ability" (in something) is X, and that in his first 40 games, he performs at a higher level Y. In an 80-game season, we'd expect his end-of-season performance would be approximated by 40X + 40Y, which would be about halfway between his true ability and his first-half performance.

Over a 1000-game season, his end-of-season expectation would be 960X + 40Y, which (obviously) is much closer to his true performance.

With that said, in most things that are being measured, players' "true abilities" are very clustered, and so while Y might be an outlier, 40X + 40Y would be right back towards the middle of the cluster.

Just a question for stats guys, who decides what a players "true ability" is? Shouldn't the stats from the 40 games tell you the true ability, or is it a number that you (not you specifically but just using your post for the question) decide is for a player because of your "bias"?

The issue that I personally have with the stats debates is everyone seems to have a bias, i.e. this team should be better, "we expect Calgary/Van to drop back to the norm". The current stats don't tell you that at all. When you track things like Corsi/Fenwick as possession I want a clearer definition of what "possession" is, because it is not shots. As an example from the '80s, Adam Oates carries the puck and sets up a play (possession of the puck for 10 seconds) then passes to Brett Hull who fires a shot (1/4 second on his stick). In today's Corsi/Fenwick talk Hull would have great possession numbers but did he really and Oates wouldn't because he never shoots the puck. I understand that he gets credited somewhat because he was on the ice but teams could get 4-5 shots against this pair and not score, but guys like Hull/Kurri/Bossy would need 1 shot to score thereby throwing the numbers off.

With the player tracking introduced in the Allstar game you may be able to tell who had actual possession of the puck.

 

[Doctor No]

Just a question for stats guys, who decides what a players "true ability" is? Shouldn't the stats from the 40 games tell you the true ability, or is it a number that you (not you specifically but just using your post for the question) decide is for a player because of your "bias"?

None of us decide what a player's true ability is (except for potentially the player himself). We're all just trying to observe it.

As far as "bias" (in quotes or otherwise) - we all have biases. You do. I do. Everyone posting in here, and everyone reading in here, has them. The key is to not let it influence your analyses - some are better at that than others.

A forty-game sample has far too much randomness in it. It's easier to exhibit with goaltenders (and I specialize in goaltenders), so I'll give you a goaltender example. Suppose that goaltender save percentages are binomially distributed (*), and a goaltender's "true ability" is a save percentage of 91%. Suppose that he plays 40 games, and faces 25 shots per game, or a total of 1,000 shots. Over the forty games, there's a 95% chance that our goaltender has an actual save percentage of between 89.2% and 92.8%. That's a pretty big range, wouldn't you say? (And one in twenty goaltenders would fall *outside* of that range).

When we say that a forty-game sample is too small to identify true levels of ability, that's what we mean.

(*) They're not, but the things that cause it to not be binomially distributed only serve to make save percentages fluctuate even more.

 

[Doctor No]

Pressing things further, take that 91% "true ability" goaltender, and have him play ten 40-game seasons. I just simulated ten seasons of 1,000 shots apiece (using a 91% chance to save each puck), and these were his save percentages in each simulated season:

91.3%
91.5%
90.6%
88.7%
91.9%
91.2%
90.6%
90.8%
92.4%
92.4%

Looking at those, would you believe those to all come from the same goaltender? Looks like at least one season where he's waiver wire material, and two seasons where he's in the Vezina talk. And it's all the same goaltender (with the exact same "true ability").

 

[charlie1]

None of us decide what a player's true ability is (except for potentially the player himself). We're all just trying to observe it.

As far as "bias" (in quotes or otherwise) - we all have biases. You do. I do. Everyone posting in here, and everyone reading in here, has them. The key is to not let it influence your analyses - some are better at that than others.

A forty-game sample has far too much randomness in it. It's easier to exhibit with goaltenders (and I specialize in goaltenders), so I'll give you a goaltender example. Suppose that goaltender save percentages are binomially distributed (*), and a goaltender's "true ability" is a save percentage of 91%. Suppose that he plays 40 games, and faces 25 shots per game, or a total of 1,000 shots. Over the forty games, there's a 95% chance that our goaltender has an actual save percentage of between 89.2% and 92.8%. That's a pretty big range, wouldn't you say? (And one in twenty goaltenders would fall *outside* of that range).

When we say that a forty-game sample is too small to identify true levels of ability, that's what we mean.

(*) They're not, but the things that cause it to not be binomially distributed only serve to make save percentages fluctuate even more.

Pressing things further, take that 91% "true ability" goaltender, and have him play ten 40-game seasons. I just simulated ten seasons of 1,000 shots apiece (using a 91% chance to save each puck), and these were his save percentages in each simulated season:

91.3%
91.5%
90.6%
88.7%
91.9%
91.2%
90.6%
90.8%
92.4%
92.4%

Looking at those, would you believe those to all come from the same goaltender? Looks like at least one season where he's waiver wire material, and two seasons where he's in the Vezina talk. And it's all the same goaltender (with the exact same "true ability").

Doc, this is so simple yet so important. I encourage you to make this into a short post and put it on your site so I can reference it repeatedly.

 

[West]

Just a question for stats guys, who decides what a players "true ability" is? Shouldn't the stats from the 40 games tell you the true ability, or is it a number that you (not you specifically but just using your post for the question) decide is for a player because of your "bias"?

I think the answer your looking for is the variance decides what's a good sample size. If something has a very low variance you don't really need a big sample size. If you looked a say goalies as a group and noticed that save % was very consistent you could look at a small number of games and figure out a good estimate.

I say this as someone who's very interested in Analytics in hockey but most of the measures we have aren't great so sample size needs to be pretty big. Some people might say there's a lot of randomness in hockey games but I honestly think it's the first thing.

The better the Analysis gets the smaller the sample size needs to be but there always going to be a fair bit of Randomness (Was player X truly not bad or was it because he kept playing while he had Mono/freak knee injury/etc).

 

[charlie1]

And if we make it more realistic the variance only increases:

"True sv%" likely varies from night to night ("that chinese food isn't sitting well tonight!") so slap a hyperdistribution on your [p] parameter in the binomial distribution.

Add auto-correlation to p as well since we know goalies go through confidence battles (and they battle injuries of course).

The opponents "true sh%" varies as well.

Etc, etc.

40 games is jack.

 

[West]

And if we make it more realistic the variance only increases:

"True sv%" likely varies from night to night ("that chinese food isn't sitting well tonight!") so slap a hyperdistribution on your [p] parameter in the binomial distribution.

Add auto-correlation to p as well since we know goalies go through confidence battles (and they battle injuries of course).

The opponents "true sh%" varies as well.

Etc, etc.

40 games is jack.

Don't be silly having a strong stomach is an important part of being a good goalie

Sorry your example made me laugh.

 

[Beatle17]

None of us decide what a player's true ability is (except for potentially the player himself). We're all just trying to observe it.


(*) They're not, but the things that cause it to not be binomially distributed only serve to make save percentages fluctuate even more.

Thank you for the excellent answer. I see your goaltender stuff all the time and find it interesting and hope for analysis of all players to reach that level.

 

[Doctor No]

Thanks, all!

Charlie, I like the idea of a short piece - I may even play with the legitimate tweaks that you describe (although I've found that the more complicated I make it, the harder it is to buy). I'll pull something together.

West - I played early this morning in goal after having nachos last night (with a total of about 120 jalapeno slices). I played okay, but it wasn't pretty.

 

[charlie1]

Thanks, all!

Charlie, I like the idea of a short piece - I may even play with the legitimate tweaks that you describe (although I've found that the more complicated I make it, the harder it is to buy). I'll pull something together.

West - I played early this morning in goal after having nachos last night (with a total of about 120 jalapeno slices). I played okay, but it wasn't pretty.

Nice.

And no please don't add those tweaks. The simplicity is the beauty, I just felt compelled to expand on it.

 

[Bps21*]

Minnesota is playing a lot better than their record and will be just fine. Chicago is too.

Buffalo is just as bad as they look. Even with the injuries Columbus is way outplaying them and are actually underperforming even with the setbacks.

It took Minnesota until the last possible second to prove me right...but they did it!

How did everyone else do?