New to Advanced NHL Stats

[Master_Of_Districts]

But considering PDO purports to be a measure of luck, isn't this basically acknowledging that it doesn't show the one and only thing that it's supposed to?

The statistic itself is what it is - it doesn't purport to measure anything.

Some have asserted that PDO is exclusively a measure of luck. But those people were and are wrong. And the person who conceived of the statistic - Vic Ferrari - never made that assertion.

Over small sample sizes - effectively, anything equivalent to an NHL regular season or less - most of the variation in PDO is attributable to luck. And that's true for both teams and individual players.

So to use PDO as a crude measure of luck is not necessarily incorrect or inappropriate. At the absolute least, it's better than taking counting numbers at face value. More nuanced approaches are, of course, preferable. But aside from myself and a few others, no one has the inclination or wherewithal to do that.

 

[Brainiac]

As I wrote before, the problem with all those stats is that hockey is, in its very nature, a chaotic game. A bunch of micro-events, each with a somewhat unpredictable outcome, and then all stiched together.

-The puck is small, travels very fast and is less regular than a ball
-Players move fast and have very little time to react
-There is very little 'memory' or 'inertia' involved. One bounce by the puck can change a PP threat into a breakaway and a goal for the other team. All within 4 or 5 seconds.

At least they are shooting 40-50 times each game, with around 25-30 shots making it to the net. So that is OK with stats.

So yeah, I wouldn't throw away 'advanced' stats just by princple. But I wouldn't make management moves based on that alone either.

From this, I formulate two interesting observations:

1- An actual descriptive simulation (i.e. the NHL video games) may, at some point in the future, achieve better predictive capability than stats.

2- The only other sports I can think of as being as chaotic as hockey are fighting sports such as boxing or MMA. And a lot of people like both hockey and fighting...

 

[tarheelhockey]

The statistic itself is what it is - it doesn't purport to measure anything.

Some have asserted that PDO is exclusively a measure of luck. But those people were and are wrong.

Seems to me you're dealing in semantics here.

The fundamental concept behind PDO is that individual and team PDOs will always regress to 1000, so we measure distance from 1000 to account for short-term variance.

The problem is that there's not a logical reason why a specific player's or team's PDO should regress to 1000. It's a completely arbitrary assertion once you get past the bird's eye view. What the stat essentially does is take authentically predictable results and dismiss them as random variation. That's... a slight problem.

And the person who conceived of the statistic - Vic Ferrari - never made that assertion.

First, Vic Ferrari didn't conceive of the statistic. A visitor on his blog named Brian King did.

Second -- here is a direct quote from Vic Ferrari's first post on the subject:

Lets pretend there was a stat called “blind luck.†Said stat was simply adding SH% and SV% together. I know there’s a way to check what this number should generally be, but I hate math so lets just say 100% for ***** and giggles.

 

[Master_Of_Districts]

Seems to me you're dealing in semantics here.

The fundamental concept behind PDO is that individual and team PDOs will always regress to 1000, so we measure distance from 1000 to account for short-term variance.

No - that's not the fundamental concept behind PDO.

The problem is that there's not a logical reason why a specific player's or team's PDO should regress to 1000. It's a completely arbitrary assertion once you get past the bird's eye view.

Sure.

But once again - anyone who made that claim was and continues to be wrong.

What the stat essentially does is take authentically predictable results and dismiss them as random variation. That's... a slight problem.

And once again - anyone who claims that 100% of the variation in PDO (whether at the team or player level) is attributable to randomness is wrong.

I can only belabor the point so many times.

First, Vic Ferrari didn't conceive of the statistic. A visitor on his blog named Brian King did.

Not to detract from what King did, but it seems to me that he merely posted the EV SH% + SV% figures for some of the Oilers' players during the 2007-08 season.

It was Vic who ran with and popularized the concept itself.

Vic ought to be credited.

Second -- here is a direct quote from Vic Ferrari's first post on the subject:

Except Vic never said that - a commenter at his blog did.

Hell - in the very next comment, Vic remarks that the SH% component of PDO is repeatable across independent samples (at the player level). Which is, of course, correct.

So you can remove that condescending icon.

 

[TheDevilMadeMe]

No - that's not the fundamental concept behind PDO..

So what is the fundamental concept behind PDO? What is it actually useful for?

 

[Hemsky is a gangsta]

PDO doesn't regress to 1000. It regresses to the true talent level of the team. It is possible to ice a line up that is so bad it has less than a 1000 PDO.

 

[Hemsky is a gangsta]

So what is the fundamental concept behind PDO? What is it actually useful for?

You look at it and ask yourself if it's reasonably sustainable. At least that's how I use it in my hockey pools!

Check out this article: http://vansunsportsblogs.com/2011/11/04/drance-numbers-an-interview-with-brian-king-inventor-of-pdo/

 

[Bear of Bad News]

PDO doesn't regress to 1000. It regresses to the true talent level of the team. It is possible to ice a line up that is so bad it has less than a 1000 PDO.

I would argue that it's more than a theoretical construct; I'd expect about half of the teams to have a "true" PDO of less than 1000.

 

[Master_Of_Districts]

So what is the fundamental concept behind PDO? What is it actually useful for?

Given that PDO, by definition, is the sum of (even strength) SH% and SV%, I suppose it serves as a measure of the extent to which any given team or player has benefited (or been hurt by) the percentages.

So it's not necessarily useful for anything.

Then again, the majority of hockey statistics would have a comparable level of utility.

 

[Bear of Bad News]

Given that PDO, by definition, is the sum of (even strength) SH% and SV%, I suppose it serves as a measure of the extent to which any given team or player has benefited (or been hurt by) the percentages.

Can you say more about this? Like a real example? I'm still a bit confused about PDO's utility.

 

[matnor]

I haven't read much about PDO but I would be really careful about drawing strong conclusions based on it. The thing about PDO, or for that matter any statistic that is composed of a random component (which can be called "luck") and a non-random component ("skill"), is that

1) on average, the higher the PDO-statistic for a player is one season the more likely it is to regress to a lower PDO next season (regression-to-the-mean), and

2) the higher PDO a player has, the more likely it is that that player will have a high PDO next season.

These two statements are not in conflict with each other. But as I said, the same is true for virtually any stat, even the traditional ones such as goals and assists. If a stat is based more on "luck" than "skill" the regression-to-the-mean effect is dominant while if it is highly dependent on skill than the second effect will be more pronounced. I would assume that the reason that PDO is often used as an indicator of luck is that some people think that shooting percentage and save percentage are more likely to be dependent on luck whereas shots taken and allowed are more dependent on skill. I suppose that is possible but to me it is not obvious. Some teams and players play a style where they take tons of shots while others are more likely to play themselves into a good position before they shoot. I would assume the players on the old Soviet teams had ridiculously high PDOs.

 

[Master_Of_Districts]

Can you say more about this? Like a real example? I'm still a bit confused about PDO's utility.

If Team A had a PDO of, say, 1030, that would tell you that the team had benefited significantly from the percentages at even strength.

Of course, the exact same thing could be ascertained from looking at the team's EV SH% and SV% in conjunction with one another.

Which relates back to my point about it's utility being somewhat limited.

 

[Bear of Bad News]

If Team A had a PDO of, say, 1030, that would tell you that the team had benefited significantly from the percentages at even strength.

Couldn't it also be evidence that 1030 was their "true" level?

 

[Master_Of_Districts]

Couldn't it also be evidence that 1030 was their "true" level?

Well - when I say "benefitted significantly", I was referring to the effects of both talent and luck.

For what it's worth, if a team has a PDO number of 1030 after a full NHL regular season, our best guess is that the team's true talent PDO number is about 1012 (if we have no other information). This follows from the fact that after 82 games, the variation in PDO across teams is approximately 40% talent, 60% non-talent.

 

[hatterson]

Well - when I say "benefitted significantly", I was referring to the effects of both talent and luck.

For what it's worth, if a team has a PDO number of 1030 after a full NHL regular season, our best guess is that the team's true talent PDO number is about 1012 (if we have no other information). This follows from the fact that after 82 games, the variation in PDO across teams is approximately 40% talent, 60% non-talent.

I'm curious where the evidence on this is. Also, what is the variation on individual player PDO?

 

[Brainiac]

Well - when I say "benefitted significantly", I was referring to the effects of both talent and luck.

For what it's worth, if a team has a PDO number of 1030 after a full NHL regular season, our best guess is that the team's true talent PDO number is about 1012 (if we have no other information). This follows from the fact that after 82 games, the variation in PDO across teams is approximately 40% talent, 60% non-talent.

What? Where do you get that kind of numbers?

After 82 games, the variation in PDO is 40% talent, 60% luck? How does that make sense? What's the 'variation' after 41 games? Or after 5 seasons?

 

[PDO]

All PDO measures is on-ice SH% + SV%. Feel free to draw your own conclusions. Trust me, I'd know.

I don't think it has any value anywhere but 5v5. I think there are outliers, but I think we can generally figure out who they are. Hands up anyone who thinks Dupuis and Kunitz have the same PDO on Edmonton?

I think it's most useful for defensemen as they have the least ability to drive the percentages.

 

[Master_Of_Districts]

What? Where do you get that kind of numbers?

I "got" *those* "kind of numbers" by actually performing the relevant calculation.

After 82 games, the variation in PDO is 40% talent, 60% luck? How does that make sense?

For the regular seasons that occurred between 2003-04 to 2010-11 (I haven't updated my spreadsheets in a while) the variation in team PDO was, for the average season during that timeframe, 39% talent, and 61% luck.

What's the 'variation' after 41 games? Or after 5 seasons?

I haven't calculated that.

Maybe you could do it though.

 

[Warden of the North]

I'm glad to see this conversation about PDO. That stat bothers the hell out of me for the exact reasons being expressed here.

Me too, especially at a team level. Statements like "over a few seasons PDO will go to 1000" are ridiculous. The team roster will be vastly different over that timeframe, of course its going to change.

Also, the expectation that a team with a high PDO will eventually regress. Well so what? If a team wins, who cares whether they were supposed to win or not. Every team will eventually regress, over the course of many seasons, bringing me back to roster turnover.

 

[hatterson]

I "got" *those* "kind of numbers" by actually performing the relevant calculation.



For the regular seasons that occurred between 2003-04 to 2010-11 (I haven't updated my spreadsheets in a while) the variation in team PDO was, for the average season during that timeframe, 39% talent, and 61% luck.



I haven't calculated that.

Maybe you could do it though.

How did you separate talent and luck? Specifically what calculations did you use?

 

[Perreault]

Back to the original topic...

I am new to advanced NHL stats and a bit overwhelmed.

Can of you recommend the best websites that give a comprehensive overview and the top oh three or five stats?
I saw the first thread but it was overwhelming.

The sites listed in [thread=1237263]Analytics Resources and Glossary[/thread] are great, but I can't find an overview that explains them from "the ground up." Is there somewhere on one of those sites or somewhere else that is good background for someone new to these stats?

Most common stats you might see are ...

That was good for under 200 words, but I want more.

 

[Master_Of_Districts]

Specifically what calculations did you use?

Variance (Luck) = Variance (Predicted) / Variance (Observed)

Variance (Predicted) = The PDO variance that would be observed between teams if there was no talent component.

 

[GuineaPig]

Couldn't it also be evidence that 1030 was their "true" level?

It would take a very, very, very great team to have a true talent level of 1030. We're talking about goaltending that averages a 0.930 ES SV% (that's elite talent) and a 10% shooting percentage (well above league-average). Really, it would probably be more sustainable if it was the save percentage driving it, because it's been shown that goaltenders are much better at repeating above- or below-average percentages than teams are at sustaining odd shooting percentages.

EDIT: See this piece for examples of regression from abnormally high/low PDOs.

 

[Master_Of_Districts]

It would take a very, very, very great team to have a true talent level of 1030. We're talking about goaltending that averages a 0.930 ES SV% (that's elite talent) and a 10% shooting percentage (well above league-average). Really, it would probably be more sustainable if it was the save percentage driving it, because it's been shown that goaltenders are much better at repeating above- or below-average percentages than teams are at sustaining odd shooting percentages.

EDIT: See this piece for examples of regression from abnormally high/low PDOs.

Yeah - all good points.

And just to expand, the standard deviation in PDO *talent* among teams is about 7.5. So a true talent PDO number of 1030 would be four standard deviations above the mean. Like you said - reasonably unlikely in a 30 team league.

 

[GuineaPig]

Just as an example, here are the PDO numbers (which might be skewed because of no access to 5v5 numbers) of the dynasty Oilers, from 83-84 to 87-88:

1054
1042
1050
1039
1035

Now, clearly those teams weren't going to regress to 1000. But they're major outliers in terms of skill and how far away they were in quality compared to the rest of the team. They were so good that even if one were to calculate their Fenwick close it would be completely irrelevant because they could actually severely improve their average shot quality.

But we simply don't see teams like that anymore. Sidney Crosby or Henrik Sedin might be able to elevate the shot quality of their linemates, but not significantly enough to transform their team's shooting %. And certainly no team has the depth or star power to spread this around their entire line-up.