Against PDO

By Hunn · Nov 26, 2019 · Updated Nov 28, 2019
  1. Hunn Registered User

    Joined:
    Feb 23, 2017
    Messages:
    1,048
    Likes Received:
    677
    Trophy Points:
    109
    SB Cash:
    $ 200,000
    First off, let's all remind what PDO is:

    From Wikipedia
    In other words, PDO is the sum of shooting percentage (the number of goals scored divided by the number of shots for) and save percentage (the number of shots saved divided by the number of shots against). Usually, PDO is evaluated at 5-on-5, but for the discussion below it's irrelevant. PDO is a team statistics, but apparently ("a player" in the quote above) it can be evaluated for players too, I suppose similarly to +/−.

    Myth No 1
    PDO converges to 100


    [A side note: actually, I've seen references to the law of large numbers and even to the central limit theorem. From that it is claimed that this myth follows from mathematics. A half-educated nerd is worse than one hundred fools.]

    Imagine a league made of Team Canada and Team Kuwait (no disrespect, I just took the first team and the last team from the IIHF rankings) playing 82 games. Each game finishes 10:1, with shots 50:10. Team Canada PDO is 10/50 + 9/10 = 110. Team Kuwait PDO is 1/10 + 40/50 = 90.

    This is of course very generous, most probably it will be like 20:0 and 80:5, but you get the point: it is league-average PDO that "converges" to 100, not individual team PDO (league-average PDO does not even "converge" in layperson's terms – it's always equal to 100 by definition).

    In a real, competitive league like NHL, individual PDO converges to something between 98 and 102: in the last 6 seasons, only 14 teams out of 182 finished outside this range, roughly 8%.

    This very narrow range in PDO (root-mean-square deviation of 1.13 points over the last 6 seasons) gives the impression that any difference in PDO is simply due to statistical noise – i. e. "luck".

    Myth No 2
    PDO is a measure of "luck"


    From the discussion above one can see that PDO in NHL varies within about 4 points. Keep in mind that this difference is due to shooting and save percentages.

    4-point difference in SV is the difference between Vezina-level goaltending and a bad backup. Imagine an average team with Tuukka Rask in goal. Replace Rask with whoever is current Toronto backup. It's clear that PDO will go down because Rask is better than the Toronto backup, otherwise there is no point to pay them vastly different salaries.

    It is quality, not luck (apart from being "unlucky" to have a Toronto backup instead of Rask in first place). Good teams (like good goalies) have higher PDO (SV) and bad teams (like bad goalies) have lower PDO (SV).

    The same applies to shooting percentages.

    Myth No 3
    Teams with high PDO crush next season


    This myth highlights a very interesting fallacy, the one that statisticians call "confirmation bias": we look at high PDO teams that crush, but we disregard high PDO teams that DON'T crush.

    Let's look at the data for the last 6 seasons (excluding 2018-19 for obvious reasons):

    Top-5 PDO teams: 9 didn't make the playoffs next season, 16 made.

    For comparison:

    Top-5 CF% teams: 8 didn't make the playoffs next season, 17 made.

    On top of that, 7 out of 30 Top-5 CF% teams didn't even make the playoffs in the season they posted Top-5 CF% numbers (compare this with just 3 teams PDO-wise). Clearly, PDO is not much worse than the cherished Corsi in predicting postseason success.

    [A side note: based on intuitively clear definition of "postseason success", I calculated correlation coefficients for how far a team went in the postseason versus its regular season PDO and CF%. PDO appears to be a BETTER predictor than CF% (correlation coefficients of 0.414 and 0.339 respectively over the last 6 years) – but this result is for another discussion.]

    Is there any luck in PDO? Yes, in short term, there is some luck there too (as almost in any other number), but usually one is not required to parse through PDO to see that. For example, right now the Islanders are in a 15-1-2 point streak, and the Stars are in a 14-1-1 of their own, their PDO over that span must be huge (102.7 and 103.7 respectively, via nhl.com), but I don't need PDO to understand that close to 150-point pace is unsustainable for ANY team.

    So, let's give a rest to PDO as a measure of luck, leave it to uneducated folks like Lambert and Luszczyszyn.
     
  2. Sick Nuzuki Registered User

    Joined:
    Feb 24, 2018
    Messages:
    3,677
    Likes Received:
    2,786
    Trophy Points:
    102
    SB Cash:
    $ 200,000
    Gender:
    Male
    So yeah, you took a micro based argument and attempted to disprove it by explaining the macro.

    Like all things, it's a tool and one of many. It can be used to see which players are having good luck and bad luck to an extent short term but nobody claims it's the Bible of stats. Yes it can show someone is unlucky short term if used in context
     
    haveandare, Dack, Spazkat and 7 others like this.
  3. Hunn Registered User

    Joined:
    Feb 23, 2017
    Messages:
    1,048
    Likes Received:
    677
    Trophy Points:
    109
    SB Cash:
    $ 200,000
    Could you please elaborate what you mean by "micro" and "macro" (in this context), I'm lost?
     
  4. iCanada Registered User

    Joined:
    Feb 6, 2010
    Messages:
    9,940
    Likes Received:
    3,578
    Trophy Points:
    156
    SB Cash:
    $ 200,000
    Location:
    Edmonton
    I like this post, a fair bit.

    The one I always point out is this; who'd you rather have shooting, Ovechkin or the 13th forward on the last place club in the league?

    This is my large problem with NHL analytics; they are largely built on two assumptions that generally have held true for the entire dead puck era.

    1. All NHL players are roughly equal in skill, to the point that there are not vast skill disparities between different clubs or players. Thus:
    2. All shots and shot attempts are equal. Thus your shooting percentage is a function of luck.
    3. All saves and save attempts are created equally. This your save percentage is a function of luck.

    Generally speaking, I think the assumptions made are accurate enough that they are a good tool for looking at the game, because the league has generally been par enough that low level talent is being sucked out and replaced, and high level talent has been stifled so much due to clutch and grab. But in edge cases they really fall apart; Colby Cave for example is an analytics darling.

    And notably, as discussed by you above... Analytics has no way to account for the fact that McDrai are McDrai. Forgetting that they have better shots then the mean by a ridiculous margin in terms of release / accuracy / power, they also are just getting better opportunities then other players. Because they play with a player also so much better than other players, they have more open ice, they have an easier time being picky with chances and creating stronger opportunities. Any shot Drai takes is blessed with so many advantages compared to any shot Khaira takes, and it's not remotely close.

    Case in point... You look at a guy like Getzlaf and he's maintained a "lucky" PDO of 101.5 over his entire career. Is that because he's creating / preventing opportunities more than most and is better at scoring? Or does that mean his entire career he's been a lucky mother? What about Crosby's 102.4?

    I think to a certain degree, a lot of analytics value should be in finding edge cases to the above; analytics as we know it generally focuses on possession and generating chances, but good players cash on a greater percentage of chances, too. Sure, generally speaking better players generate more chances, also, but I think if you just focus on generation you're going the wrong way.... Ultimately I don't care how many chances to score someone has, I care how many times they score. Scoring chances don't win games, scoring does. If I had a guy who rarely generated chances but cashed on everything, I'd take him over a guy that generates a lot but never cashes.
     
  5. supsens Registered User

    Joined:
    Oct 6, 2013
    Messages:
    4,159
    Likes Received:
    978
    Trophy Points:
    109
    SB Cash:
    $ 200,000
    Because all stats are taken after the games not before, the next game does not have a PDO or corsi yet. They are just like keeping score and you can’t win them all
     
  6. aufheben #Panarin4Hart

    Joined:
    Jan 31, 2013
    Messages:
    46,087
    Likes Received:
    15,648
    Trophy Points:
    205
    SB Cash:
    $ 200,000
    Location:
    New Jersey
    It’s not just a team statistic.


    For instance Ryan Strome has a 108.4 PDO. I could assume he’s not going to score 82 P this year, but his current production alone doesn’t show me how lucky he’s been or why, or even if there’s luck involved at all. What if—despite his 82 P-pace—his PDO was 95.0 or his SH% was 5%? Maybe he’s not getting lucky, maybe he’s just taking twice as many shots this year.

    Teams, skaters, and goalies all go through hot/cold streaks. Yes you can often assume they’re going to end, but some of us just prefer having a clearer picture of how and why.
     
    Last edited: Nov 26, 2019
    meng666 and stopclickbait like this.
  7. tarheelhockey Highest Boss

    Joined:
    Feb 12, 2010
    Messages:
    64,827
    Likes Received:
    57,353
    Trophy Points:
    225
    SB Cash:
    $ 200,000
    Location:
    The Triangle
    I've been ranting against PDO for years.

    There are two severely flawed assumptions behind using PDO as a predictive stat:

    1) Shooting % and save % are connected
    2) There is no such thing as high/low quality play, just lucky/unlucky variation

    Either of these should be considered fatal. To have both combined into one stat is just silly.

    A final point: much like +/-, the last resort in defending PDO is usually along the lines of: "Yeah but if you see an extreme outlier, it probably means something".
    And my response to this is the same as to +/-: "You can find that outlier more easily using the original stats that make up PDO, than by using PDO itself".
    If a team is shooting 25%, that tells you all you need to know about unsustainable shooting. There's no point looking at their PDO, then backtracking to figure out which element of that formula is wildly out of whack.
     
  8. Hunn Registered User

    Joined:
    Feb 23, 2017
    Messages:
    1,048
    Likes Received:
    677
    Trophy Points:
    109
    SB Cash:
    $ 200,000
    It is not indeed, I indicated that explicitly that there is also PDO for players. My point is to criticise the 3 "myths" about individual team PDO: (1) it converges to 100, (2) it's luck, (3) high PDO = overachievement. The first one is false, whereas (2) and (3) are somewhat true, but over a short-term period only, like a point streak, or a lull.
     
  9. deckercky Registered User

    Joined:
    Oct 27, 2010
    Messages:
    8,089
    Likes Received:
    1,162
    Trophy Points:
    109
    SB Cash:
    $ 200,000
    PDO, like any other statistic, is useful in proper context. Using fake theoretical counterexamples is silly - do the groundwork and find practical counterexamples.

    Worth noting that your assumptions are wrong. There are goaltenders who will naturally have above average save percentages and shooters who will naturally have above average shooting percentages, but even those players have variances (shorthanded save percentages, for example, is highly variable).

    Not all of PDO is luck - some of it comes down to usage. For example, Oates changes how Ovechkin is used and his shooting percentage plummets - there's a non-luck factor that lowered Ovechkin's PDO. Playing injured is another non-luck factor that will drop PDO.
     
    TychoFan likes this.
  10. Nick Hansen Registered User

    Joined:
    Sep 28, 2017
    Messages:
    2,564
    Likes Received:
    1,844
    Trophy Points:
    101
    SB Cash:
    $ 200,000
    Gender:
    Male
    *like*

    Always rubbed me the wrong way.
     
  11. Hunn Registered User

    Joined:
    Feb 23, 2017
    Messages:
    1,048
    Likes Received:
    677
    Trophy Points:
    109
    SB Cash:
    $ 200,000
    Exactly! One does not need PDO to see that 14-1-1 streak is ... a streak, it can't be maintained over the whole season. Why then rely on PDO?
     
    tarheelhockey likes this.
  12. SnuggaRUDE Registered User

    Joined:
    Apr 5, 2013
    Messages:
    5,572
    Likes Received:
    2,544
    Trophy Points:
    126
    SB Cash:
    $ 200,000
    A timely observation.
     
  13. Finnish your Czech J'aime Les offres hostiles

    Joined:
    Nov 25, 2009
    Messages:
    64,278
    Likes Received:
    1,635
    Trophy Points:
    185
    SB Cash:
    $ 200,000
    Occupation:
    Sometimes
    Location:
    Toronto
    I agree with the content of this post, but I don't think you're revealing anything crazy. Yes, you can't use PDO to make an overarching opinion on something/someone. But that's the same for every stat. PDO has it's uses.
     
    Pip and Garbage Goal like this.
  14. Hunn Registered User

    Joined:
    Feb 23, 2017
    Messages:
    1,048
    Likes Received:
    677
    Trophy Points:
    109
    SB Cash:
    $ 200,000
    The only fake theoretical counterexample is Team Canada vs Team Kuwait. Rask vs Sparks is sadly quite real one.

    This is what I am talking about.

    Exactly so. It also depends on style. PDO in recent years was dominated by teams like Washington much the same way CF% was dominated by LA 5-10 years ago.
     
  15. aufheben #Panarin4Hart

    Joined:
    Jan 31, 2013
    Messages:
    46,087
    Likes Received:
    15,648
    Trophy Points:
    205
    SB Cash:
    $ 200,000
    Location:
    New Jersey
    Well for one thing this isn’t always apparent to everyone. Lots of people were trashing the Dallas Stars in October, saying they called it during the off-season and so on. This happens all the time.

    Beyond the yes/no, people like to know how and why, to what degree can a streak not be maintained.
     
    stopclickbait and Aceboogie like this.
  16. Aceboogie Registered User

    Joined:
    Aug 25, 2012
    Messages:
    31,571
    Likes Received:
    2,475
    Trophy Points:
    156
    SB Cash:
    $ 200,000
    Meh, your railing against something in which noone is on the other side to argue. Noone with an appetite for "advanced stats" is going to the on the other side of the arguement.

    There are a ton of great teams who have maintain high PDOs, because they have good goalies (sv%) or shooters (sh%)

    PDO is still relevant for predicting future success. Rarely have bottom 10 CF%, top 10 PDO teams maintained consistent success.

    I find PD) is just scary because people dont understand it, nor what to. But if you break it down to its components (shooting %, save %), its very intuitive. A career,3rd line, 6% sh % player whos scores 35 goals shooting 18% one season is likely to regress in goals next season. Vs a career 15% career player wont, because hes a better shooter
     
  17. Aceboogie Registered User

    Joined:
    Aug 25, 2012
    Messages:
    31,571
    Likes Received:
    2,475
    Trophy Points:
    156
    SB Cash:
    $ 200,000
    Bingo.

    I have yet to see any PDO ultra defenders saying its the be all and end all. Its a useful tool though
     
  18. Bounces R Way Feaster's Beasters

    Joined:
    Nov 18, 2013
    Messages:
    13,668
    Likes Received:
    13,212
    Trophy Points:
    166
    SB Cash:
    $ 200,000
    Location:
    Lindholmsville
    I've always found the suggestion that every shooting and save percentage will inevitably regress or progress towards some arbitrary mean pretty ridiculous. Some players are better than other players, some teams are better than other teams. That doesn't make PDO worthless or anything.

    The world is not simply an unending sequence of 1's and 0's and neither is hockey.
     
    TychoFan and CheckingLineCenter like this.
  19. iCanada Registered User

    Joined:
    Feb 6, 2010
    Messages:
    9,940
    Likes Received:
    3,578
    Trophy Points:
    156
    SB Cash:
    $ 200,000
    Location:
    Edmonton
    The good old gamblers fallacy. Clearly if I flip a coin after getting three tails, there is less than 50% chance that next coin is Tails, so bet Heads, baby.

    Or as I like to call it... "The HF regression to the mean" fallacy.
     
    Hunn likes this.
  20. ijuka Registered User

    Joined:
    May 14, 2016
    Messages:
    16,126
    Likes Received:
    6,548
    Trophy Points:
    141
    SB Cash:
    $ 200,000
    Yeah, it's interesting - PDO's a team's goalie performance in relation to shooting performance, it's not really a luck stat. Also, even if PDO was only about luck, it still wouldn't regress to 100. Say you're starting at 115 due to a fortunate early season performance. For the rest of the season, you're predicted to be performing at 100 PDO. How you did in the past doesn't affect that. As such, it wouldn't be regressing towards the mean anyway.
    I actually tested these things when I was a kid haha, flip the coin until I get heads or tails 4 times in a row, then flip it 200 times in a row. Then compare to what happens if I just flip it 200 times without doing anything specific before. Interestingly enough, zero correlation found.
     
    iCanada likes this.
  21. kladorf2005 Registered User

    Joined:
    Apr 20, 2018
    Messages:
    1,181
    Likes Received:
    1,228
    Trophy Points:
    85
    SB Cash:
    $ 200,000
    Maybe I'm not following your argument, but if 16/25 (high PDO) teams made the playoffs, isn't that better than 16/31 (random) teams that make the playoffs?

    i.e. higher PDO = higher chance of making the playoffs? 64% vs 52%?
     
  22. TheDoldrums Registered User

    Joined:
    May 3, 2016
    Messages:
    6,482
    Likes Received:
    6,133
    Trophy Points:
    111
    SB Cash:
    $ 200,000
    Location:
    Kanada
    Most of the time people criticize advanced stats it's because they're using it out of context or expecting it to explain everything.

    PDO says nothing about a high/low quality play. It assumes nothing. Maybe people use it to form their own assumptions, but that's a human problem.
     
  23. TheOtherOne Registered User

    Joined:
    Jan 2, 2010
    Messages:
    5,442
    Likes Received:
    1,248
    Trophy Points:
    154
    SB Cash:
    $ 200,000
    PDO is weird but OP's argument is also weird.

    This is more or less what I was thinking of saying:
    To reiterate: Why in the world would you add together two unlike things? This is like saying 3 kilometers + 4 grams = 7 squirrels.

    If you have a goalie who's letting everything in all season (as a Red Wings fan I would know a lot about this), PDO goes down. What does that have to do with anything? Why would you make that a team stat? Why would you relate it to shooting %?

    If you have a player who's very good at picking his shots, PDO goes up for him and his linemates. Okay? What does that have to do with the team's save %?

    If one player does a very good job of clearing the crease but has zero offense, and another player cherrypicks all game and snipes well, they both get the same PDO? That just seems really awkward and useless.
     
    tarheelhockey likes this.
  24. Kamiccolo Truly wonderful, the mind of a child is.

    Joined:
    Aug 30, 2011
    Messages:
    26,638
    Likes Received:
    15,933
    Trophy Points:
    177
    SB Cash:
    $ 100,001
    Occupation:
    IT
    Location:
    Undisclosed research facility
    Disagree.

    1) PDO is meant to show how sustainable a teams play is. No different than when a player goes on a hot streak and shoots 40%, you know eventually it will even out. Look no further than Buffalo, they were a high PDO team through the first 10 games and then regressed back to 100 over the past 12.

    2). Sure there is. A shot from the side of the net from along the boards that has almost 0 chance to go in shouldn't be counted the same as someone right in the slot taking a shot.

    I would argue PDO is better than it ever has been with parity what it is. In general, teams are going to score generally the same and have goaltending generally the same. Over an 82 game season these things usually even out, but where PDO shines is even bigger sample sizes than that.

    In fact, using PDO is almost always a regression litmus test for teams who unexpectedly shot up the standings.
     
    ZeroPucksGiven and TychoFan like this.
  25. Machinehead CHAMP19NS

    Joined:
    Jan 21, 2011
    Messages:
    105,054
    Likes Received:
    35,798
    Trophy Points:
    156
    SB Cash:
    $ 200,000
    Occupation:
    Professor
    Location:
    NYC
    Nobody is saying there's no such thing as quality. When Holtby was at his peak, a high-scoring team like the Caps could easily hit 103.

    It becomes a luck issue when your PDO is egregious. What constitutes egregious depends on your roster.

    Everyone was surprised at what the 8th seeded Kings did in 2012 but they had a PDO below 99 with a multi-Cup roster.
     
    stopclickbait likes this.

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
monitoring_string = "358c248ada348a047a4b9bb27a146148"