Oilers, Leafs, Flames near the bottom, Sens near the top - Analytics report

[Master_Of_Districts]

The Leafs Fenwick from the year they finished 29th overall would be far better than last year despite last years team clearly being a lot better than the 29th placed team.

No one - or at least no one sensible - ever claimed that Fenwick was the only consideration.

 

[BrawlFan]

Until they include the "Don Cherry Thumbs Up" factor Toronto wont get any respect. It will also be the year that Ottawa finishes lower than every OHL team and blow up.

 

[Jerkini]

I'd be worried if we weren't the underdogs.

 

[Jyrki]

Toronto has plenty of room for improvement. Maybe the percentages went their way last season, but their underlying numbers should get a boost from all the young players coming into their own. Ditto for the Oilers, though they didn't get a lot of bounces.

It's interesting how the Senators still posted excellent possession numbers despite missing their three best skaters for most of the season. The Jack Adams went to the right coach for once.

 

[hatterson]

I haven't bothered to look too closely at this specific report, although at glance it appears to treat regression to the mean as the be all and end all of statistics which is a dubious approach at best.

I'd be interested to see this method applied to past years to see how good it is in the predictive sense.

 

[MarkGio]

Does this factor in changes in roster? If a team had too much luck in a previous year, but then has 3 different players the next season, would the parameters of the stats formula not change?

 

[hatterson]

Does this factor in changes in roster? If a team had too much luck in a previous year, but then has 3 different players the next season, would the parameters of the stats formula not change?

The article states:

To predict how a team will do in 2013-14, Vollman takes a second step, which is “to make adjustments for all the major roster changes, including mid-season trades, free agent signings, rookies and other players coming in from other leagues and, in some cases, the natural improvement or decline of particularly young or old teams.â€

However I haven't the slightest clue how he does that and I'm certainly not spending 12.95 to get a PDF of his book to hopefully figure it out.

 

[SmellOfVictory]

The Leafs Fenwick from the year they finished 29th overall would be far better than last year despite last years team clearly being a lot better than the 29th placed team.

Was that just straight Fenwick or was it Fenwick close? Score effects can be a pain in the junk. Not that there aren't other factors as well.

 

[Archangel]

I'm not sure whether this belongs in the Analytics section, but I thought it would be interesting to discuss among a bunch on non-analytics people.

Rob Vollman's hockey abstract went through a process of trying to take luck out of equation when predicting the standing for next year. When normalized for these factors, he suggested that the Oilers, Leafs, Sabres and Flames would finish at the bottom of the standings next year, and the Sens at the top.

http://blogs.edmontonjournal.com/20...ast-season-hockey-analytics-expert-concludes/

Whether you buy into this stuff or not, it is certainly interesting. I think the Sabres and Flames would not be up in arms if they finish near the bottom, but Oiler fans and Leaf fans would riot if the end up where this analysis predicts.

pretty much where the numbers I crunched had the teams mentioned. Problem wit the oilers is they have too much high end talent and not enough meat and spuds on the roster. The oilers weak spot remains toughness and grit.

 

[BraveCanadian]

How, exactly, does one take last season's results and use them to predict how teams will perform the next year?

Different Season, Different Teams, Different Training Regiments in the Off-Season, Different Schedule, Different Divisions.

The base variables have changed to such a drastic effect that I can't see how it would be effective.

Bingo.

Even if the personnel didn't change, they have all aged a year and practiced another year etc.

 

[Bank Shot]

How, exactly, does one take last season's results and use them to predict how teams will perform the next year?

Different Season, Different Teams, Different Training Regiments in the Off-Season, Different Schedule, Different Divisions.

The base variables have changed to such a drastic effect that I can't see how it would be effective.

I dunno. That's pretty much what anyone does when they make a prediction. It's pretty effective more often then not IMO. In the west for example there are 5 teams that have made the playoffs each of the last 5 seasons. I wouldn't bet against any of them to miss next year either.

 

[Hammer Time]

The Leafs Fenwick from the year they finished 29th overall would be far better than last year despite last years team clearly being a lot better than the 29th placed team.

Fenwick is a measurement of skater performance at even strength.

The year the Leafs finished 29th overall, they had Toskala as a starting goalie, their PP was ranked dead last, and their PK was ranked dead last. Last season, they had the much better James Reimer in net (him, along with Phaneuf, were probably the most valuable players on the Leafs), the #2 PK in the league thanks to Jay McClement, and their PP, while not great, was at least nowhere near last in the league.

What their Fenwick number suggests is that the Leafs have actually gotten worse in terms of how well their skaters played at even strength, but they've more than made up for it by improved special teams and goaltending.

 

[Trainspotter]

They do. They tend to defy averages based on providing a superior shot differential.

In terms of percentages, I don't have any citations on-hand, but there has been a strong correlation found between shot attempts (Fenwick, Corsi) and scoring chances, and there is very obviously a strong correlation between scoring chances and goals.

What's a "strong correlation" to you? The correlation of Fenwick/20 to Goals-For/20 hovers around 50%. More importantly, Fenwick has nothing like a "strong" correlation to regulation points over one season. An r^2 of 0.3 (i.e. correlates 30% of the time) is not strong unless you're desperate to sell your "advanced" model to the world. (http://hockeyanalysis.com/2013/02/27/the-declining-value-of-fenwickcorsi-with-increased-sample-size/)

On the defensive side, the correlation between Fenwick Against and Goals Against is abysmal. Add in the fact that there seems to be some serious bias in how shots, blocks etc. are counted rink to rink and Fenwick is a house built on sand.

The predictive value of Fenwick (i.e. using one season to predict Reg. points in the next) is even weaker than that. And for the sake of this conversation, it's important to note that it is a weaker predictor than past regulation points totals and GF totals, which aren't even strong predictors themselves. The author of the link in the OP should re-run his analysis giving Fenwick, GF and Reg Points their proper weight as predictors.

Finally, it seems a bit odd that we would admit that goals (the way hockey games are decided) are difficult to predict and rather than find ways to better predict goals, fall back on using a different stat that doesn't even reliably predict goals. If goals are hard to predict, counting something different doesn't change that fact. Aren't we just getting further away from what we want to measure?

So yes, as an earlier poster said, goals determine the winner. However, goals are variable based to a notable degree on luck (not majority, but notable) and thus very difficult to predict.The ability to outshoot, however, is based substantially less on luck and thus easier to predict.

The (over)reliance on the concept of "luck" by proponents of the new stats is a cop out and indicative of the paucity of their analytical tools. It's a placeholder for as-yet-unquantifiable/unquantified phenomena. Every time someone says something is down to "luck" you should read "something I can't explain." Or better yet, substitute "luck" for "The whims of the Man in the Moon". Reading their diatribes this way gives them their proper force.

Here's an example:

"So yes, as an earlier poster said, goals determine the winner. However, goals are variable based to a notable degree on something I can't explain (not majority, but notable) and thus very difficult to predict.The ability to outshoot, however, is based substantially less on something I can't explain and thus easier to predict."

BTW, in a recent attempt to account for "luck" in hockey by the author of the link in the OP, it was shown that many of the best teams in the game - including the Stanley Cup Champions - were amongst the "luckiest" teams in the league in 2012/13. Perhaps CHI weren't good but simply "lucky". If you go back in time you see an interesting pattern. Good teams are apparently often very lucky. BOS makes an appearance amongst the luckiest teams nearly every year in the data set. VAN has been the luckiest team in the league since 2008/9. To me, this suggests that there are likely a number of significant factors that are unaccounted for. The idea that reality consists of Fenwick + Luck is comic. (http://www.hockeyabstract.com/luck)

 

[Sypher04]

You realize the reason people use advanced stats is to limit the faults of human observation as much as possible, right? The argument your making is that a single non-expert observer (you) has done such a good job of viewing every single Leafs play of the past x number of games that this single non-expert observer provides better analysis of the games than indicators (shots and percentages) collected from every game by multiple people who are paid to do so.

Did you watch every second of the game intently? Did you never stop to talk to your buddy, eat some food, or drink some beer? Were you 100% sober for all of the games? Did you never let your mind wander to any other topic? Did you always follow the puck from end-to-end without watching what was happening on the bench or in the crowd? Did you keep track of the number of shots off the rush vs the number of dump-ins, or was it just a "feeling" you got from watching the games? Were you entirely unemotional while watching? Did you track the distance of each shot from the net?

Because those are just a chunk of the human factors that lead to faulty, garbage analysis of sports, especially fast-moving ones like hockey, and the reason that some people prefer numbers that, while imperfect, are better than a single non-expert observer (or a single observer of any kind).

In conclusion: hurr durr.

And maybe if the so-called "advanced statistics" weren't completely flawed, we'd actually care what they are trying to say.

 

[Master_Of_Districts]

What's a "strong correlation" to you? The correlation of Fenwick/20 to Goals-For/20 hovers around 50%. More importantly, Fenwick has nothing like a "strong" correlation to regulation points over one season. An r^2 of 0.3 (i.e. correlates 30% of the time) is not strong unless you're desperate to sell your "advanced" model to the world. (http://hockeyanalysis.com/2013/02/27/the-declining-value-of-fenwickcorsi-with-increased-sample-size/)

...

You know, there are often telltale signs that a person has no fundamental understanding of statistics.

Use of the phrase "correlates 30% of the time" - especially in relation to an r^2 value, which, of course, describes variance, and not the correlation between two variables - is one of them.

 

[Trainspotter]

You know, there are often telltale signs that a person has no fundamental understanding of statistics.

Use of the phrase "correlates 30% of the time" - especially in relation to an r^2 value, which, of course, describes variance, and not the correlation between two variables - is one of them.

There are telltale signs for when someone doesn't have a leg to stand on. Deflecting attention away from, or failing to argue against, someone's primary point in order to focus on semantics and/or a weaker argument is one of them. Turning toward arguing someone's qualifications is yet another.


My apologies for a poor turn of phrase in an attempt to simplify what we're talking about. Now onto the meat of the matter. Are you - with your superior knowledge of statistics - going to argue that an r^2 of 0.3 shows that Fenwick is a good predictor of Regulation Points?

 

[Master_Of_Districts]

There are telltale signs for when someone doesn't have a leg to stand on. Deflecting attention away from, or failing to argue against, someone's primary point in order to focus on semantics and/or a weaker argument is one of them. Turning toward arguing someone's qualifications is yet another.

My apologies for a poor turn of phrase in an attempt to simplify what we're talking about. Now onto the meat of the matter. Are you - with your superior knowledge of statistics - going to argue that an r^2 of 0.3 shows that Fenwick is a good predictor of Regulation Points?

The correlation between two variables over a particular sample - in this case, Fenwick percentage and regulation points - is technically irrelevant as to whether one variable predicts the other.

For example, PDO is highly correlated with points percentage over small samples (82 games or less, roughly). And yet PDO has very little predictive validity with regard to future team success.

So rather than looking at the correlation between Fenwick and team performance, the proper way to assess Fenwick's predictive validity would be to look at how well the metric predicts future success.

As an aside, you're correct that rink bias is a factor in shot recording. But rink bias is not directional - some recorders overcount or undercount, but no one seems to favor one team over the other. So rink bias is eliminated as a factor by looking at fenwick ratio/percentage, as opposed to considering fenwick for and fenwick against separately. Which everyone does, except you and David Johnson.

For samples of 82 games or less, Fenwick predicts future results more reliably than past results themselves do, although admittedly not by a lot - by the 40 game mark, the predictive validity is approximately 0.50 for Fenwick, and 0.40 for past results. The corollary of this is that, at any given point over the season, Fenwick will generally serve as a better measure of a team's underlying ability than its actual record. So unless you're prepared to go on a polemic against those that use W-L-T record as a surrogate measure of team strength - which, of course, is standard practice - then you should acknowledge the merits of Fenwick and move on.

So although Fenwick is not a great predictor of future results in an absolute sense, that's attributable to the random variation that inheres in a half-season sample, as evidenced by the fact that the predictive validity for past results is even weaker. When appropriately contextualized in this way, a co-efficient of 0.50 is relatively impressive.

Of course, an optimal predictive model would take into account both Fenwick and other relevant variables. Brian MacDonald created such a model, through incorporating hit differential as an additional variable - as it turns out, outhitting negatively predicts future success. Not surprisingly, MacDonald's model had more predictive power than using either Fenwick or past results alone.

 

[Bear of Bad News]

I want to take the time to remind everyone (on both sides of the argument):

Two additional rules apply (in addition to our site rules):

1. Respect what goes on here. If you can't do that, then feel free to ignore the forum entirely.
2. Be patient, and trust others' motives. What's basic to you may not be basic to me, and we're all in this together. Be helpful. Take opportunities to learn, and take opportunities to teach.

Remember that we're colleagues, not competitors. There's no reason that we need to be throwing around comments like "hurr durr" (just to pick one) - how does that help, exactly?

 

[Robik Karlner]

Nice to see the praised for once.

 

[Rants Mulliniks]

I'm not so sure I buy the "luck" he thinks exists in some of those categories.

 

[do0glas]

i think if wed stop using the word luck and instead say random variance, thered be less heat over analytics.

i think toronto serves as a great example, being 29th in fenwick close over 48 games but having 52% of your shots turn to goals (they were almost last in total shots for) is a great example of random variance. when you let 48 gp turn into 82, that percentage will regress and the total lack of shots on goal will begin to show in the standings.

just last season they were last in the league in shots for, at 48% Goals for, over 82 games they were 26th in the league.

 

[Trainspotter]

The correlation between two variables over a particular sample - in this case, Fenwick percentage and regulation points - is technically irrelevant as to whether one variable predicts the other.....etc., etc., ad nauseum....relatively impressive.

So, basically, despite copious hand waving your answer is "no". You can't really argue:

A) that an r^2 value of .3 shows a strong relationship between two variables.
B) Fenwick shows a strong relationship to regulation points in a single season.
C) Fenwick is a good way to predict future regulation points either within a season or season-to-season.

0.5 is not a strong relationship. You may find that a stat allowing one to predict less than half of future results is "relatively impressive", but I'm not so easily sold.

 

[MapleLeafistan*]

I can't wait until the end of next season when the Leafs are poised for the playoffs and are nowhere near the bottom. Infact I'm going to bookmark this topic and will periodically bring this topic from the dead throughout next season each time the Leafs smash the Sens...lol...it's funny how some people think hockey is baseball.

 

[Trainspotter]

I can't wait until the end of next season when the Leafs are poised for the playoffs and are nowhere near the bottom. Infact I'm going to bookmark this topic and will periodically bring this topic from the dead throughout next season each time the Leafs smash the Sens...lol...it's funny how some people think hockey is baseball.

The problem is that if the Leafs aren't in the playoffs confirmation bias will be rampant while the predictive validity of Fenwick remains weak. In fact, at least an equal proportion of any team's success or lack thereof will be accounted for by some other factor or combination of factors and The Whims of the Man in the Moon...ahem..."random variance" or whatever Bafflegab translation for "we don't have a clue how that happens" is fashionable at that point. But they'll still say, "we told you so".

 

[Morbo]

The problem is that if the Leafs aren't in the playoffs confirmation bias will be rampant while the predictive validity of Fenwick remains weak. In fact, at least an equal proportion of any team's success or lack thereof will be accounted for by some other factor or combination of factors and The Whims of the Man in the Moon...ahem..."random variance" or whatever Bafflegab translation for "we don't have a clue how that happens" is fashionable at that point. But they'll still say, "we told you so".

yeah, but saying the Leafs were lucky and will fail this year is fun and cool and none of your silly facts are going to change that.