2019/2020 Advanced Stats / Analytics Tracker

[vcv]

I didn't see a thread on this.

It's only 2 games into the season, but some of the trendy advanced stats numbers are looking good for the Sabres.

Looking at some of the numbers from Sean Tierney:

Tableau Public

Shot Rates, xG Rates, Shot Share, xG Share, "Reality vs. Expectation"... all of it looks good for Buffalo.

Again, 2 games, small sample size, but everything, so far, points in the direction of "this team is winning and it is sustainable."

Click to read more...

 

[vcv]

Shot Rates

 

[vcv]

Reality vs. Expectation

 

[SackTastic]

That "Reality vs Expectation" graph is a pristine example of how much failure there is in FancyStats these days.

"Make pretty graph without explaining your methodology or backtesting it for accuracy."

 

[vcv]

That "Reality vs Expectation" graph is a pristine example of how much failure there is in FancyStats these days.

"Make pretty graph without explaining your methodology or backtesting it for accuracy."

I assume he has more details on the methodology available. Don't know about backtesting.

My interest in creating this thread is to actually try to capture how well these models hold up for the Sabres. If these all say the Sabres are in fact good because they should be good, and not because they were lucky (like last year's win streak), will their season match that?

 

[Zman5778]

I assume he has more details on the methodology available. Don't know about backtesting.

My interest in creating this thread is to actually try to capture how well these models hold up for the Sabres. If these all say the Sabres are in fact good because they should be good, and not because they were lucky (like last year's win streak), will their season match that?

It'll be interesting, for sure. We've looked very impressive so far. Through 2 games.

Let's see how these graphs change as the season wears on

 

[Jim Bob]

That "Reality vs Expectation" graph is a pristine example of how much failure there is in FancyStats these days.

"Make pretty graph without explaining your methodology or backtesting it for accuracy."

Ummm, that graph seems pretty self explanatory to me.

It isn't a predictor of future results, it is a representation of how things have gone so far this season.

 

[jc17]

The charts use expected goals for and against based on shot volume and location. In theory since the sample size it larger, it tells how teams are playing more than just goals for and goals against which have tiny sample sizes meaning a fluke can have a huge impact.

Even using expected goals there can still be huge variation over a small amount of games.

Last year, 15 games into the season there was still about a 3% error from what teams had at that point in the season and what they had after 82. Even if 3% sounds small remember the spread between best and worst team is about 10%

 

[SackTastic]

Ummm, that graph seems pretty self explanatory to me.

It isn't a predictor of future results, it is a representation of how things have gone so far this season.

It's also not that either.

That graph is showing xgoals, which has been around for a few years now. Except nobody has ever done regression tests on it to see if the predictions it makes are accurate. ( Which is the flaw in so many FancyStats ; You can write 10 pages on why you think your model is great, but if you don't test to prove it, it's nothing but hot air.)

 

[jc17]

It's also not that either.

That graph is showing xgoals, which has been around for a few years now. Except nobody has ever done regression tests on it to see if the predictions it makes are accurate. ( Which is the flaw in so many FancyStats ; You can write 10 pages on why you think your model is great, but if you don't test to prove it, it's nothing but hot air.)

I agree and disagree.

There's definitely a rush lately for everyone to produce their own metric or visualization which can lack testing. However, if you look on many of the big names websites, they do show tests of the model, even when they aren't great.

That said, I still think xGF% is a decent stat regardless of predictive, or indicative power. Everyone can agree that shot volume and proximity to the net are important. Generally if you're playing well you'll do well in this metric. What makes this not a great prediction tool, or even snapshot of what has happened probably comes to be variability with goaltending and individual skater shooting skill. I'm fine using this to see how the team played, in general, but not a ton beyond that.

And again, even this stat, needs about 20+ games before it evens finds consistency within teams.

 

[Jim Bob]

It's also not that either.

That graph is showing xgoals, which has been around for a few years now. Except nobody has ever done regression tests on it to see if the predictions it makes are accurate. ( Which is the flaw in so many FancyStats ; You can write 10 pages on why you think your model is great, but if you don't test to prove it, it's nothing but hot air.)

xG isn't a predictive stat, though.

 

[joshjull]

xG isn't a predictive stat, though.

expected goals – Hockey Graphs

Expected goals models have been developed in a number of sports to better predict future performance. For sports like hockey and soccer where goals are inherently random and scarce, expected goals models proved to be particularly useful at predicting future scoring.

 

[Jim Bob]

expected goals – Hockey Graphs

Expected goals models have been developed in a number of sports to better predict future performance. For sports like hockey and soccer where goals are inherently random and scarce, expected goals models proved to be particularly useful at predicting future scoring.

It is a past performance stat that is better than other past performance stats at predicting future performance.

It's usage in this chart is about characterizing the past performance of teams. That can help predict future performance. But, the stat is about measuring the shot quality of past performance and not an estimate of how many goals the team will score in future games.

 

[joshjull]

It is a past performance stat that is better than other past performance stats at predicting future performance.

It's usage in this chart is about characterizing the past performance of teams. That can help predict future performance. But, the stat is about measuring the shot quality of past performance and not an estimate of how many goals the team will score in future games.

Its used as both*, not specific to that chart.

*A past performance stat and one to predict future performance.

 

[SackTastic]

I agree and disagree.

There's definitely a rush lately for everyone to produce their own metric or visualization which can lack testing. However, if you look on many of the big names websites, they do show tests of the model, even when they aren't great.

That said, I still think xGF% is a decent stat regardless of predictive, or indicative power. Everyone can agree that shot volume and proximity to the net are important. Generally if you're playing well you'll do well in this metric. What makes this not a great prediction tool, or even snapshot of what has happened probably comes to be variability with goaltending and individual skater shooting skill. I'm fine using this to see how the team played, in general, but not a ton beyond that.

And again, even this stat, needs about 20+ games before it evens finds consistency within teams.

I don't disagree with most of this.

 

[SackTastic]

Its used as both*, not specific to that chart.

*A past performance stat and one to predict future performance.

But even as a past performance gauge, it's not necessarily accurate.

Most xG models come up with their expected scoring percentage based on the aggregate of all shots from that distance taken since 07-08. There is going to be a wide variability from year to year on that (rules changes, player skill, etc), so it frequently does not reflect the true state of the game, and doesn't tell you what you think it does.

 

[jc17]

But even as a past performance gauge, it's not necessarily accurate.

Most xG models come up with their expected scoring percentage based on the aggregate of all shots from that distance taken since 07-08. There is going to be a wide variability from year to year on that (rules changes, player skill, etc), so it frequently does not reflect the true state of the game, and doesn't tell you what you think it does.

But I think few people take xG at face value for the number itself, and more as a relative stat. Like, its not "oh we should have had 3 goals and they should have scored 2" but more a number that allows you to see where a team gets its chances from.

Its used as both*, not specific to that chart.

*A past performance stat and one to predict future performance.

From corsica:

In its present condition, 5v5 xGF% is not a better predictor of future 5v5 GF% than CF% at the player level. Though descriptive of shot quality, the xG model has not yet shown to be appreciably predictive of future shot quality or goals at the on-ice level.13
xG does, however, have predictive value at the individual skater level. 5v5 ixG60 is a better predictor of future 5v5 G60 than G60 itself (0.152 for forwards and 0.128 for defence vs. 0.140 and 0.076 respectively).

Of course thats at the player level, and I thought I saw something similar on evolving hockey about the teams but cant find it.

Either way, its not that good of a prediction tool when the sample size is still small. Here's xGF% the first 20ish games of last season vs the rest of the season for all the teams:

and here's first 20 xGF% vs rest of the season actual GF%:

I still think its useful, but far from exact, especially over a couple of games

 

[Chainshot]

 

[jc17]

Manny's been reading this board

 

[Husko]

That "Reality vs Expectation" graph is a pristine example of how much failure there is in FancyStats these days.

"Make pretty graph without explaining your methodology or backtesting it for accuracy."

I don't know what's so bad about it? It's just comparing GF against fancy stat expected GF. Pretty simple.

 

[Husko]

So Okposo is... good?

 

[Fezzy126]

So Okposo is... good?

That line was good in a bad system (and by that line I mean Larsson+Girgs+[anyone], it's nice to see them immediately improve with better coaching.

The offensive-zone play has improved exponentially, there isn't an over reliance on low to high puck movement with point shots. Previously they would forecheck and hem in the opposition for long periods of time, but they insisted on staying along the boards and moving the puck back to the near side point man. All of their work rarely translated to real scoring chances. Now there is much more emphasis on cross ice puck movement and getting the puck to the high danger areas. I know it's only been a few games, but my god this has been refreshing after the last two coaches we were subjected to...

 

[Chainshot]

So Okposo is... good?

So far, very much so. That whole line has continued where they left off from last year.

That line was good in a bad system (and by that line I mean Larsson+Girgs+[anyone], it's nice to see them immediately improve with better coaching.

The offensive-zone play has improved exponentially, there isn't an over reliance on low to high puck movement with point shots. Previously they would forecheck and hem in the opposition for long periods of time, but they insisted on staying along the boards and moving the puck back to the near side point man. All of their work rarely translated to real scoring chances. Now there is much more emphasis on cross ice puck movement and getting the puck to the high danger areas. I know it's only been a few games, but my god this has been refreshing after the last two coaches we were subjected to...

Not only that, they're starting in their own zone akin to a "normal" d-zone specialist line (2 out of 3 times) rather than the Housley bizarre world (7 out of 8) AND the defense are doing a better job of moving the puck up to available options on the clear. We've seen stretch passing to Gus, we've seen a short pass to Kyle and Larry that has then lead to another in the neutral zone. It's sort of refreshing to see all the lines doing that but that was more of a hallmark of this trio who would come back responsibility (the starts) and then move up the ice as a unit.

 

[SackTastic]

I don't know what's so bad about it? It's just comparing GF against fancy stat expected GF. Pretty simple.

The number of goals scored is a quantifiable number. xGoals is a number that someone came up that they think means something, but it hasn't been proven that it does. So what are you getting out of comparing the two? I could create my own metric, xGoalsSack, and compare goals scored to that, and it would be just as meaningless.

Hockey is going through the same thing that baseball analytics did in the early days. Lots of people have ideas about how to combine certain statistics in a way that has meaning, but unless that idea is tested and validated, it's just an idea.

I realize that I likely harp on this more than most, but I spend on average about 5 hours a week reading and reviewing technical papers for 3 different organizations/conferences, so my brain is in a constant mode where I react harshly to claims made without evidence or validation. Unfortunately that's a lot of the current hockey "analytics".

 

[Husko]

The number of goals scored is a quantifiable number. xGoals is a number that someone came up that they think means something, but it hasn't been proven that it does. So what are you getting out of comparing the two? I could create my own metric, xGoalsSack, and compare goals scored to that, and it would be just as meaningless.

Hockey is going through the same thing that baseball analytics did in the early days. Lots of people have ideas about how to combine certain statistics in a way that has meaning, but unless that idea is tested and validated, it's just an idea.

I realize that I likely harp on this more than most, but I spend on average about 5 hours a week reading and reviewing technical papers for 3 different organizations/conferences, so my brain is in a constant mode where I react harshly to claims made without evidence or validation. Unfortunately that's a lot of the current hockey "analytics".

I get what you're saying. FWIW, without time to do a proper comparative analysis, there's this quick way of looking at it. He's getting his data from Evolving-Hockey. On their website, it's not a large sample size, but looking at all of last season. The top 10 teams in XG/60 are:

1. Carolina
2. Vegas
3. Toronto
4. San Jose
5. Montreal
6. Pittsburgh
7. Calgary
8. St. Louis
9. Tampa
10. Columbus

The actual top 10 were:

1. Tampa
2. Toronto
3. Washington
4. San Jose
5. Calgary
6. Montreal
7. Columbus
8. Pittsburgh
9. Chicago
10. Vegas

FWIW, Nashville, Ottawa, St. Louis, and Philly were all within .3 of #11.

So that's 7/10 of the top 10 in goals scored.

Without writing it out, they also got 7/10 of the bottom correct as well.

I don't know if that's good or not. But there's some context.