Are hockey fancystats predictive or merely evaluative?

[Cursed Lemon]

Simple question from someone who doesn't really dabble in them.

There has to be some demonstrable evidence to show that they have the ability to predict outcomes, if that is their purported use.

Click to read more...

 

[hatterson]

It all depends on what you mean by predictive.

It's partially addressed in our intro thread here and the thread linked there, although most of that discussion is around if it can actually be properly measured as opposed to the role of measuring.

Either way, the ultimate answer is that we will never get to a perfect prediction until hockey is played by robots and there's no such thing as luck.

However, it's fairly trivial to show that you can get a decent predictive model if you're willing to put up with less than 100% accuracy. If you look at games 1/3 of the way into the season and compile all of the teams Corsi/Fenwick data, using that alone you could predict who will end up in playoff spots with >50% accuracy. And that's just with simple counting. With more advanced models you could increase that chance significantly.

There's always going to be outliers and teams that don't follow the prediction for many reasons (among them may be pure luck), but you still end up with a predictive model.

 

[PenguinMario]

You might need to be a little more specific.

Some are persistent, i.e. they're likely to cluster around a certain value from year to the next. In order from most to least, corsi, then fenwick, then shots on goal, and then goal rates. That's largely a sample size issue - corsi has three components, fenwick has two, and shots on goal only one, and there are a hell of a lot more of any of them than there are goals. First half/second half season splits are more difficult to calculate, but from what I've seen the same holds true. Unlike shooting percentage, which has a central value but huge swings from month to month and season to season, shot rates and and ratios are relatively stable.

Shot events (corsi, fenwick, SOG) are also moderately predictive of goals - i.e., a team that regularly outshoots its opponents but doesn't outscore them is more likely to start outscoring than to start getting outshot. And the reverse holds true as well.

These aren't perfect, as hatterson says - corsi is most persistent of these stats, but it's also the least correlated with goal scoring. Shots on goal does best there, but it's also the least persistent. That's why you should usually supplement them with PDO - team shooting percentage plus team save percentage, which has a strong tendency to hover close to 100. Team A regularly outshoots its competition at even strength with a 55% edge, but scores only 45% of the goals with a PDO of 97? They're about to start doing a lot better.