HHOF Probability?

[Hockey Outsider]

That alone should send the method back to the drawing board

Of course, Peter Stastny is clearly worthy of the Hall of Fame. Rather than dismissing the model altogether, we can consider the blind spots it might have:

From what I can tell, the model is only looking at whether a player won a Stanley Cup (which Stastny never did). But it doesn't look at the player's level of performance. So Stastny's pretty strong playoff performances (105 points in 93 games) doesn't get captured.

Stastny was never a first- or second-team all-star. But he was blocked by some historically tough competition. Exclude Gretzky and Lemieux and he would have finished 1st once (1986), 2nd twice (1982 and 1983), and 3rd twice (1984 and 1988 - though with very few votes that year). Stastny was a consistent top three centre in the league for seven or eight years. We know that. The HOF voters know that. But a statistical model (that presumably only gives players credit for finishing in the top two - and doesn't take his historically strong competition into account) has a blind spot.

Plus the model can't give Stastny any credit for some important but subjective factors - ie being a pioneer (he was one of the first European stars in the NHL), or for being the best or maybe 2nd best player in Nordiques/Rockies history at the time he was inducted.

I think I understand how the model gave him such a low probability. I wouldn't dismiss the model entirely as a result, but this highlights some blind spots that (maybe) can be improved in a future edition.

 

[Bear of Bad News]

Adding to this (all of which I agree with), the model didn't give Stastny a 0% of HHOF induction. If a model gives a 20% chance of HHOF induction, you'd expect about two in ten (not zero in ten) to ultimately achieve induction.

Stastny's career was one of those two in ten.

 

[Hockey Outsider]

Adding to this (all of which I agree with), the model didn't give Stastny a 0% of HHOF induction. If a model gives a 20% chance of HHOF induction, you'd expect about two in ten (not zero in ten) to ultimately achieve induction.

Stastny's career was one of those two in ten.

A very good point. (Not to get into politics, but Nate Silver's model gave Donald Trump something like a 30% chance of winning the 2016 election. A lot of people criticized him for having a bad model, but it's not at all uncommon for something with "only" a 30% probability to happen. His model was much more favourable to Trump's chance than many other mainstream forecasts, many of which were giving Trump less than a 5% chance in the days leading up to the election).

Back to hockey - eyeballing the 50 players above and below Stastny, 9 of them are in the Hall, which is roughly what we'd expect based on the probabilities. (It's a bit messier than that as some of those players - like Newsy Lalonde and Eddie Gerard - spent much of their career outside the NHL. And some players are still active, and surely will get inducted - such as Henrik Lundqvist and Nicklas Backstrom). But, at a first glance, this suggests the probabilities make sense. If I were to make a model, I'd likely exclude the pre-1927 seasons, and all active players, in order to have a better definition for the population.

 

[Hockey Stathead]

Adding to this (all of which I agree with), the model didn't give Stastny a 0% of HHOF induction. If a model gives a 20% chance of HHOF induction, you'd expect about two in ten (not zero in ten) to ultimately achieve induction.

Stastny's career was one of those two in ten.

Exactly, 20% is not 0%. Can't say 20% is too low just because he ultimately was inducted.

Also, 20% propositions happen all the time. For example, 20% is around where NHL teams convert on the powerplay, as a whole. Is anyone *that* surprised when an average team scores in that situation?

 

[Steerpike]

Of course, Peter Stastny is clearly worthy of the Hall of Fame. Rather than dismissing the model altogether, we can consider the blind spots it might have:

From what I can tell, the model is only looking at whether a player won a Stanley Cup (which Stastny never did). But it doesn't look at the player's level of performance. So Stastny's pretty strong playoff performances (105 points in 93 games) doesn't get captured.

Stastny was never a first- or second-team all-star. But he was blocked by some historically tough competition. Exclude Gretzky and Lemieux and he would have finished 1st once (1986), 2nd twice (1982 and 1983), and 3rd twice (1984 and 1988 - though with very few votes that year). Stastny was a consistent top three centre in the league for seven or eight years. We know that. The HOF voters know that. But a statistical model (that presumably only gives players credit for finishing in the top two - and doesn't take his historically strong competition into account) has a blind spot.

Plus the model can't give Stastny any credit for some important but subjective factors - ie being a pioneer (he was one of the first European stars in the NHL), or for being the best or maybe 2nd best player in Nordiques/Rockies history at the time he was inducted.

I think I understand how the model gave him such a low probability. I wouldn't dismiss the model entirely as a result, but this highlights some blind spots that (maybe) can be improved in a future edition.

Adding to this (all of which I agree with), the model didn't give Stastny a 0% of HHOF induction. If a model gives a 20% chance of HHOF induction, you'd expect about two in ten (not zero in ten) to ultimately achieve induction.

Stastny's career was one of those two in ten.

Exactly, 20% is not 0%. Can't say 20% is too low just because he ultimately was inducted.

Also, 20% propositions happen all the time. For example, 20% is around where NHL teams convert on the powerplay, as a whole. Is anyone *that* surprised when an average team scores in that situation?

I think y'all are getting this slightly wrong.

The model is not even remotely correct in thinking Stastny had a 20% chance of making the HHOF.

Binary classification models don't ... really produce probabilities. They produce values between 0 and 1 which best optimize some loss function. These values are sort of like probabilities, but they aren't. You also know that the model isn't producing probabilities because the data to do that doesn't exist. The targets it's training on are "made HHOF" and "didn't make HHOF". Consider that the model will minimize its loss when it says that everyone who actually did make the hall had a 100% chance and everyone who didn't make the hall had a 0% chance.

It's not fair though to center your entire view of the model on this one datapoint. We're looking at Stastny specifically because he appears to be the most dubious of ALL of the model's predictions.