Total Hockey Rating (THoR): A comprehensive statistical rating of NHL Players

[XX]

Can't account for system differences, situational differences (playing with the lead vs protecting one) or likely even positional differences between forwards. These measurements are never very meaningful. I think trying to boil hockey down to 1 rating is missing the point. In baseball, every pitch may as well be an entire game, statistically speaking, so numbers work well. Hockey is much too fluid for accurate or all-encompassing numbers.

 

[number72]

Any ranking that doesn't have Kessel and Phaneuf in the top 10 in the NHL is not credible

 

[fedfed]

Really, it looks like a stat for the sake of stat just because it comes nowhere close to reflecting what's happening. In general, stats should give predictable results: Malkin IS better than Kennedy, Kopitar IS better than Steen. If stat doesn't give such results, it can't be trusted to be a judge in a closer comparisons and last checked, that's what we need from stats.

 

[pdd]

Any ranking that doesn't have Kessel and Phaneuf in the top 10 in the NHL is not credible

You're referring to one that's limited to current Maple Leafs, right?

 

[ChiTownPhilly]

One of the hockey related research papers is "Total Hockey Rating (THoR): A comprehensive statistical rating of National Hockey League forwards and defensemen based upon all (sic) on-ice events" - and is a very interesting look at valuing players.

"Here we present a new comprehensive rating that accounts for other players on the ice will a give player as well as the impact of where a shift starts and of every non-shooting events such as (emphasis mine) turnovers and hits that occur when a player is on the ice.

I recognize the legal maxim that states "hard cases make bad law," but let me give you an example of one of the most memorable plays I've ever seen in person- and postulate that the metric described above has little way of taking the contribution into account.

Philadelphia v. up-and-coming Ottawa team. Legion of Doom line. Renberg carries in puck from R Wing. Lindros gets deep into the zone, and angles to L side, then cuts behind the net as if preparing to start a cycle. Two defenders give complete attention to Lindros, which leaves Renberg partially open, and John LeClair completely open. Cross-ice pass/score.

Now Lindros had more to do with that goal than any player... and he didn't even touch the puck...

I for Incomplete on the measurement attempt as given above. Needs to return to the shed for some overhaul work---

 

[BoHorvatFan]

Can't account for system differences, situational differences (playing with the lead vs protecting one) or likely even positional differences between forwards. These measurements are never very meaningful. I think trying to boil hockey down to 1 rating is missing the point. In baseball, every pitch may as well be an entire game, statistically speaking, so numbers work well. Hockey is much too fluid for accurate or all-encompassing numbers.

It won't stop people with too much free time from trying to create ratings that are seriously flawed and tell you nothing more than watching the games will tell you.

 

[Bear of Bad News]

It won't stop people with too much free time from trying to create ratings that are seriously flawed and tell you nothing more than watching the games will tell you.

(1) What's it to you what people do with their free time? (Follow-up: what do you do with your free time?)

(2) Do you think that they're trying to create ratings that are seriously flawed? Knowledge evolves.

I think you're forgetting which sub-forum you are currently reading.

 

[ChiTownPhilly]

Say this with a straight face: "Grabovski is only slightly better than Crosby statistically."

I think THoR needs some work.

How does this happen?? I think I have an hypothesis...

Any reasonable system of competitive measurement needs to take into account strength of opposition, and do so in a manner that keeps from spiralling into the vortex of infinite regression. Now, particularly in the football world, Jeff Sagarin has come up with ideas that border on elegant- but really, my favorite (which doesn't necessarily mean best) solution(s) to this came from George Ignatin. Now Ignatin's work focused on the pros- and here, there are only 32 entities that call for measuring. Sagain's most famous work (BCS College Football Teams) is more challenging, by a factor of at least four. In the NHL, however, there are over 500 players that log meaningful ice-time, so answering a "strength of opposition" challenge here would be immense.

The intuitive hole in assigning parity to Grabovski (breath*) and Crosby is that perhaps strength of opposition was not sufficiently accounted for- or even ignored. One could reasonably anticipate that the former may draw the injury-fill-in/cold & flu season 7th defenseman, whereas the future Hall-of-Famer is getting a steady diet of the top pairing, continuously.

*[Salving my conscience by not mentioning them in the same breath.]

 

[ekespou]

I think there are at least two glaring problems with this paper:

1) Not sure if this is just inexcusably poor writing or whether it's actually part of the theoretical foundation for the analysis, but the paper expressly states more than once, and does seem to rely on the assumption, that it is "evaluating players for every event that happens on the ice." I.e., all stats kept by RTSS. Obviously those stats are severely deficient and incomplete, which means that if the paper does in fact assume that the stats kept by RTSS are coextensive with all meaningful on-ice actions then its results are going to be skewed, probably significantly.

2) With the exception of a few express attempted adjustments to the RTSS stats (e.g., MSG shot locations), the study makes no attempt to differentiate for quality of the on-ice actions that are recorded as stats. In a sense this criticism is included within #1, in that the idea is that just as RTSS fails to do, so THor does not make any allowance for the difference in quality of shots, hits, etc. etc.

I'm new to the boards and expect there may be some disagreement about this second point, as in many ways Corsi and Fenwick suffer from the same problem.

 

[Wizeman*]

I think there are at least two glaring problems with this paper:

1) Not sure if this is just inexcusably poor writing or whether it's actually part of the theoretical foundation for the analysis, but the paper expressly states more than once, and does seem to rely on the assumption, that it is "evaluating players for every event that happens on the ice." I.e., all stats kept by RTSS. Obviously those stats are severely deficient and incomplete, which means that if the paper does in fact assume that the stats kept by RTSS are coextensive with all meaningful on-ice actions then its results are going to be skewed, probably significantly.

2) With the exception of a few express attempted adjustments to the RTSS stats (e.g., MSG shot locations), the study makes no attempt to differentiate for quality of the on-ice actions that are recorded as stats. In a sense this criticism is included within #1, in that the idea is that just as RTSS fails to do, so THor does not make any allowance for the difference in quality of shots, hits, etc. etc.

I'm new to the boards and expect there may be some disagreement about this second point, as in many ways Corsi and Fenwick suffer from the same problem.

I agree. I do think its incomplete

 

[LiveeviL]

It's tracked by humans. One guy might see two players bump along the boards and call it a hit, while another won't. Repeat that x10 hits per game x82 games per season and you've got a large discrepancy.

I think humans are involved in a lot of hockey statistic gathering. I guess it is a reliability problem, but how does it differ from other metrics?

 

[Hardyvan123]

It's tracked by humans. One guy might see two players bump along the boards and call it a hit, while another won't. Repeat that x10 hits per game x82 games per season and you've got a large discrepancy.

This is the major problem with hockey and stats, prepared to say baseball and football to a lesser degree where plays and their impact can be isolated.

Bob McGowan from the fan in Toronto, whether one likes him or not hit the nail on the head in his hockey arguments book.

To paraphrase him roughly "take away the actual scoring plays in any NHL game and ask people to look at all the other plays in the game and then determine who won the game. This prediction is very random in hockey as scoring chances often go against the flow of the play ect... It's more indicative in baseball and football (all non scoring plays in total) in determining which team probably won the game.

Hockey Prospectus also talks about the high amount of variance or random chance in winning in the NHL today, it's more often than people think or believe or wish it is.

Hockey is always going to lag behind Baseball and even Football in any statistical analysis, and making any conclusions form the data, because of the subjective nature of non scoring plays and their value as well.

 

[doakacola*]

This is the major problem with hockey and stats, prepared to say baseball and football to a lesser degree where plays and their impact can be isolated.

Bob McGowan from the fan in Toronto, whether one likes him or not hit the nail on the head in his hockey arguments book.

To paraphrase him roughly "take away the actual scoring plays in any NHL game and ask people to look at all the other plays in the game and then determine who won the game. This prediction is very random in hockey as scoring chances often go against the flow of the play ect... It's more indicative in baseball and football (all non scoring plays in total) in determining which team probably won the game.

Hockey Prospectus also talks about the high amount of variance or random chance in winning in the NHL today, it's more often than people think or believe or wish it is.

Hockey is always going to lag behind Baseball and even Football in any statistical analysis, and making any conclusions form the data, because of the subjective nature of non scoring plays and their value as well.[/QUOT

Find taking our the goals and evaluating a game a non - starter.

 

[Bad News Bears]

Say this with a straight face: "Grabovski is only slightly better than Crosby statistically."

I think THoR needs some work.

"We also note that had Sidney Crosby, who is ranked 15th, played two full seasons he
would have been in the top 10 and possibly in the top 5."

 

[schuckers]

Update to THoR

For those interested, we've update the THoR model to add a rink effect and score effects. This to reduce bias in the RTSS and in teams changing style of play with the lead.

Latest version of THoR can be found here:
http://www.statsportsconsulting/thor

Some can be found at this link.
http://statsportsconsulting.com/2013/04/19/total-hockey-rating-thor-results-through-041413/


Top 10 in total contribution are
1. ANZE KOPITAR
2. ERIC STAAL
3. JONATHAN TOEWS
4. PATRICE BERGERON
5. LOGAN COUTURE
6. HENRIK SEDIN
7. CHRIS KUNITZ
8. ERIK KARLSSON
9. JOHN TAVARES
10. RYAN GETZLAF


-Schuckers

 

[hella rights]

I recognize the legal maxim that states "hard cases make bad law," but let me give you an example of one of the most memorable plays I've ever seen in person- and postulate that the metric described above has little way of taking the contribution into account.

Philadelphia v. up-and-coming Ottawa team. Legion of Doom line. Renberg carries in puck from R Wing. Lindros gets deep into the zone, and angles to L side, then cuts behind the net as if preparing to start a cycle. Two defenders give complete attention to Lindros, which leaves Renberg partially open, and John LeClair completely open. Cross-ice pass/score.

Now Lindros had more to do with that goal than any player... and he didn't even touch the puck...

I for Incomplete on the measurement attempt as given above. Needs to return to the shed for some overhaul work---

I think this is a really important point that speaks to the nested nature of hockey data. Using your example, even though LeClair scored the goal and Renberg got the assist, everyone on the ice (including the opposition) had an effect. I imagine Lindros being on the ice significantly increased the probability that his linemates would get a point regardless of whether he recorded an objective stat (i.e. goal, assist, hit, etc.), perhaps looking at his contribution that way might more accurately assess his value?

 

[schuckers]

I think this is a really important point that speaks to the nested nature of hockey data. Using your example, even though LeClair scored the goal and Renberg got the assist, everyone on the ice (including the opposition) had an effect. I imagine Lindros being on the ice significantly increased the probability that his linemates would get a point regardless of whether he recorded an objective stat (i.e. goal, assist, hit, etc.), perhaps looking at his contribution that way might more accurately assess his value?

If you look at the THoR model, it credits players for their effect in exactly this way.

 

[ekespou]

If you look at the THoR model, it credits players for their effect in exactly this way.

I haven't looked at the model but that absolutely is not what the Sloan conference paper says the model does.

 

[schuckers]

To quote from the THoR Sloan paper

Note that for every event on the ice we will distribute value for that event to all of the players on the ice. This is done to account for the effect of an individual, for example, a Sidney Crosby or a Shea Weber, who may impact events but is not directly involved in a given event such as a shot.

 

[hella rights]

If you look at the THoR model, it credits players for their effect in exactly this way.

Well I'll be damned, so it does! Maybe you can help clarify a few things for me:
1) Don't understand why the NP20 value is the probability that an event lead to goal after 20s for the home team minus the probability for the away team. Wouldn't a high NP20 just mean that a player was on ice for more events that led to goals in home games? BTW, I wonder what the results would look like if instead of the NP20 value being home probability - away probability it was goal for probability - goal against probability.
2) Maybe I missed it but was it coded anywhere whether the goal that was scored 20s after the event was for or against for each player on the ice?
3) When you say "The exceptions to this valuation of events are shots, goals and penalties" does it it mean there was a significant probability that a goal was scored 20s after a goal? Wasn't 'goal' the dependent?
4) Do you have any descriptive statistics? For instance, which players were on the ice for the most events that led to goals for or against?

Hope you don't mind the interrogation, I think it's a really interesting paper.

 

[Verviticus]

For those interested, we've update the THoR model to add a rink effect and score effects. This to reduce bias in the RTSS and in teams changing style of play with the lead.

how on earth did you only now add something that takes score effects into account

edit: if this sounds antagonistic, good, because i can't for the life of me fathom how in good faith one could come up with a 'comprehensive ranking of players' and not account for very obvious things like that

 

[Bear of Bad News]

edit: if this sounds antagonistic, good, because i can't for the life of me fathom how in good faith one could come up with a 'comprehensive ranking of players' and not account for very obvious things like that

Don't let "perfect" be the enemy of "good". Trying to build a model with all of the bells and whistles right out of the box is usually a good way to not build anything at all.

And "very obvious things" are all relative. He probably included some things that you wouldn't have thought of.

 

[billybudd]

One criticism of THoR from Scott Cullen at TSN can be found here and it is the fact that THoR ignores scoring rate, the rate of shots by a given player that are goals. He validly points out that shots from Steven Stamkos are more likely to go in than shots by Tyler Kennedy. True. However, we know that shooting rates fluctuate a good deal over short periods like over a season. Further, we are not just counting shots by Stamkos and Kennedy in their value but the shots both taken by their teammates and opponents when they are on the ice. We’ll continue to assess this further and potentially tweak THoR. But when we tested this idea in the past it lead to much greater fluctuation in estimated player value

So the long and short of it is that you're using Corsi as a significant aspect of these ratings. Corsi's only valuable when two teams (or ten total players) have the same objective. An objective like (oversimplified): get the puck into the offensive zone, try to get it to someone with a clean look at the net then direct it toward net.

Corsi gives misleading values for Kennedy because that's not his objective. His objective is: get the puck into the offensive zone, shoot (regardless of where he is, what the angle is, what the goalie's doing and what's between him and the net).

Some of Corsi's critics have asked "well, what happens if a team just throws the puck on net from anywhere...wouldn't that game the corsi stat?" Tyler Kennedy's consistently high values suggest that it absolutely does.

For a visual illustration of this, check out the clusters and shot volumes among these two players with similar Corsi, using distance as a rough proxy for shot quality

Good shots

Toews
81 shots

http://somekindofninja.com/nhl/inde...strength=Even&time=Not+Shootout&search=Search


Kennedy
38 shots

http://somekindofninja.com/nhl/inde...strength=Even&time=Not+Shootout&search=Search


bad shots

Toews
32 shots

http://somekindofninja.com/nhl/inde...strength=Even&time=Not+Shootout&search=Search


Kennedy
108 shots

http://somekindofninja.com/nhl/inde...strength=Even&time=Not+Shootout&search=Search


Corsi counts these as if they're the same. They're not (and I assume the differences would only get more pronounced if you were to factor in missed and blocked shots), which is why it rates Kennedy as an equivalent possession player to Jonathan Toews, even though anyone with eyes can see that idea's a joke.

Edit: updated. accidentally had it sorted to away games only

 

[schuckers]

how on earth did you only now add something that takes score effects into account

edit: if this sounds antagonistic, good, because i can't for the life of me fathom how in good faith one could come up with a 'comprehensive ranking of players' and not account for very obvious things like that

We have been planning to add score effects for some time but they didn't make it into the paper for MIT Sloan. It is and will continue to be a work in progress.

Here's a link to the slides for the conference prepared in advance that mentions score effects.
http://www.sloansportsconference.com/?p=10193

 

[schuckers]

Well I'll be damned, so it does! Maybe you can help clarify a few things for me:
1) Don't understand why the NP20 value is the probability that an event lead to goal after 20s for the home team minus the probability for the away team. Wouldn't a high NP20 just mean that a player was on ice for more events that led to goals in home games? BTW, I wonder what the results would look like if instead of the NP20 value being home probability - away probability it was goal for probability - goal against probability.
2) Maybe I missed it but was it coded anywhere whether the goal that was scored 20s after the event was for or against for each player on the ice?
3) When you say "The exceptions to this valuation of events are shots, goals and penalties" does it it mean there was a significant probability that a goal was scored 20s after a goal? Wasn't 'goal' the dependent?
4) Do you have any descriptive statistics? For instance, which players were on the ice for the most events that led to goals for or against?

Hope you don't mind the interrogation, I think it's a really interesting paper.

Not a problem.
1. NP20=P(home goal in 20s )-P(away goal in 20s) ~ home players - away players + other stuff. If NP20 is negative that gives extra value to away players. The key is coding home players positive and away players as negative to mirror the structure of NP20.

2. Absolutely.

3. For penalties we took not prob of goal in first 20 seconds of PP but average PP rate for the length of the penalty. For shots and goals we took P(Shot=Goal) + next 20 seconds.

4. We haven't done much of those as we've focused on building the model. I would think they could be found at stats.hockeyanalysis.com