All-in-one hockey value stats
- Thread starter overpass
- Start date
Czech Your Math
I am lizard king
They can definitely be improved upon.
GVT arbitrarily decides that goals are worth 1.5 assists and that "threshold" (replacement value?) is 75% of average. I dislike these types of compromises unless necessary. It should either count points equally or devalue assists in proportion to their scarcity (assist to goal ratio). What is the basis of using 75%? I'm also not generally a fan of per-minute type metrics, particularly at even strength.
Point shares if full of overly complicated calculations that don't fully make sense to me. HR's methodologies are not optimal IMO.
I've looked at alternatives using ES GF & GA estimates and pythagorean win% estimates. The results so far probably aren't any worse than point shares, but it definitely needs more thought and study.
Last edited:
TheDevilMadeMe
Registered User
Iain Fyffe's point allocation system is another.
These "all-in-one" stats are only as good as the data that goes into them. As a result, they generally do a very good job of measuring offensive contributations (well, except for H-R's point shares but that's another story), a pretty good job of measuring goaltending, and a poor job of measuring non-goaltending defensive contributions.
As for the two stats overpass mentioned, the biggest issue with GVT is that it appears to consistently overrate the value of goaltending, relative to other positions.
And h-r's "point shares" is was clearly constructed by stats people who don't actually know much about the game of hockey itself. Here's an old thread from the history board that provides a devastating criticism of that formula: http://hfboards.mandatory.com/showthread.php?t=889335
These "all-in-one" stats are only as good as the data that goes into them. As a result, they generally do a very good job of measuring offensive contributations (well, except for H-R's point shares but that's another story), a pretty good job of measuring goaltending, and a poor job of measuring non-goaltending defensive contributions.
As for the two stats overpass mentioned, the biggest issue with GVT is that it appears to consistently overrate the value of goaltending, relative to other positions.
And h-r's "point shares" is was clearly constructed by stats people who don't actually know much about the game of hockey itself. Here's an old thread from the history board that provides a devastating criticism of that formula: http://hfboards.mandatory.com/showthread.php?t=889335
Last edited:
Iain Fyffe
Hockey fact-checker
overpass
Registered User
- Jun 7, 2007
- 5,271
- 2,807
I have a basic philosophical problem with the way that all comprehensive metrics have been implemented. Basically, I don't think it's possible for them to be optimized for all time periods.
Goals vs Assists
Let's take a basic example - the value of a goal vs the value of an assist. For a single game, who was the better player - the player with the most goals, or the most assists? Goals are usually given preference, because they are a direct record of contribution. If we knew nothing about a single game beyond the stat sheet, I think it would be fair to assign goals a higher weight than assists. So if Player A had 2 goals and 0 assists and Player B had 0 goals and 2 assists, and that's all we knew about the two players, I would be fine with giving Player A more GVT, or Point Shares, or Player Contribution, or whatever.
Now let's move to the season level. Player A has 40 goals and 35 assists, and Player B has 25 goals and 50 assists. Who is the more valuable player, if we know nothing else about them? I'm still fine with ranking Player A a little higher, because goals are a more direct contribution, and it's slightly more possible that Player B is getting lucky - but we have a large enough sample to see strong evidence that Player B is a skilled playmaker. I don't think it's correct to give goals significantly more weight than assists at the season level.
Finally, at the career level, I'm not sure we should give goals any more weight than assists at all. We have a ton of information on the players now, and players who have consistently created goals by passing instead of shooting should get equal credit. There aren't any flukes out there anymore.
Let's take a simple metric - hockey-reference.com's Goals Created.
For a seasonal example, we'll pick a couple of lower-line forwards from the 1998-99 Dallas Stars.
Mike Keane - 6 goals, 23 assists, 29 points, 10 Goals Created
Tony Hrkac - 13 goals, 14 assists, 27 points, 11 Goals Created
OK, Keane has more points, Hrkac has more Goals Created. That doesn't seem unreasonable. It's very possible that Hrkac's greater direct contributions (scoring goals) were more valuable than Keane's greater indirect contributions (setting up goals).
Now let's look at some career examples.
Doug Gilmour - 450 goals, 964 assists, 1414 points, 508 Goals Created
Dave Andreychuk - 640 goals, 698 assists, 1338 points, 541 Goals Created
Henrik Sedin - 171 goals, 576 assists, 747 points, 245 Goals Created
Daniel Sedin - 279 goals, 439 assists, 718 points, 266 Goals Created
Adam Oates - 341 goals, 1079 assists, 1420 points, 479 Goals Created
Brett Hull - 741 goals, 650 assists, 1391 points, 579 Goals Created
In every case Goals Created values the playmaker below the goal scorer, despite the fact that these playmakers have demonstrated consistent outstanding abilities to create goals though passing. In fact, I would say that Goals Created is providing a worse result than simple Points and a worse result than any hockey fan who watched these players play. It's the worst case scenario for a metric - people who buy into it are "getting stupider."
The point is that any multiplier to value goals more than assists cannot be constant for the game level, the season level, and the career level, and be optimized for each time period. Yet every comprehensive metric uses a constant multiplier and reports results for at least the season level and the career level.
General application
I've used goals and assists as my example, but I think the problem goes far beyond that. Take, say, plus-minus. Is it meaningful on a game level? Not really. On a season level? Sometimes. On a career level? Absolutely. Players like Bobby Clarke and Pavel Datsyuk obviously had some ability to create value for their teams that is reflected in their career plus-minus. How can a comprehensive metric fully incorporate that value on a career level while avoiding wild game-to-game fluctuations for all players based on their plus-minus?
The same goes for less obvious areas of value. For example, the ability of a goaltender to prevent shots against. We have absolutely no way of measuring this on a game level. We can't even get a reasonable estimate on a season level. We can make an estimate on the career level, although it may still have substantial error bars. Obviously this estimate should be taken into account when rating player value on the career level. But how to allocate this value on the season level, let alone the game level?
Even in other sports, like WAR (wins above replacement) in baseball, we see this problem. Metrics used to measure defensive play apply a certain amount of regression in their estimates of opportunities - but what time period is that estimate optimized for? If for the season, then the career numbers for the metric will be less accurate than they could be, with the range overly compressed. If for the career, single season numbers will include wide swings based on random variation.
Goals vs Assists
Let's take a basic example - the value of a goal vs the value of an assist. For a single game, who was the better player - the player with the most goals, or the most assists? Goals are usually given preference, because they are a direct record of contribution. If we knew nothing about a single game beyond the stat sheet, I think it would be fair to assign goals a higher weight than assists. So if Player A had 2 goals and 0 assists and Player B had 0 goals and 2 assists, and that's all we knew about the two players, I would be fine with giving Player A more GVT, or Point Shares, or Player Contribution, or whatever.
Now let's move to the season level. Player A has 40 goals and 35 assists, and Player B has 25 goals and 50 assists. Who is the more valuable player, if we know nothing else about them? I'm still fine with ranking Player A a little higher, because goals are a more direct contribution, and it's slightly more possible that Player B is getting lucky - but we have a large enough sample to see strong evidence that Player B is a skilled playmaker. I don't think it's correct to give goals significantly more weight than assists at the season level.
Finally, at the career level, I'm not sure we should give goals any more weight than assists at all. We have a ton of information on the players now, and players who have consistently created goals by passing instead of shooting should get equal credit. There aren't any flukes out there anymore.
Let's take a simple metric - hockey-reference.com's Goals Created.
For a seasonal example, we'll pick a couple of lower-line forwards from the 1998-99 Dallas Stars.
Mike Keane - 6 goals, 23 assists, 29 points, 10 Goals Created
Tony Hrkac - 13 goals, 14 assists, 27 points, 11 Goals Created
OK, Keane has more points, Hrkac has more Goals Created. That doesn't seem unreasonable. It's very possible that Hrkac's greater direct contributions (scoring goals) were more valuable than Keane's greater indirect contributions (setting up goals).
Now let's look at some career examples.
Doug Gilmour - 450 goals, 964 assists, 1414 points, 508 Goals Created
Dave Andreychuk - 640 goals, 698 assists, 1338 points, 541 Goals Created
Henrik Sedin - 171 goals, 576 assists, 747 points, 245 Goals Created
Daniel Sedin - 279 goals, 439 assists, 718 points, 266 Goals Created
Adam Oates - 341 goals, 1079 assists, 1420 points, 479 Goals Created
Brett Hull - 741 goals, 650 assists, 1391 points, 579 Goals Created
In every case Goals Created values the playmaker below the goal scorer, despite the fact that these playmakers have demonstrated consistent outstanding abilities to create goals though passing. In fact, I would say that Goals Created is providing a worse result than simple Points and a worse result than any hockey fan who watched these players play. It's the worst case scenario for a metric - people who buy into it are "getting stupider."
The point is that any multiplier to value goals more than assists cannot be constant for the game level, the season level, and the career level, and be optimized for each time period. Yet every comprehensive metric uses a constant multiplier and reports results for at least the season level and the career level.
General application
I've used goals and assists as my example, but I think the problem goes far beyond that. Take, say, plus-minus. Is it meaningful on a game level? Not really. On a season level? Sometimes. On a career level? Absolutely. Players like Bobby Clarke and Pavel Datsyuk obviously had some ability to create value for their teams that is reflected in their career plus-minus. How can a comprehensive metric fully incorporate that value on a career level while avoiding wild game-to-game fluctuations for all players based on their plus-minus?
The same goes for less obvious areas of value. For example, the ability of a goaltender to prevent shots against. We have absolutely no way of measuring this on a game level. We can't even get a reasonable estimate on a season level. We can make an estimate on the career level, although it may still have substantial error bars. Obviously this estimate should be taken into account when rating player value on the career level. But how to allocate this value on the season level, let alone the game level?
Even in other sports, like WAR (wins above replacement) in baseball, we see this problem. Metrics used to measure defensive play apply a certain amount of regression in their estimates of opportunities - but what time period is that estimate optimized for? If for the season, then the career numbers for the metric will be less accurate than they could be, with the range overly compressed. If for the career, single season numbers will include wide swings based on random variation.
Czech Your Math
I am lizard king
HR's formula for Goals Created is inherently flawed. They assume that there are 2 assists per goal, and assign value accordingly. I think points is the fairer metric, but that doesn't mean GC isn't worth a glance.
Raw plus-minus is very flawed. Adjusted plus-minus isn't perfect, but that's due to the limits of the metric, not a design flaw.
I don't think there will ever be an all-in-one value metric that's authoritative. The best metrics to me are those that make comparisons fairer and simpler, and vastly reduce the variables one has to factor in mentally. The worst metrics are those that pretend to do that or to do everything, when in actuality they confuse and complicate the process.
Raw plus-minus is very flawed. Adjusted plus-minus isn't perfect, but that's due to the limits of the metric, not a design flaw.
I don't think there will ever be an all-in-one value metric that's authoritative. The best metrics to me are those that make comparisons fairer and simpler, and vastly reduce the variables one has to factor in mentally. The worst metrics are those that pretend to do that or to do everything, when in actuality they confuse and complicate the process.
GVT is really flawed imo, when you look at some of the scores it gives to defensive defensemen (who drive play even) and the scores it gives to soft-minutes offensive defensemen, you have to believe something fishy is happening.
Hammer Time
Registered User
- May 3, 2011
- 3,957
- 10
Defence is usually the position that all-in-one stats are the worst at judging. One stat I've seen rated Wideman as the league's best defenceman in 2009 (and ranked Carle as one of the best in 2010). The Chara-Wideman pairing did very well, but the stats can't "know" that it was Chara driving the pairing. Since most of the time when they were split up, Chara was getting the tougher minutes, the stats will make him look worse than Wideman.
true, and using heuristics you can weed out the abnormalities, which does make gvt more useful. there are certainly better ways to valuate players than gvt though, and plus, any metric that determines Jack Johnson as a pretty valuable piece is pretty flawed
Czech Your Math
I am lizard king
I'll probably wade through Point Shares again sometime, when I'm in the mood for torture. From what I remember, it's such a mess of arbitrary assumptions and strange calculations, that I don't even know where to start in critiquing it. Given how inherently flawed a very simple stat like Goals Created is on HR (assumes 2 assist per goal) and my disagreement with some of their calculations for adjusted points (using roster size, deducting each player's stats, etc.), I'm sure something as ambitious as Point Shares is a complete cluster####, and that matches my memory of looking at it previously.
I've looked at GVT a bit more thoroughly. It's a lot of information to take in at once, so please correct any misconceptions about it that I may have. It's well thought out, with generally very good methodology (the calculations are mostly coherent and logical). What I mostly have issue with are the many assumptions contained in GVT. I understand that assumptions must often be made, but the method of doing so seems a bit arbitrary in many cases. All in all, I believe it's an excellent effort, but that there is also much room for improvement. I just don't find multiple arbitrary assumptions to agree with the rigorous calculations involved.
Some of my concerns about GVT:
- Goals are 1.5x more valuable than assists. IMO either points should be considered equally or goals valued more based on scarcity. I'd probably value points equally or produce two lists and let the audience decide.
- Offensive Threshold: The 75% of average per-minute production seems completely arbitrary. This is an important number in the calculations, but the author does not indicate any basis for this figure.
- Per-minute Data: Maybe it's just a pet peeve of mine, but it's generally more difficult to sustain per-minute production over much larger minutes per game. Since the metric is based on total value, not per-minute value, I'm not sure how important this is. However, in seasons where ice time is estimated, it becomes more concerning.
- Goalie Threshold Value: The author sets threshold as "allowing 4% more goals (per shot) than league average." I tend to think he has some justification for this, but again he does not indicate this. I was recently trying to determine replacement level for goalies and took the top 2N goalies in GP, then sorted them by SV%, and used the N+1 goalie as "replacement level." I only calculated back to the '93 season. Over that period, my method yielded a number ~.3 SV% points higher per season, with the largest deviations being ~.6 SV% points. I'm not saying his number isn't as good or better, but some sort of rationale should be provided for this, since it significantly affects the results.
- Threshold SV%: This is really confusing. First, the site has these black diamonds with question marks that appear to be intended to be apostrophes in text and minus signs (-) in equations, so I may be reading this wrong. However, it seems like the equation is SPt = SPa - GAA*GTV (where SPa is league avg. SV%, GAA is league avg. GAA and GTV is .04). If I'm reading this right, then this is a major formula flaw, since if GAA = 3.00, then SPt = SPa - .12. This would mean the threshold is set 12 SV% points below league average. It may very well be a typo. What's further confusing is that in the GGVT calculation, SPt is not used, but instead SPa. Then what was the point of calculating SPt?
- Goalie Responsibility: It assigns 75% of the effect in SV% to the goalie, which again seems arbitrary. He mentions that starter/backup SV% are 15% correlated, but I'm honestly not sure how that equates to using 75% responsiblity for goalies for better than average SV%.
- Defensive Shots Prevented: It seems to assign value for fewer than average shots against based purely on ice time. This neglects the fact that possession forwards and defensive d-men should be much stronger per minute than offensive d-men and forwards that trade chances.
- Defensive Plus Minus: It gives defensemen much more credit for better relative plus minus than he does forwards. This seems reasonable given that it's a purely defensive component in his metric, and he give mathematical justification for such, but I'm uncertain whether the weighting is proper. The offensive contributions of forwards and d-men are only compared to the average at each position, and in the offensive component forwards don't receive an explicit "extra credit" as d-men do in this component. Why is an overall measure (RPM) used as a defensive component, as opposed to ESGA? Also, there seems to be no credit for being above league average by this measure, which may be an opportunity missed to measure value that isn't captured in other components.
- Defensive SV%: This is the remainder from the Goalies' SV% component, attributed by ice time. It has the same questions raised about Goalie SV% (75% goalie vs. 25% team) and defensive +/- (weighted in favor of d-men).
- Shootout Value: I have really mixed feelings about using this as a component. While shootout goals/saves do affect team points significantly, it's sort of an artificial part of post-lockout hockey. I would rather it was ignored, but understand why some may choose to include it.
- Ice Time Adjustments: It indicates that adjustments are made to ice time, based on how valuable it is. This is at least based on some sort of historical measurement, which is good. It says, for instance, that a minute of PP time is 3x as valuable offensively as a minute of ES time. However, this could vary substantially by player. A minute of PP time might be much more or much less than 3x valuable as ES time for a player who is particularly good or particularly bad on the PP or at ES, or for a player who "runs" the PP compared to one who doesn't.
- Positional/Overall Values: I wonder how the position and overall values sum. It seems to me that offensive value sums to 25% of total goals scored. I think goalies' SV% value is intended to sum to 3% of total goals scored, with the skater portion of SV% value summing to 1% of goals, for a total of 4%. I'm not sure what other components sum to a non-zero amount, but would like to know what the sums are by position. IOW, since there are 3 forwards, 2 d-men and a goalie on the ice at all times, it seems ideal if the total values of forwards is 3X, the total value of d-men 2X and the total value of goalies 1X. That's only assuming, as the author does, that each position has equal impact (and equal talent??). Perhaps goalie has more and d-man less impact than forward? Perhaps more talent goes to forward and goalie than d-man (this would vary by season)?
I don't mean to be too critical, as I know how difficult it is to develop a metric such as this, as opposed to find flaws in such a metric.
Last edited:
Iain Fyffe
Hockey fact-checker
I'm inclined to agree on valuing points equally. Assists have changed in frequency over the years, and I figure part of that is because of the changes in the relative contribution of the goal-scorer and the play-maker, and this being recognized by official scorers.
That's certainly as justifiable as assuming some fixed ratio.
As for the rest of it, I mentioned this post to Tom Awad and you never know, he might pop by to provide some answers. Can't promise anything though.
Czech Your Math
I am lizard king
Could use league assist/goal ratio as should be done in Goals Created. I'm not sure the reasons for the change in ratio. I would guess at some point an assist changed from "being an important part of the goal" to "last two teammates to touch the puck before the goal and without a change of possession in between" or whatever it is. I think the RSL must have used the former definition even relatively recently.
Thanks for mentioning this post to the author. I could have posted it on his site, but it's a lot to post and did not want it to seem like I was bad-mouthing his metric on his own site (nor do I want to here), when I know he must have put a lot of effort into it.
Last edited:
Iain Fyffe
Hockey fact-checker
Yes, but of course you have "touch" assists on the one hand and "tap-in" goals on the other. That is, on some goals, each of the two assists can be more valuable than the credited goal. So you'd need to have a good reason to use something other than points.
vecens24
Registered User
- Jun 1, 2009
- 5,002
- 1
Yeah one thing I've defintiely noticed with defensive all in one stats is that they tend to overrate contributions of the worse player and undervalue the contributions of the better player on a strong pairing. Its to the point for me where I think they should just do defensive statistics by pairing as opposed to by individual player.
Last edited:
Czech Your Math
I am lizard king
I agree. I think there are special cases, but believe points are superior to goals created or other methods that value goals much higher than assists.
Last edited:
Just reading this discussion is intriguing. Does anyone know of a stat that is a hybrid of plus minus? For instance where it takes your teammates on the ice into effect or something of that sort. I've been looking for one for some time and have been trying to concoct (spelling?) my own for some time. As we all know the usual plus minus is flawed but still relatively informational.
MadLuke
Registered User
- Jan 18, 2011
- 9,525
- 5,165
Goal created is the only things that matter imo, we have no stats for that, but point are the best one. that being said goal are a better indicator than assist.
For an example, is the montreal canadien get a secret russian player next year that score 100 goals, I could see the habs score almost a Goal a game more than last year.
If they add a 20goal-80assist guy they are nowhere near a GBG more than last year (that is by far the team with the most goal in the league), a 100 points guy from FA would not do that, a 100 goal scorer would. in my opinion that did not study this at all.
A other exemple imagine a line of super rick Nash, 3 player, they score 210 goals, 210 points (0 assists) that line created 210 goals.
Imagine a normal first line of 210 points, 3- 30 goals 40 assists guy, that created about 100 goals during the season (some goal created via a pass to a defencemen, etc...), would you choose the first or the second ?
If you use point no difference right ?, you can say you can to this for line but not for player, then again, make the ranking player by player, if you use point they are all egal.
Czech Your Math
I am lizard king
I don't see a line of Rick Nashes scoring 210 goals in any season, except in a bizarro universe.
I think it depends on the team as to whether a 20G/80A or 60G/40A would be more valuable. Generally the latter is more valuable, but not always.
One of the classic examples is Hull & Oates. Look at the Blues in the early 90s. In '89, Hull scored 41G/43A, with Federko a fairly balanced 22G/45A. Federko is traded for Oates, who had 16G/62A the previous year.
1990: Hull 72G/41A, Oates 23G/79A
1991: Hull 86G/45A, Oates 25G/90A
1992: Hull 70G,39A, Oates 10G/59A (54gms)
They added Shanahan in '92 and he had 33G/36A. In '93, Hull & Shanahan each had 50+ G and combined for 195 points. In '94, they each had 50+ goals and combined for 199 points. The Blues were 8th in GF in '90 and 4th in '91. Then they were 12th in '92, 18th in '93 and 17th in '94. It didn't help them to replace a playmaker with a goal scorer.
I think goals are generally more valuable than assists, but if they are, it's probably not proportional to their scarcity. Maybe goals are ~1.25x assists? Also depends on the situation though.
The goal of the player is to maximize the points and total goals of the line and this often is best accomplished by being more of a playmaker. To me, the most valuable is a player that can do both, because he can adapt to different situations much more easily.
Iain Fyffe
Hockey fact-checker
Your example makes no sense. If a line, playing together, produced 210 goals, they would not record 210 goals and 0 assists. They would have 210 goals and something like 320 assists.
So yes, I would choose the line with 530 points over the line with 210 points. But your example isn't saying what you think it says, because you've ignored the fact that goals and assists are interrelated, and not independent of each other.
MadLuke
Registered User
- Jan 18, 2011
- 9,525
- 5,165
I put extreme just to make them clear, but super rick nash would have more goal than assist in my scenario
People say goal = assist, and I say ok give me your 100 goal 0 assist player for my 0 goal 100 assist guy.
It is a extreme exemple to say that generaly goal > assist, when you look a team you will only look at the number of goals, when you look at a line or a power play unit the goal is easy to see as more valuable than assits, but for individual player all of the sudden you have an exact 1 for 1 ratio of importance, not likely.
I want a 30 goals 80 assists guy at least for my 70 goals - 30 assits players. to consider it. The ratio 1 goal = 1.5 assist is to high, but I can easily see a 1.1 or 1.2 assist for goal. (The numbers of assists by goals in the nhl could be a good indicator to look too).
Iain Fyffe
Hockey fact-checker
But if there were three of them on the same line, they probably would not. And if they didn't have someone taking the role of playmaker, the effectiveness of the line would be reduced.I put extreme just to make them clear, but super rick nash would have more goal than assist in my scenario
MadLuke
Registered User
- Jan 18, 2011
- 9,525
- 5,165
Still it is just for making a point, with no other information I take the guy with more goal than assists for the same point totals.
The sedins are not as good together that a 200 goal scorer, because of lot of their points are on the same goal.
ted2019
History of Hockey
MadLuke
Registered User
- Jan 18, 2011
- 9,525
- 5,165
No, you are right it is not sure and for every goal each player on the ice will have created/caused it by an certain ratio.
We do not have perfect stat for goal created, goal prevented and those are the two stat that we want, how many goal were scored or prevented because that the player was on the ice instead of the normal average nhler as his place....
Screening the goaltender can create a goal without having you on the score sheet (and in power play not even in the +/-).
Point is the best stat to know how much offense was created by a player for, but for each goal the team scored a goal not for every assist (lot of goal have 2 assists) so it is a more sure value.
But if you have the context and the hole picture you can make a better choice (and know that a player X with less point created more goal (by screening the goaltender, making the opponent take penalty's, etc...) but with no information at all I would give goal 1.1 or 1.2 time the value of an assist for judging player.
Thus a 70 goals - 30 assists guy probably have create more goal for his team than if the average first line nhler was taking is minute vs a 30 goals - 70 assits guy, but it is just a probability. (Did the guy created is goal by himself despite a poor line a la Pavel Bure or was it the work of a super Joe Thorthon feeding a Cheechoo....)
Ad
Upcoming events
-
Pinguins Bremerhaven @ Eisbaren Berlin - Eisbaren Berlin leads series 2-1Wagers: 3Staked: $415.00Event closes- Updated:
-
-
-
-
