Subjective +/- (3-year project -- insights appreciated)

[tarheelhockey]

Hi guys,

I've just finished up the third season of an ongoing Canes board project under the title "Subjective +/-". At this point the sample of games is large enough that I think it can be used for analysis by the BTN board.

For those who are familiar with how Roger Neilson used to break down game film, that is the essence of this project (I've actually thought of renaming it Neilson Ratings, but that makes it sound like a TV project... anyway.). The basic philosophy is to remove the "white noise" from the traditional +/- statistic by awarding pluses and minuses ONLY to those players who had a material impact on the scoring play.

I've reviewed game film of every Carolina Hurricanes game over the past 3 seasons (212 games in total) and focused on even-strength goals only (approx. 370 goals). In order to mitigate my personal biases as much as possible, the project is open for correction by other forumers; also, the final tables contain a "Margin of Error" column that provides a clear record of situations where there was a lack of consensus over the final decision.

For more information about the rules and results of the project, see the season-by-season threads below:

2013-14
2012-13
2011-12

Click to read more...

 

[tarheelhockey]

Here are the results to date:

2011-12

Forwards | Plus | Minus | Total | Margin Of Error
Drayson Bowman | 15 | 3 | +12 | 1
Zach Boychuk | 2 | 1 | +1 | -
Tim Brent | 16 | 7 | +9 | 3
Zac Dalpe | 2 | 2 | even | -
Patrick Dwyer | 11 | 6 | +5 | -
Jussi Jokinen | 23 | 10 | +13 | 1
Chad LaRose | 20 | 12 | +8 | -
Riley Nash | 1 | - | +1 | -
Andreas Nodl | 8 | 3 | +5 | -
Alexei Ponikarovsky | 9 | 5 | +4 | -
Tuomo Ruutu | 27 | 10 | +17 | -
Jerome Samson | 3 | 1 | +2 | -
Jeff Skinner | 26 | 12 | +14 | 1
Eric Staal | 33 | 22 | +11 | -
Anthony Stewart | 16 | 8 | +8 | 1
Brandon Sutter | 25 | 10 | +15 | -
Brett Sutter | 3 | 3 | even | -
Jiri Tlusty | 27 | 14 | +13 | 1

Defensemen | Plus | Minus | Total | Margin Of Error
Bryan Allen | 7 | 20 | -13 | 1
Justin Faulk | 7 | 22 | -15 | -
Tim Gleason | 11 | 19 | -8 | 1
Jay Harrison | 10 | 29 | -19 | -
Derek Joslin | 3 | 8 | -5 | -
Tomas Kaberle | 2 | 10| -8 | -
Jamie McBain | 17 | 26 | -9 | 2
Joni Pitkanen | 8 | 13 | -5 | 1
Bobby Sanguinetti | 1 | - | +1 | -
Jaroslav Spacek | 7 | 15 | -8 | -

2012-13

Forwards | Plus | Minus | Total | Margin Of Error
Nicholas Blanchard | 2 | - | +2 | -
Drayson Bowman | 6 | 2 | +4 | -
Zach Boychuk | - | - | 0 | -
Tim Brent | 2 | 2 | 0 | -
Zac Dalpe | 2 | - | +2 | -
Patrick Dwyer | 16 | 7 | +9 | +1
Dan Ellis | 1 | x | +1 | -
Adam Hall | - | 2 | -2 | -
Jussi Jokinen | 4 | 5 | -1 | -
Chad LaRose | 3 | 1 | +2 | -
Riley Nash | 11 | 8 | +3 | -
Andreas Nodl | 1 | - | +1 | -
Tuomo Ruutu | 8 | 1 | +7 | -
Alex Semin | 27 | 5 | +22 | -
Jeff Skinner | 16 | 9 | +7 | -
Eric Staal | 38 | 13 | +25 | -
Jordan Staal | 18 | 8 | +10 | -
Jiri Tlusty | 24 | 3 | +21 | -
Tim Wallace | 3 | 1 | +2 | -
Kevin Westgarth | 4 | - | +4 | -

Defensemen | Plus | Minus | Total | Margin of Error
Brett Bellemore | 2 | - | +2 | -
Marc-Andre Bergeron | 1 | 3 | -2 | -
Joe Corvo | 8 | 12 | -4 | +1
Justin Faulk | 10 | 15 | -5 | -
Tim Gleason | 9 | 10 | -1 | -1, +1
Jay Harrison | 10 | 10 | even | -
Michal Jordan | - | 2 | -2 | -
Jamie McBain | 6 | 12 | -6 | -
Ryan Murphy | - | 1 | -1 | -
Joni Pitkanen | 4 | 7 | -3 | -
Bobby Sanguinetti | 5 | 18 | -13 | -

2013-14

Forwards | Plus | Minus | Total | Margin Of Error
Drayson Bowman | 11 | 4 | +7 | -
Zach Boychuk | 4 | - | +4 | -
Radek Dvorak | 12 | 6 | +6 | -
Patrick Dwyer | 16 | 8 | +8 | +1
Nathan Gerbe | 18 | 4 | +14 | -
Elias Lindholm | 19 | 6 | +13 | -1
Andrei Loktionov | 6 | 6 | even | -
Manny Malhotra | 14 | 7 | +7 | -
Riley Nash | 21 | 7 | +14 | -
Tuomo Ruutu | 14 | 11 | +3 | +1
Alex Semin | 32 | 7 | +25 | -
Jeff Skinner | 30 | 16 | +14 | +1
Eric Staal | 42 | 26 | +16 | -
Jordan Staal | 25 | 6 | +19 | -
Brett Sutter | 2 | 4 | -2 | -
Chris Terry | 1 | 1 | even | -
Jiri Tlusty | 27 | 7 | +20 | +1
Kevin Westgarth | 1 | - | +1 | -

Defensemen | Plus | Minus | Total | Margin of Error
Brett Bellemore | 5 | 8 | -3 | -
Justin Faulk | 23 | 21 | +2 | +1
Tim Gleason | - | 3 | -3 | -
Ron Hainsey | 11 | 20 | -9 | +1
Jay Harrison | 8 | 16 | -8 | -
Mike Komisarek | 1 | 6 | -5 | -
John-Michael Liles | 6 | 5 | +1 | -
Ryan Murphy | 10 | 13 | -3 | -1
Andrej Sekera | 25 | 18 | +7 | +1

 

[tarheelhockey]

I realize the immediate concern with this sort of thing is, "if the decisions are subjective, then individuals will record different results". Hopefully, crowd-sourcing the project has helped to mitigate that issue by forming the results by group consensus rather than individual opinion. But even more significantly, I really do think the margins of error are quite small -- when you actually sit down and run through hundreds of goals over a long period of time, you don't encounter THAT many situations which require a lot of debate. For the most part, it's pretty obvious who contributed to a goal on both sides of a puck.

Assuming that the data above is valid, I would like to get as much value out of it as possible. I've already begun a deep-dive into correlating these results with the Canes' advanced stats to tease out a new layer of analysis for their individual performances, but I'm not nearly the math whiz that some of you folks are. I'd really appreciate some help in applying this data to some larger philosophical questions about quantifying events in hockey.

In particular, a few questions spring to mind:

- What does this data say about the credibility of secondary assists? Is there a way that we can show the degree to which 2As are (not) accurate descriptions of a player's contribution to a goal?

- Does this metric confirm Corsi, challenge Corsi, or say nothing about Corsi at all?

- At a glance, we can easily see that defensemen are directly responsible for fewer GF and more GA than forwards. Does that tell us anything about the position-blind nature of traditional +/-?


I'm hoping that someone here knows how to really put these numbers to work...

 

[Micklebot]

I realize the immediate concern with this sort of thing is, "if the decisions are subjective, then individuals will record different results". Hopefully, crowd-sourcing the project has helped to mitigate that issue by forming the results by group consensus rather than individual opinion. But even more significantly, I really do think the margins of error are quite small -- when you actually sit down and run through hundreds of goals over a long period of time, you don't encounter THAT many situations which require a lot of debate. For the most part, it's pretty obvious who contributed to a goal on both sides of a puck.

Assuming that the data above is valid, I would like to get as much value out of it as possible. I've already begun a deep-dive into correlating these results with the Canes' advanced stats to tease out a new layer of analysis for their individual performances, but I'm not nearly the math whiz that some of you folks are. I'd really appreciate some help in applying this data to some larger philosophical questions about quantifying events in hockey.

In particular, a few questions spring to mind:

- What does this data say about the credibility of secondary assists? Is there a way that we can show the degree to which 2As are (not) accurate descriptions of a player's contribution to a goal?

- Does this metric confirm Corsi, challenge Corsi, or say nothing about Corsi at all?

- At a glance, we can easily see that defensemen are directly responsible for fewer GF and more GA than forwards. Does that tell us anything about the position-blind nature of traditional +/-?


I'm hoping that someone here knows how to really put these numbers to work...

I would think that for the data to be compared to Corsi, you'll first have to convert it to per min or a percentage, as that's how Corsi tends to be expressed.

 

[tarheelhockey]

I would think that for the data to be compared to Corsi, you'll first have to convert it to per min or a percentage, as that's how Corsi tends to be expressed.

I consolidated these numbers in the 2014 thread this morning -- not sure if it's quite the right thing but it has both Corsi and a per-60 version of the +/- stats:

Forwards | Subjective +/- | ES TOI (to nearest minute) | Plus-per-60 | Minus-per-60 | Effect-per-60 | Corsi Rel QOC | Off Zone Start %
Drayson Bowman |+7 |675|0.98|0.36| +0.62 |-0.187|34.7
Zach Boychuk | +4 |102|2.35|0| +2.35 |-0.955|51.0
Radek Dvorak | +6 |572|1.26|0.63| +0.63 |-0.469|31.6
Patrick Dwyer | +8 |957|1.00|0.50| +0.50 |+0.831|40.9
Nathan Gerbe | +14 |1,106|0.98|0.22| +0.76 |+1.032|45.7
Elias Lindholm | +13 |694|1.64|0.52| +1.12 |+0.128|62.3
Andrei Loktionov | even |246|1.46|1.46| 0 |+0.212|62.9
Manny Malhotra | +7 |682|1.23|0.62| +0.62 |+0.112|21.0
Riley Nash | +14 | 811|1.55|0.52| +1.04 |-0.158|59.3
Tuomo Ruutu | +3 |696|1.21|0.95| +0.26 |?|?
Alex Semin | +25 |1,038|1.85|0.40| +1.45 |+0.867|58.9
Jeff Skinner | +14 |999|1.80|0.96| +0.84 |+0.227|66.7
Eric Staal | +16 |1,269|1.99|1.23| +0.76 |+0.786|62.3
Jordan Staal | +19 |1,236|1.21|0.29| +0.92 |+1.075|50.3
Brett Sutter | -2 |121|0.99|1.98| -0.99 |-1.690|39.5
Chris Terry | even |101|0.59|0.59| 0 |+0.473|66.0
Jiri Tlusty | +20 |891|1.82|0.47| +1.35 |+0.911|58.8
Kevin Westgarth | +1 |59|1.02|0| +1.02 |?|?

Defensemen | Subjective +/- | ES TOI (to nearest minute) | Plus-per-60 | Minus-per-60 | Effect-per-60 | Corsi Rel QOC | Off Zone Start %
Brett Bellemore | -3 | 985|0.30|0.49| -0.18 |+0.756|42.7
Justin Faulk | +2 | 1,434|0.96|0.88| +0.08 |+1.019|51.5
Tim Gleason | -3 | 247|0|0.73| -0.73 |?|?
Ron Hainsey | -9 | 1,538|0.43|0.78| -0.35 |+0.722|48.0
Jay Harrison | -8 | 1,014|0.47|0.95| -0.47 |-0.462|54.2
Mike Komisarek | -5 | 348|0.17|1.03| -0.86 |-0.400|53.2
John-Michael Liles | +1 | 603|0.60|0.50| +0.10 |+0.115|50.6
Ryan Murphy | -3 | 750|0.80|1.04| -0.24 |-0.719|62.0
Andrej Sekera | +7 | 1,362|1.10|0.79| +0.31 |+0.951|51.5

 

[Micklebot]

I consolidated these numbers in the 2014 thread this morning -- not sure if it's quite the right thing but it has both Corsi and a per-60 version of the +/- stats:

Forwards | Subjective +/- | ES TOI (to nearest minute) | Plus-per-60 | Minus-per-60 | Effect-per-60 | Corsi Rel QOC | Off Zone Start %
Drayson Bowman |+7 |675|0.98|0.36| +0.62 |-0.187|34.7
Zach Boychuk | +4 |102|2.35|0| +2.35 |-0.955|51.0
Radek Dvorak | +6 |572|1.26|0.63| +0.63 |-0.469|31.6
Patrick Dwyer | +8 |957|1.00|0.50| +0.50 |+0.831|40.9
Nathan Gerbe | +14 |1,106|0.98|0.22| +0.76 |+1.032|45.7
Elias Lindholm | +13 |694|1.64|0.52| +1.12 |+0.128|62.3
Andrei Loktionov | even |246|1.46|1.46| 0 |+0.212|62.9
Manny Malhotra | +7 |682|1.23|0.62| +0.62 |+0.112|21.0
Riley Nash | +14 | 811|1.55|0.52| +1.04 |-0.158|59.3
Tuomo Ruutu | +3 |696|1.21|0.95| +0.26 |?|?
Alex Semin | +25 |1,038|1.85|0.40| +1.45 |+0.867|58.9
Jeff Skinner | +14 |999|1.80|0.96| +0.84 |+0.227|66.7
Eric Staal | +16 |1,269|1.99|1.23| +0.76 |+0.786|62.3
Jordan Staal | +19 |1,236|1.21|0.29| +0.92 |+1.075|50.3
Brett Sutter | -2 |121|0.99|1.98| -0.99 |-1.690|39.5
Chris Terry | even |101|0.59|0.59| 0 |+0.473|66.0
Jiri Tlusty | +20 |891|1.82|0.47| +1.35 |+0.911|58.8
Kevin Westgarth | +1 |59|1.02|0| +1.02 |?|?

Defensemen | Subjective +/- | ES TOI (to nearest minute) | Plus-per-60 | Minus-per-60 | Effect-per-60 | Corsi Rel QOC | Off Zone Start %
Brett Bellemore | -3 | 985|0.30|0.49| -0.18 |+0.756|42.7
Justin Faulk | +2 | 1,434|0.96|0.88| +0.08 |+1.019|51.5
Tim Gleason | -3 | 247|0|0.73| -0.73 |?|?
Ron Hainsey | -9 | 1,538|0.43|0.78| -0.35 |+0.722|48.0
Jay Harrison | -8 | 1,014|0.47|0.95| -0.47 |-0.462|54.2
Mike Komisarek | -5 | 348|0.17|1.03| -0.86 |-0.400|53.2
John-Michael Liles | +1 | 603|0.60|0.50| +0.10 |+0.115|50.6
Ryan Murphy | -3 | 750|0.80|1.04| -0.24 |-0.719|62.0
Andrej Sekera | +7 | 1,362|1.10|0.79| +0.31 |+0.951|51.5

Looks like you've used Corsi QOC. Are you trying to show a link between subjective +/- and quality of competition, or S +/- and shot attempts (Corsi) or relative Corsi?

I'm not really a stats guy (2 or three university courses 10 years ago fades from the memory pretty quick when you don't use it), so beyond what excel tells me the correlation is, I won't be of much help.

 

[nobuddy]

I consolidated these numbers in the 2014 thread this morning -- not sure if it's quite the right thing but it has both Corsi and a per-60 version of the +/- stats:

Forwards | Subjective +/- | ES TOI (to nearest minute) | Plus-per-60 | Minus-per-60 | Effect-per-60 | Corsi Rel QOC | Off Zone Start %
Drayson Bowman |+7 |675|0.98|0.36| +0.62 |-0.187|34.7
Zach Boychuk | +4 |102|2.35|0| +2.35 |-0.955|51.0
Radek Dvorak | +6 |572|1.26|0.63| +0.63 |-0.469|31.6
Patrick Dwyer | +8 |957|1.00|0.50| +0.50 |+0.831|40.9
Nathan Gerbe | +14 |1,106|0.98|0.22| +0.76 |+1.032|45.7
Elias Lindholm | +13 |694|1.64|0.52| +1.12 |+0.128|62.3
Andrei Loktionov | even |246|1.46|1.46| 0 |+0.212|62.9
Manny Malhotra | +7 |682|1.23|0.62| +0.62 |+0.112|21.0
Riley Nash | +14 | 811|1.55|0.52| +1.04 |-0.158|59.3
Tuomo Ruutu | +3 |696|1.21|0.95| +0.26 |?|?
Alex Semin | +25 |1,038|1.85|0.40| +1.45 |+0.867|58.9
Jeff Skinner | +14 |999|1.80|0.96| +0.84 |+0.227|66.7
Eric Staal | +16 |1,269|1.99|1.23| +0.76 |+0.786|62.3
Jordan Staal | +19 |1,236|1.21|0.29| +0.92 |+1.075|50.3
Brett Sutter | -2 |121|0.99|1.98| -0.99 |-1.690|39.5
Chris Terry | even |101|0.59|0.59| 0 |+0.473|66.0
Jiri Tlusty | +20 |891|1.82|0.47| +1.35 |+0.911|58.8
Kevin Westgarth | +1 |59|1.02|0| +1.02 |?|?

Defensemen | Subjective +/- | ES TOI (to nearest minute) | Plus-per-60 | Minus-per-60 | Effect-per-60 | Corsi Rel QOC | Off Zone Start %
Brett Bellemore | -3 | 985|0.30|0.49| -0.18 |+0.756|42.7
Justin Faulk | +2 | 1,434|0.96|0.88| +0.08 |+1.019|51.5
Tim Gleason | -3 | 247|0|0.73| -0.73 |?|?
Ron Hainsey | -9 | 1,538|0.43|0.78| -0.35 |+0.722|48.0
Jay Harrison | -8 | 1,014|0.47|0.95| -0.47 |-0.462|54.2
Mike Komisarek | -5 | 348|0.17|1.03| -0.86 |-0.400|53.2
John-Michael Liles | +1 | 603|0.60|0.50| +0.10 |+0.115|50.6
Ryan Murphy | -3 | 750|0.80|1.04| -0.24 |-0.719|62.0
Andrej Sekera | +7 | 1,362|1.10|0.79| +0.31 |+0.951|51.5

Corsi Rel QOC isn't Corsi relative to the quality of competition, it's quality of competition based on corsi relative. I think just using CF% would be the way to go here.

 

[Dr Quincy]

Just at a quick glance, I wonder how much fan subjectivity/wishful thinking affects the ratings (even unconsciously). Carolina was outscored 5-on-5 by 17 goals, yet only 1 F is a subjective minus, and the F group as a whole is a +169. It just makes me question the results a little.

 

[TheDevilMadeMe]

Just at a quick glance, I wonder how much fan subjectivity/wishful thinking affects the ratings (even unconsciously). Carolina was outscored 5-on-5 by 17 goals, yet only 1 F is a subjective minus, and the F group as a whole is a +169. It just makes me question the results a little.

I think it shows that forwards are much less likely to be responsible for goals against than defensemen, which is not surprising at all. Clearly forwards and defensemen should not be compared directly with this method.

Generally, a superstar defenseman will tack up big pluses by controlling possession and the pace of the game in a way that won't necessarily show up as a direct contribution to a goal.

 

[Michael Farkas]

Yeah, I would think it most systems (as that has an effect too, responsibilities will vary) a winger would have to really try hard to be a minus. As they are most likely the beneficiary or creator of offense while often having the least responsibility on defense. Depending on how far back one goes in the sequence...does a poor read/play on the forecheck 170 feet in the other direction count negatively against a player? Or is it pretty much just the 75 feet of the defensive zone where things can go wrong?

I've kept an eye on this project over the years from a distance, as I like the idea of it...

 

[charlie1]

Wow that's a lot of work. Nice job!

Like someone else said, I wouldn't compare Offensemen to Defensemen using subjective +/-. But if you want to know how well it predicts corsi it would be straightforward to do a regression of sub +/- vs. CF%, with a separate regression for O-men and D-men.

 

[tarheelhockey]

Corsi Rel QOC isn't Corsi relative to the quality of competition, it's quality of competition based on corsi relative. I think just using CF% would be the way to go here.

Sorry, I completely forgot what I had included in that table -- at one point it actually did have a Corsi column, but then I deleted that one and ended up included the QualComp column.

I think I still have the Corsi table somewhere, will try and find it for posting when I'm not on my phone.

 

[tarheelhockey]

Just at a quick glance, I wonder how much fan subjectivity/wishful thinking affects the ratings (even unconsciously). Carolina was outscored 5-on-5 by 17 goals, yet only 1 F is a subjective minus, and the F group as a whole is a +169. It just makes me question the results a little.

The responses above covered what I would have said -- these numbers really seem to highlight the degree to which forwards, especially wingers, participate in GF as opposed to GA. As MF said above, a winger would have to try pretty hard to be a minus. But if he's in the single-digit pluses, that's not very impressive because it means he's either horrid defensively or impotent offensively. But a defenseman in the single-digit pluses is pretty much dominating his opponents. It's something to consider when people make a huge deal out of a winger's defensive ability -- unless he's a Hossa or Lehtinen, he's probably not making THAT big a difference defensively.

In regard to the bolded, that would be my first critique if I weren't overseeing the project. But one thing I've learned is that probably 80% of the time, it's actually pretty cut-and-dry which players are responsible for the scoring of a goal. It's surprising how little ambiguity there is with the majority of them. And by the time you spread the other 20% over the entire team, the margin of error really is quite small... maybe 1-3 goals per player per season, not enough to move the needle dramatically.

 

[tarheelhockey]

Depending on how far back one goes in the sequence...does a poor read/play on the forecheck 170 feet in the other direction count negatively against a player? Or is it pretty much just the 75 feet of the defensive zone where things can go wrong?

My philosophy is to start with the moment of the goal-against, and "rewind" mentally to the last point where the defense was covering the play correctly. There aren't too many 200-foot goals, so the forecheck usually isn't part of the problem -- but it does happen from time to time, particularly when a forechecker fails to get back and cover the trailing man. If the sequence of events is linear from the bad forecheck to the GA, yeah, it counts as a minus.

(Another factor in this is that we're at the mercy of available replay film, which unfortunately doesn't always show things like zone entries or faceoffs, so that's just a logistical limitation that can't be helped. )

 

[Kellogs]

This is really interesting. A few of my comments/questions:

-Would it be possible to somehow normalize the S +/- values for the various positions (C, W & D) such that you could make a direct comparison between players of different positions?

-Did you compare the S +/- values with the standard +/- values of players at 5on5 to see if the differences are significant? Since the purpose of the S +/- is to remove the white noise from the standard +/- rating, it would be interesting to examine how the two compare. It's generally accepted that a hard comparison between +/- values of players from different teams doesn't make sense, but +/- can provide a decent comparison between players on the same team, and you could potentially argue that some of the "white noise" eventually works itself out over a long period of time (ex. a player who gets a minus for stepping onto the ice just as his team is getting scored on might later on benefit from a plus in a similar fashion).

-You compare the S +/- with Corsi in an attempt to examine if there is some sort of correlation between the two. However, isn't it comparing two completely different things? S +/- would also be highly dependent on the finishing ability of a particular team. A team (or player) could have great possession numbers, but if they lack finish, then those two numbers could differ substantially.

 

[tarheelhockey]

-Would it be possible to somehow normalize the S +/- values for the various positions (C, W & D) such that you could make a direct comparison between players of different positions?

That would be a useful next-step. It's clear that each position has a different "expected +/-".

I'm not sure what would be the best basis for normalizing them, though. One forumer normalized for TOI inthis post, which is less daunting than trying to figure out a baseline for each position.

My gut says that the cut-off for a defenseman is right around 0, or maybe a few points into the negative, per full season played.

-Did you compare the S +/- values with the standard +/- values of players at 5on5 to see if the differences are significant?

Comparisons were done in 2013 and 2014.

I'm not sure how to interpret the results, as they seem randomly distributed at face value.

One factor to consider is that Subjective +/- strives to measure "true 5-on-5", to the extent that it eliminates not only SHGs and ENGs, but also goals scored with a pulled goaltender and goals scored immediately upon the expiration of a penalty.

-You compare the S +/- with Corsi in an attempt to examine if there is some sort of correlation between the two. However, isn't it comparing two completely different things? S +/- would also be highly dependent on the finishing ability of a particular team. A team (or player) could have great possession numbers, but if they lack finish, then those two numbers could differ substantially.

Probably so... I'm mostly curious whether there's any meaningful information to be gleaned from taking the two together or in contrast to each other.

Thanks for your insights!

 

[Dr Quincy]

The responses above covered what I would have said -- these numbers really seem to highlight the degree to which forwards, especially wingers, participate in GF as opposed to GA. As MF said above, a winger would have to try pretty hard to be a minus. But if he's in the single-digit pluses, that's not very impressive because it means he's either horrid defensively or impotent offensively. But a defenseman in the single-digit pluses is pretty much dominating his opponents. It's something to consider when people make a huge deal out of a winger's defensive ability -- unless he's a Hossa or Lehtinen, he's probably not making THAT big a difference defensively.

In regard to the bolded, that would be my first critique if I weren't overseeing the project. But one thing I've learned is that probably 80% of the time, it's actually pretty cut-and-dry which players are responsible for the scoring of a goal. It's surprising how little ambiguity there is with the majority of them. And by the time you spread the other 20% over the entire team, the margin of error really is quite small... maybe 1-3 goals per player per season, not enough to move the needle dramatically.

Thanks for all of your work and for your thoughtful answer. I wasn't trying (despite how it may have sounded) to tear down your work. Like I said it was an "at first glance" reaction. I think what is interesting to me is how this version of +/- relates to traditional +/-.

What I mean is, often when we talk about the standard stat, people say "You can't compare players from different teams in +/- because of context" (which I don't necessarily disagree with). With your stat, I think it's necessary for us to be able to see how these numbers compare to those from different teams (at least at the start so we can get some baselines of what truly is a good number for a F or a D).

I defintely appreciated the effort to isolate player performance when it comes to a very teammate dependent stat like traditional +/-.

 

[tarheelhockey]

What I mean is, often when we talk about the standard stat, people say "You can't compare players from different teams in +/- because of context" (which I don't necessarily disagree with). With your stat, I think it's necessary for us to be able to see how these numbers compare to those from different teams (at least at the start so we can get some baselines of what truly is a good number for a F or a D).

Definitely so. A few forumers have suggested that I go and recruit volunteers from other team boards to expand the scope of the data set, and provide some contrast with other performances. I'm not sure how feasible it would be to coordinate volunteers over the course of an entire season, but ideally it would be helpful to have a more complete view of the entire league -- particularly, as you said, to establish baselines for what constitutes a good or bad season at each position.

 

[charlie1]

Can you post that table with a CF% column? Just eye-balling it would probably tell us something.

 

[Bear of Bad News]

I haven't responded yet because I don't have anything interesting add, but this is fascinating work. Very well done.

 

[Legionnaire11]

This is a very cool subject that I've seen a few teams tackle now.

But I would have to agree on the skew here with so many forwards getting a + and so many defenders getting a -... It's really not so simple as saying a winger has less involvement in a GA than a defender, or that a forward has more involvement in a GF than a defender.

At this level of hockey goals are almost always the result of a mistake made by the opponent, or rather a chain of mistakes. If you look at all of the goals scored in the NHL over a season, you would find that the overwhelming majority of them are scored when there is a combination of three errors made in sequence by the team who gave up the goal.

So the traditional NHL +/- really isn't that terrible of a stat, there are a handful of odd circumstances each season, but for the most part it's much more accurate than most hardcore stat guys give it credit for.

 

[tarheelhockey]

But I would have to agree on the skew here with so many forwards getting a + and so many defenders getting a -... It's really not so simple as saying a winger has less involvement in a GA than a defender, or that a forward has more involvement in a GF than a defender.

IMO, it really is that simple. How often do you see a winger materially contribute to a goal against?

At this level of hockey goals are almost always the result of a mistake made by the opponent, or rather a chain of mistakes. If you look at all of the goals scored in the NHL over a season, you would find that the overwhelming majority of them are scored when there is a combination of three errors made in sequence by the team who gave up the goal.

I'm glad you said this -- it reminded me of a stat I intended to crunch when I finished with this season's project.

Season | Eligible GF | Pluses | GF/Plus | Eligible GA | Minuses | GA/Minus
2011-12|139|340| 2.45 |147|291| 1.98
2012-13|89|241| 2.87 |96|157| 1.64
2013-14|137|384| 2.80 |140|236| 1.68

I think we have fairly concrete evidence here to suggest that for every goal scored for or against the Hurricanes over the past 3 seasons, there were about 2 1/2 offensive players and 1 1/2 defensive players involved.

Stated differently: if we think about "an NHL goal" in the abstract, it exists somewhere between a 2-on-1 and a 3-on-2 in favor of the offensive team. Whether that means an actual odd-man break, or an outnumbered situation due to puck movement, that's an informative way to think about what does (and by extension, what does not) lead to scoring in the NHL.

So the traditional NHL +/- really isn't that terrible of a stat, there are a handful of odd circumstances each season, but for the most part it's much more accurate than most hardcore stat guys give it credit for.

I'd say there are a LOT more than a handful of odd circumstances each season for traditional +/-. In fact the very definition of traditional +/- (including SHG, ENG, 6th-skater goals, goals scored 1 second after a penalty expires, fluke goals from center ice, etc) is highly problematic even if you exclude the odd circumstances. Part of the baseline for this project was to correct for those issues by only measuring "true 5-on-5" scenarios.