GM Analytics since Benning took over before 2014 season

[Gampbler]

I posted this in another section yesterday but thought it might warrant it's own thread if there is any interest.

I've been playing around with the theory of developing a GM Rankings based on objective data that can be measured. Surely the minds here can come together and rationally measure where Benning fits among the other GMs in the NHL. My original concept was to quickly compare him to the other seven GMs hired before the 2014-2015 season so there is an even playing field. Those GM's are Brad Treveling, Ron Francis, Joe Sakic, Ron Hextall, Jim Rutherford and Brian Maclellan. Using trades, transactions, signings, buy outs, getting value for UFA, etc. can be the data used. A simple measurable point system could be put in place. As an example, I've quickly come up with perhaps a starting point but would most certainly welcome any and all help to fine tune:

A GM gets points/penalty for acquiring/trading/not re-signing the following:

top 3 forward/top 2 d-man 10 points
Starting NHL goalie 8 points
top 6 forward/top 4 d-man 7 points
NHL forward/d-man that plays more than 25gp for team 4 points
Backup NHL goalie 4 points
AHL forward/d-man/goalie 1-2 points
1st round pick from 1-15 6 points
1st round pick from 16-30 4 points
2nd round pick 3 points
3rd to 4th round pick 2 points
5th to 7th round pick 1 point

Penalties or bonuses could be for the following:

-3 points for a player bought out that the GM signed
3 points for a 1st round pick acquired for a pending UFA
2 points for a 2nd round pick acquired for a pending UFA
1 point for 3rd round pick acquired for a pending UFA

Please feel free to adjust/add/delete any ideas but please refrain from adding any points/penalties into the model that are based on subjective measurements such as giving a penalty for not trading a guy with a no trade clause or measuring playing ability of two players that have not played in the NHL (as examples). I realize there a few other intangibles like cap management, drafting, overspending on contracts, etc..but we just need to add them somehow. Thanks!

I took the time to go through the seven GM's that started in 2014 to generate some "scorecards" for each in order based on the above parameters since these guys took over:

Tim Murray Additions Points Subtractions
TRADING 80 -5 85
SIGNINGS 26 23 3
DRAFTING 66 66 0
BONUS/PENALTIES 8 8 0
TOTAL 180 92 88
Brad Treveling Additions Points Subtractions
TRADING 49 6 43
SIGNINGS 31 31 0
DRAFTING 49 49 0
BONUS/PENALTIES 8 5 3
TOTAL 137 91 46
Ron Francis Additions Points Subtractions
TRADING 47 4 43
SIGNINGS 12 12 0
DRAFTING 62 62 0
BONUS/PENALTIES 16 13 3
TOTAL 137 91 46
Joe Sakic Additions Points Subtractions
TRADING 50 5 45
SIGNINGS 35 35 0
DRAFTING 45 45 0
BONUS/PENALTIES 0 -6 6
TOTAL 130 79 51
Ron Hextall Additions Points Subtractions
TRADING 30 3 27
SIGNINGS 20 20 0
DRAFTING 52 52 0
BONUS/PENALTIES 0 -2 2

TOTAL 102 73 29
Jim Benning Additions Points Subtractions
TRADING 65 1 64
SIGNINGS 18 18 0
DRAFTING 52 52 0
BONUS/PENALTIES 0 0 0
TOTAL 135 71 64
Brian Maclellan Additions Points Subtractions
TRADING 35 1 34
SIGNINGS 23 23 0
DRAFTING 35 35 0
BONUS/PENALTIES 0 -2 2
TOTAL 93 57 36
Jim Rutherford Additions Points Subtractions
TRADING 71 6 65
SIGNINGS 16 16 0
DRAFTING 27 27 0
BONUS/PENALTIES 0 0 0
TOTAL 114 49 65

The irony isn't lost on me that Rutherford ranks 7th out of 7 after just winning the cup.

Click to read more...

 

[DL44]

This is actually very good and looks consistent with a handful of other analytic stats where the eye test that won him GM of the Year is in complete contradiction with what actually really is happening that you may not notice or be in tune with.

So perfect metric.

 

[JackFr]

This is actually very good and looks consistent with a handful of other analytic stats where the eye test that won him GM of the Year is in complete contradiction with what actually really is happening that you may not notice or be in tune with.

So perfect metric.

Not really sure what this means? How did the "eye test" win Rutherford GM of the year? Every personnel decision he made since July 1st 2015 directly and positively contributed to a Stanley Cup which wouldn't have happened without those moves (Kessel/Cullen/Sullivan/Hagelin/Daley).

Anyway, the issue with analytics like this is that it's hard to account for "overpaying" in trades and signings and managing the cap. Benning has obviously been weakest in that respect. Not to mention that the analytic doesn't seem to take into account team success obviously.

 

[me2]

Buyout get a negative value but waiving a player gets nothing?

You will need to post your points in detail if you want people to give you fair assessment.

8 points for a starting goalie without taking into account the level of that goalie, is flawed. You could get 8 points for acquiring pavelec on a $10m/y deal....

 

[The Drop]

I lol'd at objective

 

[The Drop]

Buyout get a negative value but waiving a player gets nothing?

You will need to post your points in detail if you want people to give you fair assessment.

8 points for a starting goalie without taking into account the level of that goalie, is flawed. 8 points for acquiring pavelec on a $10m/y deal....

Sign Brodeur and get 8pts auto

 

[Verviticus]

I lol'd at objective

this is very objective, its just really bad at being useful

 

[Gampbler]

Anyway, the issue with analytics like this is that it's hard to account for "overpaying" in trades and signings and managing the cap. Benning has obviously been weakest in that respect. Not to mention that the analytic doesn't seem to take into account team success obviously.

It's actually not difficult to ascertain overpaying in trades and signings and managing the cap, if you measure actual results after the transaction. Where we tend to assign blame, is at the moment of the deal and injecting a subjective opinion on the players or picks involved at what they will become, when obviously none of us have an actual clue. Agreed it's very hard to measure team success and take that into account, but a point value for regular season/playoff success could surely be given.

My intent was only to provide a level comparison field. I came up with the point system very quickly before I even entered the results. Sources were www.hockeydb.com; www.capgeek.com;www.nhltradetracker.com

 

[VanillaCoke]

The p in your user name is very annoying to me.

 

[Gampbler]

Trade score example using the basic scoring system:

Vancouver Canucks acquire
Erik Gudbranson 4 pts for an established NHL d-man
2016 5th round pick 1 point
Florida Panther acquire
Jared McCann 4 points for an NHL forward
2016 2nd round pick 3 pts
2016 4th round pick 2 pts

As it stands now Florida is ahead 9-5, but if any of the players evolve into a much bigger role or the draft picks pan out, then the score gets adjusted. The only tricky part is what type of requirement of GP in the bigger roles is required to give the higher point values?

Can the parameters and scoring be adjusted/refined? Hell yes! But it's the same for every GM.

 

[Gampbler]

Buyout get a negative value but waiving a player gets nothing?

You will need to post your points in detail if you want people to give you fair assessment.

8 points for a starting goalie without taking into account the level of that goalie, is flawed. You could get 8 points for acquiring pavelec on a $10m/y deal....

I hadn't taken that into account, but I'd think it would only be a negative value if the player was picked up on waivers and became an NHL player. Good point.

Like I've said the point system was done very quickly so is wide open to any adjustments. As for overpaying, I'd think you'd need to set up a range of what is an acceptable salary for position and performance and score based on that.

Maybe something along the lines of:

First line forward $6m to $9m
Top six forward $3m to $5m
NHL forward $min to $2m

Top two d-man $6m to $9m
Top four d-man $3m to $5m
NHL d-man $min to $2m

Starting goalie $4m to $7m
Backup goalie $min to $3m

????

 

[xtra]

I hadn't taken that into account, but I'd think it would only be a negative value if the player was picked up on waivers and became an NHL player. Good point.

Like I've said the point system was done very quickly so is wide open to any adjustments. As for overpaying, I'd think you'd need to set up a range of what is an acceptable salary for position and performance and score based on that.

Maybe something along the lines of:

First line forward $6m to $9m
Top six forward $3m to $5m
NHL forward $min to $2m

Top two d-man $6m to $9m
Top four d-man $3m to $5m
NHL d-man $min to $2m

Starting goalie $4m to $7m
Backup goalie $min to $3m

????

but then how do you define top 6 forward. Within our own organization is sutter a top 6 forward or top 9? It gets tricky because you can't base their quality on points only.

Also the loss of cap space is huge. 10 mil for a borderline #1 goalie is a huge detriment to the team

its a good start for sure though.

 

[Gampbler]

but then how do you define top 6 forward. Within our own organization is sutter a top 6 forward or top 9? It gets tricky because you can't base their quality on points only.

Also the loss of cap space is huge. 10 mil for a borderline #1 goalie is a huge detriment to the team

its a good start for sure though.

Perhaps it comes down to TOI for skaters 5v5 and for goalies, some sort of comparative to the league average for other starters for a few select goalie stats, etc..and if they are below in league average they lose a point(s) from their value of 8?

GeneralFanager has a pretty cool tool that lets you bring up any salary comparable for an individual player. Use Luca Sbisa for example and it brings up all d-men with historical and current contracts that are similar. Using only current salaries and ages 24-29 it comes up with:

2015-2017 Luca Sbisa D 25 3 $3,600,000 $3,600,000

2014-2017 Alexei Emelin D 28 4 $4,100,000 $4,100,000
2015-2016 Michael Del Zotto D 25 2 $3,875,000 $3,875,000
2014-2017 Mark Fayne D 27 4 $3,625,000 $3,625,000
2013-2016 Ladislav Smid D 27 4 $3,500,000 $3,500,000
2015-2016 Cody Franson D 28 2 $3,325,000 $3,325,000
2014-2017 Clayton Stoner D 29 4 $3,250,000 $3,250,000
2013-2016 Jared Cowen D 22 4 $3,100,000 $3,100,000

Surprisingly he doesn't come in last here. lol

The point is to keep anything subjective out, but it's tough to do, but if you don't it diminishes the comparatives. The key in my mind is to have everything the same when allotting points or penalties.

 

[RewBicks]

How exactly is this "analytics"? You've assigned subjective weights to different characteristics (which seem to have little relationship to winning percentage, etc.), compared GMs with dramatically different goals (some aren't/shouldn't even be trying to maximize win percentage), and provided no indication what "GM quality" is, or why we should be worried about it.

Teams go through natural cycles of success and failure over different time horizons depending on a number of factors, including luck. These cycles and the variations in fortune that accompany them are a natural part of what GMs need to navigate over the course of their tenures. You can't get a solid sense of their actual quality (i.e., how well they do navigate these cycles+variations relative to their peers, perhaps measured in terms of win percentage + control variables) without having access to performance periods that far outlast even the longest of NHL GM tenures.

In other words: I'm not sure this is analytics, and I doubt a useful statistical measure of GM quality is really possible given the time horizons over which useful measurement would be required.

 

[Archangel]

I lol'd at objective

Yeah. There is an agenda there.

 

[Fat Tony]

Like I've said the point system was done very quickly so is wide open to any adjustments.

Which makes it subjective.

 

[DadBod]

I love this. I don't really have any constructive criticism but if you could figure a way of doing this without any bias, or more clear eg. What makes a top 6 forward? what makes a top 4 d?






good job!

 

[Gampbler]

How exactly is this "analytics"? You've assigned subjective weights to different characteristics (which seem to have little relationship to winning percentage, etc.), compared GMs with dramatically different goals (some aren't/shouldn't even be trying to maximize win percentage), and provided no indication what "GM quality" is, or why we should be worried about it.

Teams go through natural cycles of success and failure over different time horizons depending on a number of factors, including luck. These cycles and the variations in fortune that accompany them are a natural part of what GMs need to navigate over the course of their tenures. You can't get a solid sense of their actual quality (i.e., how well they do navigate these cycles+variations relative to their peers, perhaps measured in terms of win percentage + control variables) without having access to performance periods that far outlast even the longest of NHL GM tenures.

In other words: I'm not sure this is analytics, and I doubt a useful statistical measure of GM quality is really possible given the time horizons over which useful measurement would be required.

an·a·lyt·ics
noun
the systematic computational analysis of data or statistics.

I'm not sure what your definition is, but the above is definitely my intent. I'm also not sure you or some others read my statement that I'm looking for help in determining that "subjective" weight in the point system to remove that aspect as much as possible. I'm just trying to take a stab at measuring some part of a GM's success. Are there other factors that weigh that success? Of course, but they aren't measurable. Even if this process can shed light on 30% of what GM's do, isn't that a step in the right direction? There is constructive criticism and then there is just criticism.

As for agenda, I'm just curious and didn't know how the numbers would fall out one way or the other.

 

[ferroid]

While I do think that hockey analytics will begin to get value from models of GM behaviour sooner rather than later, I don't think it will be like this. Valid measurement criteria is the absolute barebones first requirements for this sort of analysis, and assigning arbitrary numbers based on other arbitrary distinctions doesn't seem to me to be the way to go about that.

 

[Samzilla]

I think the eye test for rating Benning will do.