Understanding GM Success

[Calirose]

Hey,

As fans its hard to see exactly how good GMs are at their job.

They make a few selections per year, maybe a trade or 2 and then after a few years we see if they were on point.

Not a huge sample size in a largely random endeavour.

I was wondering if any owners have their GMs list the players, other GMs selected, they they never would have selected, to get a better idea of their GMs predictive ability.

Click to read more...

 

[Bear of Bad News]

If you can agree on a "all in one" metric, and a discount rate, then it's somewhat straightforward (albeit tedious) to go through and value each trade made by a general manager.

Similar process for drafts, although (unlike trades) drafts aren't a zero-sum proposition, so I imagine that you'd want to benchmark against the expected value of a particular draft slot.

The "all in one" metric is a large hurdle (and perhaps impossible), but it's discussed many places, so I won't attempt to go into it here.

The discount rate is important, so that (for instance) a #1 pick three years from now isn't worth the same as a #1 pick now (since a GM has some incentive to win now, and might not even be around in three years). Conceivably, one could calibrate the discount rate on actual transactions made (although I've done a similar calibration exercise in draft picks over the same year, and the value that GMs ascribe to draft picks of different levels doesn't correlate very well to the measured value of draft picks of different levels).

 

[XX]

the value that GMs ascribe to draft picks of different levels doesn't correlate very well to the measured value of draft picks of different levels).

Well, can you really account for pressure and market dynamics when weighting the picks? A small market team needs to keep those picks because it's their only shot at high end talent. A big market can trade them away. Then you mix in the phrases "rebuilding" or "contending" and things have another layer of complexity. A GM could have a surplus of picks in one year, leading to a wildly aggressive pattern. Then there's the personal philosophy of each individual GM. A GM could be making brilliant picks, but then the organization fails to develop them. How do you account for that? Wildcards like career ending injuries? The answer is that you probably cant, at least not in a statistically significant way.

Predictions, especially in the draft, are just mildly educated guesses. An outlier like the Wheeler pick or Hickey pick wouldn't tell you much, and would be impossible to predict. The only sure thing is that GMs like who they like.

 

[MarkGio]

That's a tough one. Let's consider drafting. So while you could say GM Johnny has a draft success rate of 15% (ie, 15% of all draftees make the NHL for 100+ games), that tells you nothing.

Someone who drafts a bunch of 4th liners is better than someone who drafts 1st liners less frequently? Do they force these guys ice-time to up the success rate? What about the scouts and development team?

Who won a trade? Very subjective, where do you start? Even if you clearly won a trade, but that player help win you a stanley cup, was not that a good move for a GM (ie, Kaberle)?

Wins? Perhaps, but assume the Tambellini loses his job today, and while he was present during the years the Oilers went downhill, the next GM would reap the rewards of the team going up hill. And that doesn't credit the coach. Dave Tippett, for example, coaches a poor (fiscally) hockey club to be successful.

Tenure? This assumes a president keeps a GM because he's doing his job well. But then you see guys like Kevin Lowe get promoted or Doug Mclean last nearly 10 years (no offense Oilers & Jackets fans).

Stanely Cup? Well, what if Mike Gillis or Paul Holmgren never win a Cup? Better yet, what if the Oilers do it with Tambellini?

I'm not sure how you would figure this out...

 

[Mayor Bee]

Hey,

As fans its hard to see exactly how good GMs are at their job.

They make a few selections per year, maybe a trade or 2 and then after a few years we see if they were on point.

Not a huge sample size in a largely random endeavour.

I was wondering if any owners have their GMs list the players, other GMs selected, they they never would have selected, to get a better idea of their GMs predictive ability.

It's really difficult to determine or interpret GM success.

In Columbus, Scott Howson was just canned, and plenty of people around the league (and within our own fanbase) made it sound like he was some type of simian creature. And yet when you look at the draft record, it's very good. When you look at his trade record, it's very good, dare I say unusually good. And yet...somehow the results simply weren't there on the ice.

 

[Pokecheque]

It's really difficult to determine or interpret GM success.

In Columbus, Scott Howson was just canned, and plenty of people around the league (and within our own fanbase) made it sound like he was some type of simian creature. And yet when you look at the draft record, it's very good. When you look at his trade record, it's very good, dare I say unusually good. And yet...somehow the results simply weren't there on the ice.

In the end, if you fail to acknowledge a mistake or fail to address just one key weakness, it can be your downfall, no matter how many other things you do right. Howson was successful his first two seasons--everything he did worked. It was failing to realize his franchise goaltender wasn't a franchise goaltender that unraveled EVERYTHING. It led to the firing of Hitchcock and the hiring of Scott Arniel, and everything went way, way south from there.

That's more or less a truncated version of Aaron Portzline's summary of Howson's downfall, and I think it's pretty accurate.

If Burke's undoing in Toronto is any indication as well, it sure looks like a GM's first move should be to secure solid goaltending. A great goaltender can cover for a lot of mistakes both on and off the ice, terrible goaltending leads to losing records and lost jobs. I'm not meaning to downplay the success of Lou Lamoriello, but how successful would he have been without Brodeur? MB at the very least allowed him the flexibility to make some ballsy deals over the years, things he probably wouldn't have done if he didn't have supreme confidence that his franchise goaltender would make up for a lot of weaknesses those teams have had over the years. How successful has Lacroix been since Roy retired?

That's obviously not the ONLY thing you should do to be a successful GM, but it sure as hell is a good cover-your-$%# move, and in the end, that's as important as anything to be successful as a front office exec or coach.