How would we compare the importance of drafting against development?

[agentfouser]

Inspired by a conversation about whether the failure of the Edmonton rebuild to this point could be put on poor drafting or poor development, I started to wonder if there would be a way to measure the importance of drafting (in terms of evaluating and selecting talent) against that of development (the maximization of the talent that is already selected).

I'm having a hard time imagining how we might do this, since simply looking at which organizations produce more successful NHL players from the draft only tells us that they are doing at least one of those things well, but not which of those things.

Perhaps we could look at which organizations produce more NHL players whose careers start later, a possible indicator of development, but of course not a necessary one.

Thoughts? Have people tried to separate the effects of drafting and development?

Click to read more...

 

[The Nemesis]

Inspired by a conversation about whether the failure of the Edmonton rebuild to this point could be put on poor drafting or poor development, I started to wonder if there would be a way to measure the importance of drafting (in terms of evaluating and selecting talent) against that of development (the maximization of the talent that is already selected).

I'm having a hard time imagining how we might do this, since simply looking at which organizations produce more successful NHL players from the draft only tells us that they are doing at least one of those things well, but not which of those things.

Perhaps we could look at which organizations produce more NHL players whose careers start later, a possible indicator of development, but of course not a necessary one.

Thoughts? Have people tried to separate the effects of drafting and development?

The immediate thing that comes to mind is to compare the career paths of prospects that stay in house (ie with the team that drafts them) with those of prospects the organization trades. And then compare the success/failure rates of in-house prospect development with the development of prospects that were acquired in trade and/or dealt away.

If a team sucks at drafting, but is good at developing, you'd expect to see that trade acquisition prospects would perform better on average than drafted prospects.

If the team sucks at developing but is good at drafting, then they should show better career results on prospects traded away vs ones they keep and develop themselves.

Of course, that would raise several more issues:

• The impact of the scouting eye in the front office. If a team sucks at drafting, would not that scouting ineptitude also transfer over to being unable to evaluate already-drafted prospects for trade acquisition? You'd have to think that the same scouts that can't pick good talent in the draft would be in the GM's ear on young prospects, making trade acquisitions subject to the same problem as straight drafting.
• The small sample size for success given that a lot of trades are going to involve medium-to-low tier prospects who won't pan out, giving a very narrow pool of successes to have to draw conclusions from
• Isolating for the "natural" bust rate. That is to say guys that are just bound to eventually flame out simply because that sort of thing is going to happen from time to time.
• And the same thing that I'm having trouble with in my own attempt to analyze drafts: How do you define success and measure it objectively? Is Mike Ricci a success if he is drafted as a high end scorer and becomes a checking/grinding bottom-6 guy? Do you credit an organization for getting anything out of a prospect? or do you hold it against a team that acquires a prospect in trade if his ultimate ceiling is less than projected?

Those are just my own random thoughts off the top of my head though. Sorry I can't provide more direction in how to deal with those issues.

 

[The Nemesis]

Just quickly to add to my thoughts above since I didn't address the premise in the thread title:

comparing the importance of drafting vs developing would come down to finding the most successful prospects overall and then comparing their draft and development team(s) for commonality.

If drafting is more important, than the best prospects should correlate more with being picked by good drafting teams than with being trained by good developing teams. And vice versa.

In other words, if we find out that teh best teams at drafting are Teams A, B, and C, and the best development teams are Teams X, Y, and Z, then the best overall prospects regardless of drafting/development path should routinely come from A, B, and C if drafting is more important (and a more random mix of teams on the development side), or from X, Y, and Z if development is more important (and a more random mix of drafting teams).

 

[wgknestrick]

Drafting trumps development 9:10 times. You are picking the players who have received the best "development" over the last 8-12 years vs the 2-3 years you have with them in AHL/NHL. Also note that (IMO) development is very similar at the AHL/NHL level. I don't see huge "secret" competitive advantages in training and coaching like there is in player evaluations/stats. All pro teams preach insane nutritional and physical training behavior. Those sciences are very well established vs statistical hockey analysis.

There is only so much shine you can put on fake gemstones.

Proving this will be next to impossible, unfortunately. There just aren't enough data points.

 

[Mathletic]

It's something that's discussed in Moneyball. Beane says they've tried a bunch of things to turn free swingers into disciplined batters. He says they didn't have much success at it.

For me, the evidence points in the direction of drafting being way more important than development. Not that development is not important itself. But there's only so much that can be done for a player to improve at a much faster rate than his peers. Not that it can't be done. I'm sure the process could be improved in many ways. But I doubt teams differ that much in the way they develop players.

 

[Freudian]

Another interesting thing to study is if you can ruin players by bringing them to the league too early or if the failure rate for similarly talented players that are groomed in lower leagues for longer is the same.

 

[West]

I believe that there's a study someone did with a conversion rate that says 1pt in CHL is worth 0.26pts in the NHL(example) and rates for most other high leagues. It's probably reasonable to assume any team that significantly/consistently over or under perform these numbers has it's player development group to thank or blame.

I would think that the above especially holds true once player join an organizations AHL team. For example I don't know if Calgary's player development group can take any credit for Sean Monahan.

Finally I'd look at players year over year performance after joining the clubs farm team. You'd think good player development program would show improvements in a players stats year over year.

Not something that I have time to look into currently but would love to see the results for the above ideas if anyone else did.