Probability of a First Round Pick = Top 6/4

[Grind]

So i decided to break down the first round of the draft a little more indepthly. I've read a number of studies on the average likelyhood of draft picks succeeding, but i found the studies to either be too broad (using "averaged" probability for entire rounds, or considering 3rd liners/bottom pairing D's "successful") or too narrow (looking only for "elite" talent).

So to get a decent middle grounds I looked at the rate at which each pick in the first round developed into a top 6 or top 4 d man.

criteria for study:

10 years study (99-08 draft) I have excluded previous years due to changing nature of game and change of draft. I would like to revisit this and add more years, but i believe criteria will need to be adjusted for different "era's"

Forwards: Scored at 45 pts/82gp rate in 2 or more season, with at least 35 games played in each season or 1 season if it was last year.

defensemen: ETOI/per game of 19 minutes + in at least 2 or more seasons, with at least 35 games played, or one season if it was last year.

draft table. A 1 designates success where a 0 designates failure (Failure does not mean total bust, just did not meet the criteria laid out).

YEAR|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|TOTAL|Avg % success
2008|1|1|1|1|0|0|0|0|0|1|0|1|0|0|1|0|1|0|0|1|0|1|0|0|0|1|1|0|0|0|12|40
2007|1|1|1|0|1|1|1|0|1|0|0|1|0|1|0|0|0|0|0|0|0|1|0|0|0|1|0|0|0|0|11|36.67
2006|1|1|1|1|1|1|1|0|0|1|1|1|0|0|0|0|0|1|0|0|0|1|1|0|1|0|0|1|0|0|15|50.00
2005|1|1|1|0|1|0|0|1|0|0|1|1|0|0|0|0|0|0|1|0|1|0|0|1|0|0|0|0|1|0|11|36.67
2004|1|1|0|1|1|0|0|0|1|0|0|0|1|1|1|0|0|0|0|1|1|0|1|0|0|1|1|0|1|0|14|46.67
2003|1|1|1|1|1|1|1|1|1|1|1|0|1|1|0|0|1|0|1|1|0|0|1|1|0|0|0|1|0|0|19|63.33
2002|1|1|1|1|1|1|1|1|0|0|1|1|1|1|0|0|1|0|0|0|0|0|0|1|1|0|0|0|0|0|15|50.00
2001|1|1|0|1|0|1|1|0|1|0|0|1|1|0|0|0|0|0|1|1|1|0|1|0|0|0|0|0|0|0|12|40.00
2000|1|1|1|1|1|1|0|0|0|0|0|0|1|0|0|0|0|1|0|1|1|0|0|1|0|0|0|1|1|0|13|43.33
1999|0|1|1|0|1|0|0|0|0|0|0|0|0|0|0|1|1|0|0|0|1|0|0|0|0|1|0|0|0|0|7|23.33

Totals (success count in players and probability)

pick|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|Average
Total successful players out of 10|9|10|8|7|8|6|5|3|4|3|4|6|5|4|2|1|4|2|3|5|5|3|4|4|2|4|2|3|3|0|4.3
Avg percent chance of success|90|100|80|70|80|60|50|30|40|30|40|60|50|40|20|10|40|20|30|50|50|30|40|40|20|40|20|30|30|0|43

Probability by 5 pick "blocks"

Block|Avg %|
1 to 5|84
6 to 10|42
11 to 15|42
16 to 20|30
21 to 25|36
26 to 30|24

The Low down

Average success in the first round: 42.3%
Average success outside of the top 5: 34.8%
Average success in the bottom 15: 30%
Average amount of players per year: 12

highest rate of success: pick # 2: 100% success rate
Highest rate of success outside of top 5: 6,7,12,13,20,21

top 5: 84% 1st over all 90% #2: 100%, #3: 80%, #4: 70%, #5: 80%. - Obviously the most successful block of 5. Nothing outlandish here. Your top 3 picks are pretty much the only spots your gauraunteed a top 6 player.

6-10: 42% 6: 60%, 7: 50%, 8: 30%, 9: 40%, 10: 30% -As found initially, a very sharp drop off from this and the last category. A pick in the 8, 9, 10 range really is not worth the value it seems to hold.

11-15: 42%11: 40%, 12: 60%, 13: 50%, 14: 40%, 15: 20%-Interesting to noe that outside the top 7, picks 12 and 13 are the best bang for your buck both with suprisingly higher success rates then a number of picks in front of them. If not for the poor success rate of pick # 15, this block would be better then the block before it.

16-20: 30%16: 10%, 17:40%, 18: 20%, 19: 30%, 20: 50% - An interesting block in that it is worse then both the block before it and after it, with it's strongest pick being it's last at number 20. The anomaly of how poorly pick # 16 performs is interesting to note.

21-25: 36% 21: 50%, 22: 30%, 23: 40%, 24: 40%, 25: 20% - Surpising...4 of the 5 picks in this block have the same statistical returns as those in the 6-10 block. Pick 21 is interesting to say the least. That top 10 sure isn't looking that swell anymore...

26-30: 24% 26: 40%, 27: 20%, 28:30%, 29: 30%, 30: 0% - Value sure can still be found for the cup contenders in the tail end of the draft, unfortunately just not for the cup winner. Over the 10 drafts examined not a single 30th overall pick became an impact player.

to sum it up, Whats it All Mean?

We can draw a number of conclusions based off this though its hardly concrete (a much bigger sample size would be ideal).

Conclusion number 1: 1st round picks are overrated

It's true, especially here on HFBoards where the 1st round pick is worth it's weight in gold. In the offseason, when nothings been determined for the following year, that pick has only a 43% chance of being anything more then 3rd pairing or 3rd line grinder. A good third liner that's not on the wrong side of 30 should be considered a decent return for 1st round pick from a top 15(standings) team. As shown above, outside of the top 5 thats a less then 35% chance of being more then a third liner, and a 305 chance in the bottom 15.

Conclusion Number 2: Trading up is a bad idea

Given the cost often associated with trading up, GM's are certainly almost always better off sitting tight. In fact, it can be argued that if your sitting just inside the top 10 (8th-10th) you should be burning up the phone lines trying to find that gm with the 2 first rounders in the bottom 13 for you. Even in the final 5, if your not sitting 30th, it hardly seems worth moving any additional assets for 10% increase in probability of landing something meaningful. On the flipside, the Stanley cup champ should always move his first rounder.

Of course, GM's will always trust their scouts and trade to get "their guy" but in a non-specific situation, it doesn't seem worth it.


Conclusion Number 3: Go Easy on Your Prospects

It definitely seems that the idea is every first round pick is almost guaranteed to be a top 6 player, and has a good shot at being a "first liner"- that is simply not the case. The average success rate of becoming a top 6, through the whole draft EDIT: First Round, is only 42%.

Outside of the top 5, it drops to 34.8%.

so outside of the top 5, you less then a 35% chance of drafting a top 6 or top 4 player, meaning the vast majority of your coveted first round prospects or those even given the title of "blue chipper" are quite far from a sure bet.

Final thoughts:

Hopefully this helps highlight a more accurate and realistic sense of worth for First round picks. Correcting for the high success rate in the top 5 is integral to not only properly understanding the likelihood of your prospect developing, but understanding what really is "a good deal". This is be no means a completed study as a much larger sample size would be ideal, but I think it does do a good job of adding a little perspective to the much ballyhoo'd First Day of the Draft.

Comments/Criticisms Encouraged. If any more advanced statheads would like to work on doing something similar with "better" criteria, i'd be all for that.

EDIT: Below is a reposting of the more indepth look by DaveG on Page two. See his post for the attached data.

so breaking it down farther:

top 5:
8 of 44 forwards didn't pan out (.181)
5 of 21 dmen didn't pan out (.238)
all 5 goalies panned out... shockingly. People can argue semantics with DP, but he was a starter for the Isles for a while and a pretty solid one at that.
overall 13 of 70 picks in that span were busts (.186)

6 through 10:
here it starts to get dicey
20 of 46 forwards didn't pan out (.465)
9 of 18 dmen didn't pan out (.500)
all 6 goalies did not pan out
overall, selecting in this zone over the 1994-07 time frame meant batting .500

11 through 15:
24 of 46 forwards selected didn't pan out (.522)
7 of 18 dmen selected didn't pan out (.389)
3 of 6 goalies selected didn't pan out (.500)
34 of 70 picks in this range over that time didn't pan out (.486)
#15 overall pick was particularly a death sentence spot, as only 2 weren't complete misses and one of those two was Radulov.

16 through 20:
34 of 46 forwards didn't pan out (.739)
8 of 21 dmen didn't pan out (.381)
2 of 3 goalies didn't pan out (.667)
44 of 70 players selected in this range did not pan out (.629) basically leaving teams slightly better then 1 in 3 chance of selecting a solid contributor to their team here.

21 through 25:
17 of 37 forwards didn't pan out (.459)
14 of 23 dmen didn't pan out (.609)
5 of 10 goalies didn't pan out (.500)
36 of 70 picks in this draft range didn't pan out (.514), though the question is why it's so much better then the 16-20 zone. Great place to pick a goalie if you're taking one in the first round.

26 through 30:
20 of 30 forwards didn't pan out (.667)
22 of 32 dmen didn't pan out (.688)
5 of 8 goalies didn't pan out (.625)
overall 47 of 70 (.671) selected in this range did not pan out.
#30 is the only draft position worse then #15, with all but 1 being a total bust, and that one (Daron Quint) wasn't a star by any stretch.

Click to read more...

 

[NickRash61]

Everybody on HFBoards needs to see this.

I found the comments interesting about trading up. Unless you have a particular player in mind it doesn't seem to make an awful lot of sense to trade up for a pick outside of the top 5.

Must have been fun going through the stats of the 300 players! Well done

 

[Grind]

Everybody on HFBoards needs to see this.

I found the comments interesting about trading up. Unless you have a particular player in mind it doesn't seem to make an awful lot of sense to trade up for a pick outside of the top 5.

Must have been fun going through the stats of the 300 players! Well done

I definitely agree that the numbers are not inline with the common sentiment. There's a pretty extreme overvaluation on both the Picks themselves and as of yet unproven prospects(draft pedigree)

As for the 300 players, it wasn't a full three hundred, pretty much every year there's at least 4-5 guys that you know are elite talent and thus qualify. the farther back i got a lot harder it was (had to check everyone).

Glad you enjoy'd it. I had often suspected the "true" success rate of the first round was a lot lower then the 70% you often hear.

 

[BigPapaSmurff]

Excellent work!

The one stat that jumps out at me is that 6-10 and 11-15 are both 42%.

Did you happen to notice during your data collection if there was a disparity between forwards and defencemen? I mean, if the first round success rate is 42%, were forwards 45% and defencemen 39%? Or vice versa?

 

[Grind]

Excellent work!

The one stat that jumps out at me is that 6-10 and 11-15 are both 42%.

Did you happen to notice during your data collection if there was a disparity between forwards and defencemen? I mean, if the first round success rate is 42%, were forwards 45% and defencemen 39%? Or vice versa?

hmm i did not but it woudln't take much more then a preliminary check. After i was finished i kicked myself for not making note of more data while i was looking at all the players. This is a good idea, i'll definitly look into this this evening

 

[DaveG]

Noticed a lot of the same trends when I did my comparison about 2 years back heading into the 2010 trade deadline when I was getting into constant arguments about the trade value of Ray Whitney and others.

Basically the only thing that stood out to me differently is that our criteria may have been a bit different for what we considered to be a successful pick. But the end result looks very similar.

Another thing I seemed to recall is that the bust rate on dmen turned out to be higher then the forwards. I'll crunch some numbers but it's led to me shying away from dman heavy drafts like this past one unless it's a matter of there being that many who are very high level talents (think 2008) which this draft IMO flat out wasn't.

I'll crunch the numbers I have, add in the 07 and 08 drafts since those are now useful information, and see what I come up with. But I know 08 is going actually skew dman drafting to look much more positive then it did when I did my analysis.

 

[TrillMike]

This is really interesting. Hopefully a majority of the posters on HF read this. Good work!

 

[Grind]

Noticed a lot of the same trends when I did my comparison about 2 years back heading into the 2010 trade deadline when I was getting into constant arguments about the trade value of Ray Whitney and others.

Basically the only thing that stood out to me differently is that our criteria may have been a bit different for what we considered to be a successful pick. But the end result looks very similar.

Another thing I seemed to recall is that the bust rate on dmen turned out to be higher then the forwards. I'll crunch some numbers but it's led to me shying away from dman heavy drafts like this past one unless it's a matter of there being that many who are very high level talents (think 2008) which this draft IMO flat out wasn't.

I'll crunch the numbers I have, add in the 07 and 08 drafts since those are now useful information, and see what I come up with. But I know 08 is going actually skew dman drafting to look much more positive then it did when I did my analysis.

that's a very good point on different drafts changing weights. i look forward to comparing what you were able to pull. I PM'd you about my criteria, so i won't repeat it here- generally i was trying to focus on offense first top 6 players as that seems to be the "consensus" as to what a "top 6" player is- and i even know many who would scoff at 45 pts being top 6 worthy but thats just ignorance.

 

[MrFunnyWobbl]

Very interesting! The one point about 12-13 being worth the same as some top tens makes sense when you think of all the Cam Fowlers who have the talent but slide down the list After some shiny player goes on a streak three months before the draft.

 

[J17 Vs Proclamation]

Conclusion number 1: 1st round picks are overrated

It's true, especially here on HFBoards where the 1st round pick is worth it's weight in gold. In the offseason, when nothings been determined for the following year, that pick has only a 43% chance of being anything more then 3rd pairing or 3rd line grinder. A good third liner that's not on the wrong side of 30 should be considered a decent return for 1st round pick from a top 15(standings) team. As shown above, outside of the top 5 thats a less then 35% chance of being more then a third liner, and a 305 chance in the bottom 15.

Firstly, interesting and productive analysis.

The above is true to an extent. When analysing it in broad objective terms you are entirely correct. Of course, the value of any particular 1st round pick is dependant on variables that this study cannot incorporate. Needs of team, phase of development, the market and availabilities and of course, the percieved strength of that particular draft. You can have two teams who pick 20 and 21 (or project to pick there at the deadline), but the team at 20 might value the pick much more highly because of their particular situation.

I'd agree, from a fan perspective, 1st round picks are over estimated in worth. For NHL teams however, it will vary.

Conclusion Number 2: Trading up is a bad idea

Given the cost often associated with trading up, GM's are certainly almost always better off sitting tight. In fact, it can be argued that if your sitting just inside the top 10 (8th-10th) you should be burning up the phone lines trying to find that gm with the 2 first rounders in the bottom 13 for you. Even in the final 5, if your not sitting 30th, it hardly seems worth moving any additional assets for 10% increase in probability of landing something meaningful. On the flipside, the Stanley cup champ should always move his first rounder.

Of course, GM's will always trust their scouts and trade to get "their guy" but in a non-specific situation, it doesn't seem worth it.

Statistically yes. However the data you've provided in no way inherently says trading up is a bad idea. It's a very simplistic crude way to come to this conclusion and ignores so many variables. One only has to see the limitations of this study when making assessments like this when we see such glaring anamolies within the analysis. Nobody would argue pick #12 is better to have than pick #10, or that pick #29 is significantly better than pick #30, even though the data here suggests that is the case. So even if we forget the many variables that affect moving up decisions, the study itself shows that moving up in certain spots would make sense ...

Strength of draft and the percieved tiers, the particular guy you believe in combined with the actual price of moving up cannot begin to be quantified by this study, and would require much more indepth analysis before we can saying trading up is a bad idea.

Conclusion Number 3: Go Easy on Your Prospects

It definitely seems that the idea is every first round pick is almost guaranteed to be a top 6 player, and has a good shot at being a "first liner"- that is simply not the case. The average success rate of becoming a top 6, through the whole draft EDIT: First Round, is only 42%.

Outside of the top 5, it drops to 34.8%.

so outside of the top 5, you less then a 35% chance of drafting a top 6 or top 4 player, meaning the vast majority of your coveted first round prospects or those even given the title of "blue chipper" are quite far from a sure bet.

Anybody with common sense should already know the above. Unfortunately, HFboards is populated by illogical, passionate and hopeful fans who see what they want to see when something is affliated to their team.

Conclusion Final thoughts:

Hopefully this helps highlight a more accurate and realistic sense of worth for First round picks. Correcting for the high success rate in the top 5 is integral to not only properly understanding the likelihood of your prospect developing, but understanding what really is "a good deal". This is be no means a completed study as a much larger sample size would be ideal, but I think it does do a good job of adding a little perspective to the much ballyhoo'd First Day of the Draft.

Comments/Criticisms Encouraged. If any more advanced statheads would like to work on doing something similar with "better" criteria, i'd be all for that.

The top 5 idea seems relatively fullproof, although it adds credence to what you hear discussed every year ; there is a top tier, followed by a drop-off. In Joyce's book if i recall, Columbus said their was a clear top 6 (or 7?). Being inside this percieved tier for any given draft is obviously hugely beneficial.

The sample is and isn't a problem. Of course it is nowhere near enough data to formulate any strong concrete definitives. However, data from 15-20 years ago from the draft isn't neccessairly relevant to the data today and may have very different statistical implications because of this.

 

[Grind]

Firstly, interesting and productive analysis.

The above is true to an extent. When analysing it in broad objective terms you are entirely correct. Of course, the value of any particular 1st round pick is dependant on variables that this study cannot incorporate. Needs of team, phase of development, the market and availabilities and of course, the percieved strength of that particular draft. You can have two teams who pick 20 and 21 (or project to pick there at the deadline), but the team at 20 might value the pick much more highly because of their particular situation.

I'd agree, from a fan perspective, 1st round picks are over estimated in worth. For NHL teams however, it will vary.

Definitly, i by no means am attempting to tell GM's of teams their wrong. The biggest thing no study will be able to account for is simply how high a teams scouts may be on certain players, which will always change the percieved "value" of any pick.
[/quote]

Statistically yes. However the data you've provided in no way inherently says trading up is a bad idea. It's a very simplistic crude way to come to this conclusion and ignores so many variables. One only has to see the limitations of this study when making assessments like this when we see such glaring anamolies within the analysis. Nobody would argue pick #12 is better to have than pick #10, or that pick #29 is significantly better than pick #30, even though the data here suggests that is the case. So even if we forget the many variables that affect moving up decisions, the study itself shows that moving up in certain spots would make sense ...

Strength of draft and the percieved tiers, the particular guy you believe in combined with the actual price of moving up cannot begin to be quantified by this study, and would require much more indepth analysis before we can saying trading up is a bad idea.

Yes, I was really only paying attention to statistically.

I think it would be entirely impossible to determine whether or not it would ever be worth trading up, as you said it's just not a "quantifiable" decision- but at the base level, dealing in "blind" terms, the quality of players available/picked barely regresses from outside the top 5- meaning one should not, from an analytic perspective, trade up without outside, qualitative reasoning(their guy, an unusual perceived drop off in talent in a particular draft, etc etc).

While it's true to say the data doesn't say trading up is a bad idea, it certainly doesn't suggest that you are in anyway assured of a better player. Perhaps I worded my conclusion a little too brashly, but i still stand by it.

Anybody with common sense should already know the above. Unfortunately, HFboards is populated by illogical, passionate and hopeful fans who see what they want to see when something is affliated to their team.

Well we all know what they say about common sense....

The top 5 idea seems relatively fullproof, although it adds credence to what you hear discussed every year ; there is a top tier, followed by a drop-off. In Joyce's book if i recall, Columbus said their was a clear top 6 (or 7?). Being inside this percieved tier for any given draft is obviously hugely beneficial.

The sample is and isn't a problem. Of course it is nowhere near enough data to formulate any strong concrete definitives. However, data from 15-20 years ago from the draft isn't neccessairly relevant to the data today and may have very different statistical implications because of this.

Definitely agree with that. Obviously you expect there to be a drop off of talent between the "upper echelon" and the rest, I guess I was mostly just surprised by the severity of the drop.

One thing i would like to look into is possibly the reasoning for an uneven regression. As one poster mentioned, I wonder how much 'Shiny new toy" syndrom effects the top 10 with late risers pushing out those who are "old news"- while everyone forgets that old news was news at some point for a good reason.

I also agree that the study is probably as good as its going to get as far as sample size. When the event only happens once a year any sample size large enough is going to suffer from severe shifts in technology/advancement and require very fluid adjustments to criteria.

 

[wgknestrick]

The binary grading here is what is misleading. A successful 20th pick =/ a successful top 3 pick. You need to show weighted PPG or something else that shows "quality" of pick.

I equate it to the top prize of the powerball vs the top prize of a scratch off ticket. This study is more of a risk analysis relative to draft position

This is more what we'd expect to see.
http://www.hockeyprospectus.com/article.php?articleid=849

 

[Grind]

The binary grading here is what is misleading. A successful 20th pick =/ a successful top 3 pick. You need to show weighted PPG or something else that shows "quality" of pick.

I equate it to the top prize of the powerball vs the top prize of a scratch off ticket. This study is more of a risk analysis relative to draft position

This is more what we'd expect to see.
http://www.hockeyprospectus.com/article.php?articleid=849

Its not misleading at all. If you read what I set out to do its very clear. I don't care about success relative to where the pick is that's not the point of the study.

Its not to determine what success should be at different picks but simply get a probably value of returning a top 6 forward or a top 4 d man- something a binary system can do if we assume my criteria is valid.

 

[wgknestrick]

Its not misleading at all. If you read what I set out to do its very clear. I don't care about success relative to where the pick is that's not the point of the study.

Its not to determine what success should be at different picks but simply get a probably value of returning a top 6 forward or a top 4 d man- something a binary system can do if we assume my criteria is valid.

Let me re-word it. You analysis does just what you claim, but then your "conclusions" about draft value are not based off what you learned in your analysis IMO. You would need additional quality data to make your conclusions (which I don't generally agree with). Would you trade a 40% chance to hit powerball jackpot for a 40% chance to hit scratch-off ticket? Sure the probabilities of "success" are the same, but the rewards are anything but. (referencing the hockeyprospectus article above)

I think your analysis is very worthwhile, but for another reason...comparing success rates between forwards / defenders relative to where they are drafted. I highly suspect that you are much better drafting forwards in top 10, while defenders might be a better risk in late 1st-2nd rounds. It would be nice to see some data on it.

There aren't many top forwards in the league that were taken out of the top 10, but many top defenders have been (Chara, Lindstrom, Letang, Suter). I would expect to see this trend in your analysis.

 

[NickRash61]

Let me re-word it. You analysis does just what you claim, but then your "conclusions" about draft value are not based off what you learned in your analysis IMO. You would need additional quality data to make your conclusions (which I don't generally agree with). Would you trade a 40% chance to hit powerball jackpot for a 40% chance to hit scratch-off ticket? Sure the probabilities of "success" are the same, but the rewards are anything but. (referencing the hockeyprospectus article above)

Don't think that comparison is valid, more like win $5 million or $1 million. I'd be happy with either.

OP said:

"So to get a decent middle grounds I looked at the rate at which each pick in the first round developed into a top 6 or top 4 d man."

And did exactly that. He could have done a top 3 forward or number 1 d-man but he chose to do top 6 / top 2.

Give the man some credit professor

 

[Grind]

Let me re-word it. You analysis does just what you claim, but then your "conclusions" about draft value are not based off what you learned in your analysis IMO. You would need additional quality data to make your conclusions (which I don't generally agree with). Would you trade a 40% chance to hit powerball jackpot for a 40% chance to hit scratch-off ticket? Sure the probabilities of "success" are the same, but the rewards are anything but. (referencing the hockeyprospectus article above)

I think your analysis is very worthwhile, but for another reason...comparing success rates between forwards / defenders relative to where they are drafted. I highly suspect that you are much better drafting forwards in top 10, while defenders might be a better risk in late 1st-2nd rounds. It would be nice to see some data on it.

There aren't many top forwards in the league that were taken out of the top 10, but many top defenders have been (Chara, Lindstrom, Letang, Suter). I would expect to see this trend in your analysis.

fair deuce.

My conclusions are based only on the assumption your hoping to get a top 6/4 player, NOT including the improved likely hood of top 1f/2d or reduce chances for complete bust. I still do not think the benefit of trading up is all it's cracked out to be, even looking at the HP graphs, without using the regression curve there's significant peak and valleys within the top 30, which is all i'm concerned about.

Your reference of the 40% scratch ticket vs powerball is also an oversimplification of what I was trying to do.

As outside of the top 5, a more accurate analogy would be trading 40% scratch ticket AND a 10 % powerball for a 40% scratch ticket and 13% powerball, which your right, this analysis doesn't show as it doesn't include your "powerball" data. I'd have to check the numbers on your likely hood of an top line talent between the rounds, but even based of the jagged nature of the top 30 on the HF graphs, I don't think it is a significant improvement once your outside the top 5.


If i was REALLY possessed I would go back, and compare the likelihoods of every pick into 4 categories, (top end, top6/4, bottom 6, bust) as well as recording Defence vs offence....but frankly, I probably won't be doing that anytime soon.

 

[mrzeigler]

Interesting analysis, but I think a bit incomplete ... or at least not complete enough to draw your No. 1 conclusion. Here's why:

The criteria you used — although it's reasonable for assessing top-6 F and top-4 D as a whole — is not suitable for determining value between those who make it to top-6 F or top-4 D status. (EDIT: I know you didn't set out to do this, but if you're not going to do this, I think it's a stretch to come to your conclusion.)

Once a player has crossed the minimal threshold that you've established, your analysis considers their value to be equal. It's not. A first-line center who regularly tops 90 points is worth a hell of a lot more than a second-line winger who has notched 45 points in two seasons. And I'm not even talking about the fact that some skills and worth are not easily quantified.

A tiered evaluation would produce a more useful analysis. If you were to establish reasonable criteria to grade those who make the Top-6/Top-4 cut into "elite" "very good" "good" grouping and see what draft spots those players were taken at, then you might have even data to support such a claim or — more likely — come to another conclusion.

But to say that a first overall pick is only slightly more likely than a later pick to be a serviceable first- or second-liner is a simplistic conclusion. Show me a GM who covets the first-overall draft pick so he could get a reliable 45-point forward. You can't.

Is a later pick only slightly less likely than a first overall selection to break the 40-goal mark regularly? Is a later pick only slightly less likely to score 100 points in a season? Is he only slightly less likely to average about 30 mins of icetime on the blueline while running the PP and being its top PKer?

In other words, is a later pick only slightly less likely than a top-5 pick to be a difference maker? I don't know, but my gut instinct is to say no.

Also, you fail to realize the fact that the first-overall pick has additional value because that GM has his pick of everyone in the draft class. He can take whoever he thinks is the best player. The guy with the second pick doesn't always have that luxury. And the guy with the third pick has it even less frequently ... and the further along you go in the first round, the more reactive GMs have to become with their picks.

By the time the first round of the draft gets into the 20s, most teams — unless you have a situation like Calgary this year — will have scratched off theall of the top 10 guys they had ranked and a most of the guys they had ranked 11-20.

I wouldn't be so quick to dismiss the value of being in complete control and getting the player you wanted. Granted, that GM might be wrong, but if you ask anyone in any career if they would rather make a decision and be allowed all possible solutions to choose from or if they'd rather make a decision with only a limited field of solutions, they'll always opt for the first choice.

So i decided to break down the first round of the draft a little more indepthly. I've read a number of studies on the average likelyhood of draft picks succeeding, but i found the studies to either be too broad (using "averaged" probability for entire rounds, or considering 3rd liners/bottom pairing D's "successful") or too narrow (looking only for "elite" talent).

So to get a decent middle grounds I looked at the rate at which each pick in the first round developed into a top 6 or top 4 d man.

criteria for study:

10 years study (99-08 draft) I have excluded previous years due to changing nature of game and change of draft. I would like to revisit this and add more years, but i believe criteria will need to be adjusted for different "era's"

Forwards: Scored at 45 pts/82gp rate in 2 or more season, with at least 35 games played in each season or 1 season if it was last year.

defensemen: ETOI/per game of 19 minutes + in at least 2 or more seasons, with at least 35 games played, or one season if it was last year...

Conclusion number 1: 1st round picks are overrated

It's true, especially here on HFBoards where the 1st round pick is worth it's weight in gold. In the offseason, when nothings been determined for the following year, that pick has only a 43% chance of being anything more then 3rd pairing or 3rd line grinder. A good third liner that's not on the wrong side of 30 should be considered a decent return for 1st round pick from a top 15(standings) team. As shown above, outside of the top 5 thats a less then 35% chance of being more then a third liner, and a 305 chance in the bottom 15.

Conclusion Number 2: Trading up is a bad idea

Given the cost often associated with trading up, GM's are certainly almost always better off sitting tight. In fact, it can be argued that if your sitting just inside the top 10 (8th-10th) you should be burning up the phone lines trying to find that gm with the 2 first rounders in the bottom 13 for you. Even in the final 5, if your not sitting 30th, it hardly seems worth moving any additional assets for 10% increase in probability of landing something meaningful. On the flipside, the Stanley cup champ should always move his first rounder.

Of course, GM's will always trust their scouts and trade to get "their guy" but in a non-specific situation, it doesn't seem worth it.


Conclusion Number 3: Go Easy on Your Prospects

It definitely seems that the idea is every first round pick is almost guaranteed to be a top 6 player, and has a good shot at being a "first liner"- that is simply not the case. The average success rate of becoming a top 6, through the whole draft EDIT: First Round, is only 42%.

Outside of the top 5, it drops to 34.8%.

so outside of the top 5, you less then a 35% chance of drafting a top 6 or top 4 player, meaning the vast majority of your coveted first round prospects or those even given the title of "blue chipper" are quite far from a sure bet.

Final thoughts:

Hopefully this helps highlight a more accurate and realistic sense of worth for First round picks. Correcting for the high success rate in the top 5 is integral to not only properly understanding the likelihood of your prospect developing, but understanding what really is "a good deal". This is be no means a completed study as a much larger sample size would be ideal, but I think it does do a good job of adding a little perspective to the much ballyhoo'd First Day of the Draft.

Comments/Criticisms Encouraged. If any more advanced statheads would like to work on doing something similar with "better" criteria, i'd be all for that.

 

[mrzeigler]

In other words, I'm hungry for an analysis that offers the following percentages for each draft position:

• chance of being elite
• chance of being very good
• chance of being good
• chance of being a 3rd or 4th liner
• chance of not spending significant time in the NHL

Those percentages then could be fed into a weighted formula (because the value of those results decreases as you move down the list) that determines the 10-year success rate of the draft position.

 

[Grind]

Interesting analysis, but I think a bit incomplete ... or at least not complete enough to draw your No. 1 conclusion. Here's why:

The criteria you used — although it's reasonable for assessing top-6 F and top-4 D as a whole — is not suitable for determining value between those who make it to top-6 F or top-4 D status. (EDIT: I know you didn't set out to do this, but if you're not going to do this, I think it's a stretch to come to your conclusion.)

Once a player has crossed the minimal threshold that you've established, your analysis considers their value to be equal. It's not. A first-line center who regularly tops 90 points is worth a hell of a lot more than a second-line winger who has notched 45 points in two seasons. And I'm not even talking about the fact that some skills and worth are not easily quantified.

A tiered evaluation would produce a more useful analysis. If you were to establish reasonable criteria to grade those who make the Top-6/Top-4 cut into "elite" "very good" "good" grouping and see what draft spots those players were taken at, then you might have even data to support such a claim or — more likely — come to another conclusion.

But to say that a first overall pick is only slightly more likely than a later pick to be a serviceable first- or second-liner is a simplistic conclusion. Show me a GM who covets the first-overall draft pick so he could get a reliable 45-point forward. You can't.

Is a later pick only slightly less likely than a first overall selection to break the 40-goal mark regularly? Is a later pick only slightly less likely to score 100 points in a season? Is he only slightly less likely to average about 30 mins of icetime on the blueline while running the PP and being its top PKer?

In other words, is a later pick only slightly less likely than a top-5 pick to be a difference maker? I don't know, but my gut instinct is to say no.

Also, you fail to realize the fact that the first-overall pick has additional value because that GM has his pick of everyone in the draft class. He can take whoever he thinks is the best player. The guy with the second pick doesn't always have that luxury. And the guy with the third pick has it even less frequently ... and the further along you go in the first round, the more reactive GMs have to become with their picks.

By the time the first round of the draft gets into the 20s, most teams — unless you have a situation like Calgary this year — will have scratched off theall of the top 10 guys they had ranked and a most of the guys they had ranked 11-20.

I wouldn't be so quick to dismiss the value of being in complete control and getting the player you wanted. Granted, that GM might be wrong, but if you ask anyone in any career if they would rather make a decision and be allowed all possible solutions to choose from or if they'd rather make a decision with only a limited field of solutions, they'll always opt for the first choice.

As per the previous poster, I think your splitting hairs and putting words in my mouth.

In previous posts i've addressed the inability to account for "wanting your guy" or what you've refered as to the complete control of the player you wanted.

Also to say a pickoutside is only "slightly less" then a top 5 pick is ridiculous, I never said nor implied that. I have stated and the data suggests that a pick from 10-15 is only slightly less valuable or of equivelent value to that of a pick in the 6-10 range. Both are at best HALF as valuable as a top 5 pick (84% VS 42%).

I don't disagree with your assertions, but i didn't say or present most of what your refuting.

 

[Grind]

In other words, I'm hungry for an analysis that offers the following percentages for each draft position:

• chance of being elite
• chance of being very good
• chance of being good
• chance of being a 3rd or 4th liner
• chance of not spending significant time in the NHL

Those percentages then could be fed into a weighted formula (because the value of those results decreases as you move down the list) that determines the 10-year success rate of the draft position.

One day i'll do it....one day....

 

[mrzeigler]

As per the previous poster, I think your splitting hairs and putting words in my mouth.

In previous posts i've addressed the inability to account for "wanting your guy" or what you've refered as to the complete control of the player you wanted.

Also to say a pickoutside is only "slightly less" then a top 5 pick is ridiculous, I never said nor implied that. I have stated and the data suggests that a pick from 10-15 is only slightly less valuable or of equivelent value to that of a pick in the 6-10 range. Both are at best HALF as valuable as a top 5 pick (84% VS 42%).

I don't disagree with your assertions, but i didn't say or present most of what your refuting.

My bad. I misread these following lines of yours, which led to my use of "slightly":
Average success outside of the top 5: 34.8%
Average success in the bottom 15: 30%

But looking back at the success rate of the top 5 picks, we can see that even in those top five there is a meaningful dropoff. The combined likelihood of the first two picks of any draft meeting your metric is 95 percent — nearly a lock. The average likelihood of someone drafted 3rd, 4th or 5th meeting your metric drops by 25 percentage points down to 75 percent. That's still good, but enough failure to make GMs/owners/fans nervous ... so I read this and come away thinking that the first and second picks aren't overrated. They're nearly certain to at least give you a serviceable player.

 

[Grind]

My bad. I misread these following lines of yours, which led to my use of "slightly":
Average success outside of the top 5: 34.8%
Average success in the bottom 15: 30%

But looking back at the success rate of the top 5 picks, we can see that even in those top five there is a meaningful dropoff. The combined likelihood of the first two picks of any draft meeting your metric is 95 percent — nearly a lock. The average likelihood of someone drafted 3rd, 4th or 5th meeting your metric drops by 25 percentage points down to 75 percent. That's still good, but enough failure to make GMs/owners/fans nervous ... so I read this and come away thinking that the first and second picks aren't overrated. They're nearly certain to at least give you a serviceable player.

Totally agree. I do not feel first and 2nd overall picks are overrated. I would almost say top 5 picks are not overrated, it's close, but yes, there's still 25% chance you end up with something dissapointing (outside the top 2). It's afterwards where i feel there is a fairly large disconnect between the perceived value and actual success rate (top 6-10 pick sbeing held in a much higher esteem then they're worth).

 

[DaveG]

so after some number crunching of 1994-07, using similar criteria this is what I came up with. I left a good bit of leniency for prospects drafted in the 2005-07 time frame since many are still establishing themselves at the NHL level and even a few who could have been seen as busts as recently as last season have been shedding that label (Tlusty, Helenius, Boyle)

Overall Bust Rate: .502
Forward Bust Rate: .494
Defense Bust Rate: .504
Goalie Bust Rate: .590

so drafting goalies is basically the surest route to drafting a bust with defensemen falling mostly into line with the overall aggregate level. That said I did notice some interesting trends. Since 2003, which is generally regarded as a landmark draft, the trends seem to have reversed on just who ends up a bust.

1994 - 2003:
Foward Bust Rate: .556
Defense Bust Rate: .446

2004 on:
Forward Bust Rate: .333
Defense Bust Rate: .634

Not sure if this is indicative of a change in trends due to scouting changes, if it means that the rule changes that were implemented post-lockout have led to greater success in smaller forwards who otherwise may not have had the same opportunities pre-lockout, a change in team theories about player development and long term contract status (IE: Rutherford's stated aversion to dmen in the first round), or some combination of the 3. I still felt that 2008 was too early to determine, though the only dman from that draft that I would list as a likely bust is Cuma, which when you consider that 12 dmen were selected in that first round brings the dman bust rate significantly lower to around a .400 ratio since the 04 draft. Of course using the same criteria that I would consider for dmen in that draft on forwards leaves all of 3 busts there, which historically we know will not be the case.


Going to crunch a bit more this afternoon and see what trends there were at the positions in certain draft ranges.

 

[Grind]

so after some number crunching of 1994-07, using similar criteria this is what I came up with. I left a good bit of leniency for prospects drafted in the 2005-07 time frame since many are still establishing themselves at the NHL level and even a few who could have been seen as busts as recently as last season have been shedding that label (Tlusty, Helenius, Boyle)

Overall Bust Rate: .502
Forward Bust Rate: .494
Defense Bust Rate: .504
Goalie Bust Rate: .590

so drafting goalies is basically the surest route to drafting a bust with defensemen falling mostly into line with the overall aggregate level. That said I did notice some interesting trends. Since 2003, which is generally regarded as a landmark draft, the trends seem to have reversed on just who ends up a bust.

1994 - 2003:
Foward Bust Rate: .556
Defense Bust Rate: .446

2004 on:
Forward Bust Rate: .333
Defense Bust Rate: .634

Not sure if this is indicative of a change in trends due to scouting changes, if it means that the rule changes that were implemented post-lockout have led to greater success in smaller forwards who otherwise may not have had the same opportunities pre-lockout, a change in team theories about player development and long term contract status (IE: Rutherford's stated aversion to dmen in the first round), or some combination of the 3. I still felt that 2008 was too early to determine, though the only dman from that draft that I would list as a likely bust is Cuma, which when you consider that 12 dmen were selected in that first round brings the dman bust rate significantly lower to around a .400 ratio since the 04 draft. Of course using the same criteria that I would consider for dmen in that draft on forwards leaves all of 3 busts there, which historically we know will not be the case.


Going to crunch a bit more this afternoon and see what trends there were at the positions in certain draft ranges.

Awsome stuff, really look forward to seeing the rest. Now, you didn't explicitly state, is this throughout the entire draft, or only in the first round?

 

[DaveG]

First round only