Ranking of Teams' Draft Success

Top 6 Spaling

Registered User
Jun 23, 2010
12,341
219
Smashville
The 2015 NHL Draft has come and gone, and as with every draft, there will be winners and losers. Some teams, such as the Detroit Red Wings, are widely considered to be winners as their prospects mature into NHL caliber talent, while others, such as the Edmonton Oilers, well, they just win draft lotteries. Whether a team is a seen as a good or bad drafting team is usually based on a few examples and general perception, but as part of a larger research project on trends in the NHL draft, I dove headfirst into a pile of data to see what teams truly are the best at drafting, statistically speaking. I looked at every pick made in the drafts between 1998 and 2007, which includes as many picks as possible without including selections too distant to represent the current landscape or too near to fairly judge whether or not they were successful. One Excel file with 2640 players later, here is what I found:

The Method

Before getting into the analysis, some of the method has to be explained. If you don’t care how I got the numbers and just want to see the analysis, here’s the gist: I created a formula for success and judged teams based on how the success of their picks compares to the average success of players picked at that exact pick in other years. You can skip to “The Results†if you don’t want any more depth than that.

Whenever a team’s drafting ability is being discussed, one of the biggest issues is the lack of an objective metric to represent success. I fixed that by creating one. I want to make it very clear that the purpose of this research wasn’t to create a hard and fast formula for success, so the formula is just meant to give a general idea of how well a player has developed over their career.

The success metric I settled on incorporates two stats – total games played and points per game for skaters and total games played and save percentage for goalies – to represent how much each player played and how effective they were when they did. Each of these stats was then weighed until I got a reasonable looking list of players that seemed to have forwards, defensemen and goalies accurately represented. People may disagree slightly over the weights and stats used, but it is the best a one man team can do.

I also divided Games Played by Years Since Drafted, because otherwise someone like David Legwand who has been around since 1998 would show up as more successful than someone like Patrick Kane who was drafted in 2007. This led to a slight overrating of players who broke into the league soon after being drafted, but again, close enough for our purposes.

For reference, here are the players with the top 10 success ratings:

RFyhhvk.jpg

It may not be perfect, but it’s close enough to give us a good idea of how different teams have done.

The stat that I actually used to examine a team’s performance wasn’t straight success, but what I am calling “success differenceâ€. This is an individual player’s success minus the average success of a player picked at that spot in the draft. For instance, Jonathan Toews has a success rating of 49.30. The average player picked third overall from 1998-2007 had an average success rating of 39.31. So, Toews’ “success difference†score is a 9.99, meaning he was 9.99 points better on this scale than the average third overall pick. This is a better system to measure how well a team drafted since it takes into account the difference between having the first overall pick is than the thirtieth overall, even if both are in the first round.

Using this scale, a pick with a positive success difference is an above average pick, and one with a negative success difference is a below average one. A zero score means the player succeeded exactly as much as the average player at this pick.

The Analysis

Each team’s average success rating across all positions and rounds is shown on the chart below, for all picks between 1998 and 2007. The team with the most successful picks relative to the average performance of everyone else taken at that pick was the San Jose Sharks. The team that performed the worst was the (now re-named) Phoenix Coyotes.


7icq7aV.jpg

d02f3LI.jpg

San Jose and Montreal formed a strong top two quite a bit above the pack. The best picks for the Sharks in this era, with their corresponding success difference ratings, were Marc-Edouard Vlasic (38.16), Joe Pavelski (30.41), and Christian Ehrhoff (28.74). Montreal’s high score was fueled by great picks such as Mark Streit (39.81), Mike Ribeiro (28.74), and Mikhail Grabovski (27.92). Note that these players were not necessarily the most successful pick by each team, but were the picks that performed the highest above the average player taken at their specific draft position.

On the negative side, Phoenix, Tampa Bay, and Florida performed very poorly in this decade. The two worst picks for each team were Patrick DesRochers (-21.14) and Jakub Koreis (-17.74) for the Yotes, Petr Taticek (-23.45) and Denis Shvidki (-15.23) for the Panthers, and Alexander Svitov (-28.78) and Erkki Rajamaki (-17.86) for the Bolts. Rajamaki scores so low despite being an eighth rounder because, strangely enough, Anton Stralman and Michael Ryder were both 216th overall picks.

While viewing all of the data at once is interesting, we can dive deeper than that. For instance, let’s examine what teams performed the best when only players of a certain position are taken into account.

Kli7TJR.jpg

It is pretty easy to see how Phoenix ended up with the worst score of the decade, since they were in the bottom two at all three positions. Montreal, on the other hand, was in the top five in each case. Keep in mind that while all picks at all positions have success difference scores that sum to zero, this is not true of individual positions. Across the board, forwards (0.28) did better than the average pick at their position, while defensemen (-0.06) and goalies (-1.33) did worse. Generally, teams get more value out of picking forwards than any other position. Centers proved to be the best of all, with an average score of 1.33. In fact, every single other position – wingers, defensemen, and goalies – all had negative average scores because centers outperformed them by so much. This isn’t just a one round fluke either: centers outperformed the average across positions in every single round.

It’s also compelling to see how different teams performed in each individual round. Here are the top and bottom five teams in the first round.

eeP3SsP.jpg

Philadelphia was deadly in the first round in this decade, picking players with positive success difference ratings with eight of their ten picks. Headlining this group is number one center Claude Giroux, who scored a 25.05. The Rangers, however, picked seven negative players with their ten picks, and six of these players were double digit negatives. Hugh Jessiman, the bust of the unbelievable 2003 draft, had the lowest rating, scoring a -24.10.

What if, rather than breaking down the pool by round, we dissect it by nationality of draftees. Which teams drafted best from certain regions of the world? I broke the countries that had a good chunk of draftees into North America, the Russian Block (Russia, Belarus, and Yugoslavia), and Europe. There were a few other countries that produced NHL picks in this time period, but their sample sizes were too small to produce meaningful data (the country with the highest average player difference rating is Brazil because Robin Regehr is the only player ever drafted from that country). Note that, like the above grouping by position, not every region has to have a total score of 0. North America outperformed Europe and the Russian Block.

PB20goO.jpg

Whoever was Montreal’s Russian scout in this decade needs a bonus. That area of the world was a whopping -2.95 on average, but Montreal was one of only five teams that managed to be positive, positing an 8.028. They were more than twice as good as the next team. They drafted four NHL regulars from this region: Andrei Markov, the Kostitsyn brothers, and Alexei Emelin. They also had the best record drafting Europeans, but fell exactly in the middle of the pack at 15th in the North American rankings. On the macro level, it is worth nothing that at least in this decade, the “Russian Factor†is supported by data. In part due to lower GP ratings after some players left for the KHL, Russian Block players posted objectively worse success ratings.

Obviously this can be dissected much further, but let’s move on from teams and focus on the best individual picks on the decade. Here are the ten best and ten worst picks from 1998 to 2007:

xh39Eey.jpg


1cJtTQO.jpg

Note that the Red Wings’ two franchise forwards both are in the top five best picks, while the Rangers managed to make two of the nine worst selections.

Obviously there is just too much to cover in one post – this could probably fill a novel – but if you have specific question about a team or player (i.e. How did the Maple Leafs do drafting defensemen in the second round? Was David Legwand a positive or negative pick?), I’m happy to answer them in the comments or, if there are enough, in a second post.
 
Last edited:

Uncle Bill

Know it all
Sep 21, 2011
770
327
I stopped reading when I discovered at least one team is missing from the analysis.
 

Doctor No

Registered User
Oct 26, 2005
9,250
3,971
hockeygoalies.org
As for the analysis, I like what I see with a coarse overview - I'll dig in more, and hopefully have something constructive and helpful.
 

Helistin

Dustin's equilibrium
Aug 12, 2006
4,222
3,027
Close to you
I have nothing constructive to add. Just wanted to say I find this kind of number stuff very interesting and love to read em. Thanks for the great work. :)
 

tempofound

Registered User
Oct 18, 2013
358
202
Suppose a certain player X is picked at spot N. When you calculate the average sucsessrating of players drafted at spot N, does that include the player X? Or is it the average of all other players picked at position N?
 

Rebels57

Former Flyers fan
Sponsor
Sep 28, 2014
76,737
123,287
This pretty much confirms what Flyers fans knew already. Great job in the first round between 98 and 07 drafting:

98-Simon Gagne
00-Justin Williams
02-Joni Pitkanen
03-Mike Richards
04-Jeff Carter
05-Steve Downie
06-Claude Giroux
07-James van Riemsdyk

But after the 1st round, they were just plain bad.
 

25Bieksa3

Registered User
Apr 28, 2009
446
0
Vancouver
Go Canucks, go! I mean... look were almost heading up that worst possible North American scouting category. We're probably already there if this analysis includes more recent picks! :yo:
 

Doctor No

Registered User
Oct 26, 2005
9,250
3,971
hockeygoalies.org
I like this analysis.

For the games played metric, I agree that the number of eligible seasons needs to be taken into consideration. I'd recommend developing a baseline expectation, with forwards/defensemen/goalies on separate scales, and using that instead of just a straight season count. I'd also adjust for the seasons of different length over the past few years.

As a goalie advocate, I'm worried that they are being punished since goaltender's aren't really "expected" to play a full season (where non-goalies are). On the other hand, goalie drafting is notoriously noisy data anyhow, as can be seen here:

http://hockeygoalies.org/stats/drafts/drafts.html
 

barneyg

Registered User
Apr 22, 2007
2,383
0
On the negative side, Phoenix, Tampa Bay, and Florida performed very poorly in this decade. The two worst picks for each team were Patrick DesRochers (-21.14) and Jakub Koreis (-17.74) for the Yotes, Petr Taticek (-23.45) and Denis Shvidki (-15.23) for the Panthers, and Alexander Svitov (-28.78) and Erkki Rajamaki (-17.86) for the Bolts. Rajamaki scores so low despite being an eighth rounder because, strangely enough, Anton Stralman and Michael Ryder were both 216th overall picks.

(...)

Across the board, forwards (0.28) did better than the average pick at their position, while defensemen (-0.06) and goalies (-1.33) did worse. Generally, teams get more value out of picking forwards than any other position. Centers proved to be the best of all, with an average score of 1.33. In fact, every single other position – wingers, defensemen, and goalies – all had negative average scores because centers outperformed them by so much.

(...)

I broke the countries that had a good chunk of draftees into North America, the Russian Block (Russia, Belarus, and Yugoslavia), and Europe.

That's some very interesting stuff, thanks for doing all the dirty work. I've got a few comments, feel free to disregard if you think I'm being annoying.. they refer to the parts I quoted above:

1) the big negative score for Rajamaki points out a flaw in your methodology that can easily be corrected. I see a super easy but suboptimal correction (use median score instead of average score -- get rid of the effect of outliers on the average), and a slightly more complicated but better calculation of the expected score for each draft spot. I don't know about your background -- the shortest way to phrase my suggestion is: you should regress the average score on draft rank to obtain the predicted score at each spot, and then use that predicted score (instead of the 'real' average score) to calculate "difference". If that makes no sense I can rewrite this in plain English.

2) unless I've misunderstood what you've done, IMO your positional analysis only points out a flaw in your formula for skaters: centers get more points/game on average than wingers or d-men. I wouldn't translate that as "getting more value out of drafting centers".

that said, you could use that info to dissect team performance in a different way, with a 'position-neutral' team score.. i.e. right now a good team score is either due to drafting good players, drafting more forwards and less goalies, or both. you could re-weight each team score and get some theoretical score the team would have achieved if it had drafted X forwards, Y d-men and Z goalies (X,Y,Z being league averages over 98-07 not actuals for that team).

3) I don't understand the strange geographical choice to include Slovenes (and I guess Croats... who else? was a Serb ever drafted?) in the "Russian block", countries from ex-Yugoslavia are historically and geographically further removed from Russia than the Baltic States as well as Czechs/Slovaks/Poles/etc. anyway I doubt it changes anything but thought I'd be a nerd and point it out, I would have made a geographical split between the historical Euro powerhouses (Sweden, Finland, Russia with or without Belarus, Czech Rep, Slovakia) and other countries (Switzerland, Denmark, Norway, Slovenia, Austria, Poland, Germany, etc).
 

Top 6 Spaling

Registered User
Jun 23, 2010
12,341
219
Smashville
Thanks for all the comments! Glad to hear people are enjoying it, plus there are some great suggestions.

Suppose a certain player X is picked at spot N. When you calculate the average sucsess rating of players drafted at spot N, does that include the player X? Or is it the average of all other players picked at position N?

Excellent question, player X is included in the average of N. I hadn't thought about that, it's definitely not good. The solution to that is the same as....

1) the big negative score for Rajamaki points out a flaw in your methodology that can easily be corrected. I see a super easy but suboptimal correction (use median score instead of average score -- get rid of the effect of outliers on the average), and a slightly more complicated but better calculation of the expected score for each draft spot. I don't know about your background -- the shortest way to phrase my suggestion is: you should regress the average score on draft rank to obtain the predicted score at each spot, and then use that predicted score (instead of the 'real' average score) to calculate "difference".

This is a great idea. My first thought was to use median, and I'd love to had a better reason than this for using mean, but it was just an Excel limitation. I calculated the Average Pick Success using an averageif() function, and there's no such thing as a meanif() function. A regression line being used to find predicted values would solve this and the issue tempofound brought up. I'll take a crack at this over the next few days, shouldn't be too difficult.

2) unless I've misunderstood what you've done, IMO your positional analysis only points out a flaw in your formula for skaters: centers get more points/game on average than wingers or d-men. I wouldn't translate that as "getting more value out of drafting centers".

I understand your argument here, but I disagree. If centers are playing more games and scoring more PPG, then I would argue they are more successful. I don't agree with the idea that all positions HAVE to be equal when it comes to successful drafting. Goalies are a great example: there have been a disproportionate number of bad picks in the first that have been goalies, so it makes sense to me that their averages should be lower. I get your argument, but I don't plan on changing this.

that said, you could use that info to dissect team performance in a different way, with a 'position-neutral' team score.. i.e. right now a good team score is either due to drafting good players, drafting more forwards and less goalies, or both. you could re-weight each team score and get some theoretical score the team would have achieved if it had drafted X forwards, Y d-men and Z goalies (X,Y,Z being league averages over 98-07 not actuals for that team).

Similar to the above, I don't think this is fair to teams that were smart enough to, say, not pick goalies and instead take centers. It all comes down to whether you believe one position to produce better picks than another, and I do.

3) I don't understand the strange geographical choice to include Slovenes (and I guess Croats... who else? was a Serb ever drafted?) in the "Russian block", countries from ex-Yugoslavia are historically and geographically further removed from Russia than the Baltic States as well as Czechs/Slovaks/Poles/etc. anyway I doubt it changes anything but thought I'd be a nerd and point it out, I would have made a geographical split between the historical Euro powerhouses (Sweden, Finland, Russia with or without Belarus, Czech Rep, Slovakia) and other countries (Switzerland, Denmark, Norway, Slovenia, Austria, Poland, Germany, etc).[/QUOTE]

This is just stupidity/laziness on my part. I am awful with European geography. If you break down what you would consider "The Russian Block" and "Europe", I'd be happy to re-run that and give new averages. I'm not expert in that area and just made a dumb mistake.

Doctor No said:
For the games played metric, I agree that the number of eligible seasons needs to be taken into consideration. I'd recommend developing a baseline expectation, with forwards/defensemen/goalies on separate scales, and using that instead of just a straight season count. I'd also adjust for the seasons of different length over the past few years.

I did weigh the averages differently, so goalies aren't expected to play the same amount as forwards each year. To get the weights, I just kind of eyeballed it until I saw a decent mix of each position. I would love to say I had a more exact method, but it was all just changing the weights on GP and SV%.

That said, I do there is room for improvement in my goalie and defensemen rankings. I REALLY want to us TOI/Game for defensemen, but can't find that in a format that wouldn't require me to manually search for every single defenseman's value. Goalies...I'm not happy that Fluery is the most "successful" goalie in this decade, but not matter how I change weights, it's the same. You're the goalie expert (I played tender growing up, but haven't done much data analysis on the position), so if you have a better way to quantify success, I'm very open to it.

I'll try to make a few of these adjustments (regression line, Russian Block fix) over the next couple days and post updates, keep the ideas coming!
 

barneyg

Registered User
Apr 22, 2007
2,383
0
I understand your argument here, but I disagree. If centers are playing more games and scoring more PPG, then I would argue they are more successful. I don't agree with the idea that all positions HAVE to be equal when it comes to successful drafting. Goalies are a great example: there have been a disproportionate number of bad picks in the first that have been goalies, so it makes sense to me that their averages should be lower. I get your argument, but I don't plan on changing this.

I don't disagree with your point regarding goalies. I was mostly suggesting this as an additional analysis not a replacement -- is a team's good score solely due to drafting more centers than the others and avoiding goalies, or is it due to picking better players at each position? Both potentially define "good drafting" but those are different arguments.

I do disagree with your point when it comes to forwards vs. d-men. Forwards, esp. centers it seems, clearly compile more points than d-men, but if you only draft scoring forwards you end up like the Oilers. I see two solutions here -- either dumb the scoring down to GP only, or get a more in-depth formula for 'value' here i.e. with TOI or something else (which might be complicated if you used hockeydb.com for career stats). I don't think using GP only would change the rankings all that much but it would correct for the bias against d-men that you currently have. Using GP only is clearly leaving useful data on the table but at least you don't know which direction the bias is going at the team level.

This is just stupidity/laziness on my part. I am awful with European geography. If you break down what you would consider "The Russian Block" and "Europe", I'd be happy to re-run that and give new averages. I'm not expert in that area and just made a dumb mistake.

I don't think geographical groupings are that interesting, instead as I wrote earlier I would separate Euro countries between "mainstream" (Sweden etc) and "emerging" (Slovenia, Denmark etc). In other words is there a way to evaluate teams' drafting strategies here -- are they better at spotting talent in countries where everybody already is, or are they better at spotting the Anze Kopitars or Mark Streits or Lars Ellers i.e. future stars in non-traditional countries.
 

Top 6 Spaling

Registered User
Jun 23, 2010
12,341
219
Smashville
Upated top and bottom five teams with log regression used instead of mean:

Top 5
Montreal (2.719)
San Jose (2.428)
Toronto (1.940)
Detroit (1.916)
Pittsburgh (1.892)

Bottom 5
Phoenix (-2.872)
Tampa (-2.074)
Florida (-2.026)
Washington (-1.706)
Calgary (-1.361)

Toronto jumped all the way from 8 to 3, which chocked me. Buffalo falls out of the top 5. Washington takes the biggest tumble down to #4, and Tampa moves below Florida.
 

Top 6 Spaling

Registered User
Jun 23, 2010
12,341
219
Smashville
I don't think geographical groupings are that interesting, instead as I wrote earlier I would separate Euro countries between "mainstream" (Sweden etc) and "emerging" (Slovenia, Denmark etc). In other words is there a way to evaluate teams' drafting strategies here -- are they better at spotting talent in countries where everybody already is, or are they better at spotting the Anze Kopitars or Mark Streits or Lars Ellers i.e. future stars in non-traditional countries.

I like it a lot. Before I run it, I'm curious what your thoughts are on these groupings:

Established: Sweden, Finland, Czech

Emerging: Everyone else
 

barneyg

Registered User
Apr 22, 2007
2,383
0
I like it a lot. Before I run it, I'm curious what your thoughts are on these groupings:

Established: Sweden, Finland, Czech

Emerging: Everyone else

Add Russia to established (or separate them altogether). Not sure where Slovakia should be, I guess the easy thing to do is to have some arbitrary cutoff based on the number of 1998-2007 draftees.
 

Mickey Marner

Registered User
Jul 9, 2014
19,649
21,395
Dystopia
As a Leafs fan, I'm shocked Toronto is so high. Antropov, Ponikarovsky, Steen, Stajan, Kulemin, Gunnarson and Reimer are basically the cream of the crop from that time period. A whole lotta' meh if you ask me.

Anyhow, interesting and informative read nonetheless.
 

67Leafs67

Registered User
Nov 8, 2014
774
631
As a Leafs fan, I'm shocked Toronto is so high. Antropov, Ponikarovsky, Steen, Stajan, Kulemin, Gunnarson and Reimer are basically the cream of the crop from that time period. A whole lotta' meh if you ask me.

Anyhow, interesting and informative read nonetheless.

As a fellow Leaf fan, I would guess that the ability to find good players in late rounds is what helps boost Toronto. Just speculation, but:
- Alexei Ponikarovsky went 87th overall (650+ games played)
- Kyle Wellwood (450+ games played) was 134th overall
- Ian White (500+ games played) was 191st overall
- John Mitchell (400+ games played) was 158th overall
- Anton Stralman (475+ games played) was 216th overall
- Leo Komarov (100+ games played) was 180th overall
- Viktor Stalberg (330+ games played) was 161st overall
- James Reimer (170+ games played) was 99th overall
- Carl Gunnarsson (350+ games played) was 194th overall


To the OP, this is great work, really cool numbers to look at. No pressure, but it would be cool to do the numbers by GM as well!
 

Top 6 Spaling

Registered User
Jun 23, 2010
12,341
219
Smashville
As a fellow Leaf fan, I would guess that the ability to find good players in late rounds is what helps boost Toronto. Just speculation, but:
- Alexei Ponikarovsky went 87th overall (650+ games played)
- Kyle Wellwood (450+ games played) was 134th overall
- Ian White (500+ games played) was 191st overall
- John Mitchell (400+ games played) was 158th overall
- Anton Stralman (475+ games played) was 216th overall
- Leo Komarov (100+ games played) was 180th overall
- Viktor Stalberg (330+ games played) was 161st overall
- James Reimer (170+ games played) was 99th overall
- Carl Gunnarsson (350+ games played) was 194th overall


To the OP, this is great work, really cool numbers to look at. No pressure, but it would be cool to do the numbers by GM as well!

That's a great idea! It'd be a ton of legwork, but if I can find a decent way to do it, I will.

Toronto wasn't rated highly because of any great successes, but because they avoided any big busts. Here are their top and bottom 10 picks: (Using the new fancy regression difference instead of average!)

qzqUDFg.jpg


Stralman, their best pick, was #33 in the league, so they really lacked a superstar. Luca Cereda was only the #38th worst pick though. They avoided the extremes and generally had some good picks in the late rounds. Quantity of success, not quality.
 

tempofound

Registered User
Oct 18, 2013
358
202
I've always thought Leafs drafting is better than their fans seem to think. Their trouble has been trading away their firsts and young players.

Like in the timespan covered here, the Leafs only had 7 first round picks in 10 years, Their highest picks were #10, #13, #17. Given that it's not surprising that their picks don't look good. But it's not really the quality of drafting that's been poor.

And the good players they drafted, Steen, Rask, Strålman, are now elsewhere.
 

schuckers

Registered User
Feb 21, 2013
80
0
One thing you should look at is correlation between two subsets of your data. For example you might take team AvgDiff for 1998 to 2002 and team AvgDiff for 2003 to 2007 and look at the correlation between these. (Alternatively you could randomly split picks by team into two groups and look at the correlation, though doing so by year is likely easier).

That should let you know whether or not the results you get are due to chance or otherwise.
 

Ad

Upcoming events

  • Gold Coast Suns @ Brisbane Lions
    Gold Coast Suns @ Brisbane Lions
    Wagers: 3
    Staked: $36,790.00
    Event closes
    • Updated:
  • Cagliari vs Lecce
    Cagliari vs Lecce
    Wagers: 2
    Staked: $25.00
    Event closes
    • Updated:
  • Osasuna vs Real Betis
    Osasuna vs Real Betis
    Wagers: 2
    Staked: $85.00
    Event closes
    • Updated:
  • Empoli vs Frosinone
    Empoli vs Frosinone
    Wagers: 1
    Staked: $10.00
    Event closes
    • Updated:
  • Hellas Verona vs Fiorentina
    Hellas Verona vs Fiorentina
    Wagers: 1
    Staked: $10.00
    Event closes
    • Updated:

Ad

Ad