Getting 4 regular NHLers out of your prospect pool would be a huge success, statistically speaking. Criticising the model on those grounds suggests an unrealistic understanding of how many drafted prospects turn into NHL players.
Does it though?
It would surely be interesting to try and create a team that's drafted solely based on such models.
Dmitri Klopov over Drew Doughty?
Axel Holmström over Dylan Larkin?
Cody Corbett over Hampus Lindholm?
Ryan Pilon over Thomas Chabot?
David Kvasnicka over Cale Makar?
Ryan Siedem over Moritz Seider?
Extreme examples but it's not hard to find.
My impression is that when those models are correct, they're correct by a little bit (like ranking someone drafted in the 2nd/3rd as a 1st round caliber talent). But when they're wrong, they're extremely wrong.
HABS
1. Cole Caufield
2. Kaiden Guhle
3. Logan Mailloux
4. Luke Tuch
5. Jan Mysak
6. Jordan Harris
7. Mattias Norlinder
8. Jesse Ylonen
9. Riley Kidney
10a. Jayden Struble
10b. Sean Farrell
Well your examples basically prove my point, that the model's "successes" are minor (such as having Suzuki above Rasmussen when they were both rated as top 6 of that draft), while it's errors can be insane, i.e. giving Cale Makar close to a 4th round ranking.Picking anecdotes doesn't say much. I could easily pick many examples the other way.
Claude Giroux > James Sheppard.
Nylander/Ehlers/Pastrnak > Virtanen/Fleury
Brayden Point > Connor Bleakley
Alex Debrincat > Riley Tufte
Nick Suzuki > Michael Rasmussen
Mat Barzal > Zach Senyshyn
Kyle Connor > Jakub Zboril
Sebastian Aho > Mitchell Stephens
Connor Garland > Devante Stephens
You would need to look at backward casts of many drafts to make a determination. And though most NHLe models are primarily points-based, they do adjust for age and size (as Top Down Hockey's model does). The point isn't that they're the end-all of drafting. It's that if a team can't consistently out-perform a model based on limited information, that's a pretty pathetic indication of the club's scouting.
IIRC, Canucks Army created a "model" that literally just drafted the highest scoring draft eligible CHL player. They excluded all other leagues from being selected to the model's team, including European leagues and USHL/college. They looked at like 15 drafts and the model still rated in the middle of the pack in terms of selecting regular NHLers.
Well your examples basically prove my point, that the model's "successes" are minor (such as having Suzuki above Rasmussen when they were both rated as top 6 of that draft), while it's errors can be insane, i.e. giving Cale Makar close to a 4th round ranking.
If a calculator gave you good answers 50% of the time, but 50% of the time calculated 1+1 to be 17, that would be known as a broken calculator.
It's also telling that you use mainly forwards for your examples; my original argument being that this type of model serves no purpose when ranking d-men. On the forward side there's more of a case to be made that it's not useless. I'm guessing the Canucks Army example ended up with a team of some decent forwards but zero d-men?
Mysak a bit high; Tuch too.
My list would be the following
1. Caufield
2. Guhle
3. Harris
4. Mailloux
5. Norlinder
6. Ylonen
7. Struble
8. Primeau
9. Mysak
10. Tuch
Does it though?
It would surely be interesting to try and create a team that's drafted solely based on such models.
Dmitri Klopov over Drew Doughty?
Axel Holmström over Dylan Larkin?
Cody Corbett over Hampus Lindholm?
Ryan Pilon over Thomas Chabot?
David Kvasnicka over Cale Makar?
Ryan Siedem over Moritz Seider?
Extreme examples but it's not hard to find.
My impression is that when those models are correct, they're correct by a little bit (like ranking someone drafted in the 2nd/3rd as a 1st round caliber talent). But when they're wrong, they're extremely wrong.
Melvin on the Canucks board created his "potato" drafting model based solely on points/NHLe. It drafts better than quite a few NHL teams.
On this topic, may I please have a link to this thread?
Kings:
With Bjornfot, Anderson, Vilardi graduating, I'd say
Quinton Byfield
Brandt Clarke
Alex Turcotte
Arthur Kaliyev
Brock Faber
Sammy Fagemo
Tyler Madden
Francesco Pinelli
Helge Grans
Akil Thomas
HM
Rasmus Kupari, Jordan Spence, Kim Nousiainen, and like 30 more 'B' prospects and i'm sure someone i'm forgetting haha
Sticking to players under 22 who haven't played in the NHL, since I consider 24-year-olds who haven't made the NHL yet to be depth players and not prospects, the Flames' prospect power ranking looks something like this:
1. Matthew Coronato
2. Connor Zary
---
3. Jakob Pelletier
4. Dustin Wolf (my favourite prospect personally, but goalies are voodoo and I'll wait to see how he transitions to the AHL before I proclaim him our next saviour)
5. Jérémie Poirier
---
6. Ryan Francis
T-7. William Strömgren
T-7. Cole Jordan
9. I'm running out of plausible/non-embarrassing names here... let's say Emilio Pettersen
10. Yan Kuznetsov?
The top 5 is pretty clear to me and after that there's a whole lot of huh and maybe.
Not a fan of Grans.
But, and I mean this with respect...**** You guys. It's a haul.
Well your examples basically prove my point, that the model's "successes" are minor (such as having Suzuki above Rasmussen when they were both rated as top 6 of that draft), while it's errors can be insane, i.e. giving Cale Makar close to a 4th round ranking.
If a calculator gave you good answers 50% of the time, but 50% of the time calculated 1+1 to be 17, that would be known as a broken calculator.
It's also telling that you use mainly forwards for your examples; my original argument being that this type of model serves no purpose when ranking d-men. On the forward side there's more of a case to be made that it's not useless. I'm guessing the Canucks Army example ended up with a team of some decent forwards but zero d-men?
No the peeve is the way this stuff is starting to get passed around as established science/fact, reaching people who don't understand how the models are built and what their flaws are, nor their track record.You're assuming that people who use such models are imbeciles (or that they can't differentiate between a defenseman and a forward). They don't have to be.
Things like this are very cool, but as expected it makes some really great forward selections (a theoretical draft year where you land Nylander, Pastrnak, Point and Arvidsson) and mostly fails to find any d-men (or goalies of course).I think this is what the poster was talking about...
The Data-Based Drafting Thread (what players would a Potato pick?)
I consider your examples mostly small victories for the model and they kind of get replaced by errors that are just as big if not bigger (in 2015 for example underrating Chabot, Werenski, vastly overrating Bracco).These are just names I pulled glancing at hockeyDB. I could find weirder comparisons if I wanted. But selecting a guy like Rasmussen vs. Suzuki is a massive miss. You've got, at best, a bottom-six player vs. a high-end piece. And further, the rest of the "misses" again are either replacement level players or not in the NHL. These are massive misses. You just choose to ignore that too. Pairwise comparisons are also a very poor way to evaluate draft success. You would need to evaluate multiple entire drafts, as I already said.
These are just names I pulled glancing at hockeyDB. I could find weirder comparisons if I wanted. But selecting a guy like Rasmussen vs. Suzuki is a massive miss. You've got, at best, a bottom-six player vs. a high-end piece. And further, the rest of the "misses" again are either replacement level players or not in the NHL. These are massive misses. You just choose to ignore that too. Pairwise comparisons are also a very poor way to evaluate draft success. You would need to evaluate multiple entire drafts, as I already said.
The Potato Pick concept isn't trying to be the "best drafter"... it isn't even really trying at all, just selecting based solely upon weighted stats. The ideas was to see if, based upon a few basic criteria, it could do better than Benning (or Gillis). IIRC, the Potato's picks are around middle of the pack... some hits, some misses, but arguably better than a good chunk of NHL GM's and scouting staffs.
No the peeve is the way this stuff is starting to get passed around as established science/fact, reaching people who don't understand how the models are built and what their flaws are, nor their track record.
In my estimation, the people creating the models and visualisations are a bit overconfident in their work and seem to happily ignore sample size issues or the fact there are obvious flaws in the way the model rates d-men especially.
Things like this are very cool, but as expected it makes some really great forward selections (a theoretical draft year where you land Nylander, Pastrnak, Point and Arvidsson) and mostly fails to find any d-men (or goalies of course).
I consider your examples mostly small victories for the model and they kind of get replaced by errors that are just as big if not bigger (in 2015 for example underrating Chabot, Werenski, vastly overrating Bracco).
While drafting Suzuki ahead of Rasmussen would be the right move, there are far too many examples where the model rates players highly who go undrafted and don't even come close to sniffing NHL time.
1. Vitali Kravtsov
2. Nils Lundkvist
3. Braden Schneider
4. Zac Jones
5. Brett Berard
6. Matthew Robertson
7. Morgan Barron
8. Lauri Pajuniemi
9. Tarmo Reunanen
10. Brennan Othmann