NHL Draft Software: 2018 Top100 End of the regular season rankings

[ProspectsFanatic]

First--yours is my favorite of all the "top prospects" lists on HF. Specifically because you use data.

I was glad to see you have Wassenius as a high 3rd rounder. I have mentioned him several times. And yes, I am a stat watcher. Because stats are objective. In fact, my only question to you would be related to this.

I agree on the aspects you noted and really like that you take strength of league and line mates into consideration. Those both make a big difference. Where I am not so sure is in the "subjective" measures of skating, shooting, puck control, and playmaking. Because the difference in any of those individually or all combined must surely show up in goals/assists. Over any season of 30 or more games luck should be mitigated. With luck minimized, an excellent skater with a very good shot will score more goals; an exceptional puck handler with excellent playmaking ability will have more assists. So either the subjective measures become redundant or introduce bias to explain why players that scouts really like aren't more productive.

Again I really like that you have created a model and I think it will consistently outperform subjective rankings. I think you might "trust" it a little more by letting the data stand on their own.

I am glad to hear you think I have the best approach.
I will truly cover everything on this thread, it is quite interesting that I was arguing with the previous poster the value of data over the eye test, now I will be arguing the opposite with you. Finding the right balance isn't obvious. But, I see that for the most part we vastly agree. What you said is very reminiscent of something similar I stated in the past, coming to similar conclusions as you:
"How do I evaluate players? I mainly look at statistics [...] Things like IQ and shooting don’t need a further look because they should translate into the criteria I value the most, statistics."

I still believe it to be true for the most part, but only for the most part. "Over any season of 30 or more games luck should be mitigated." I found this not to be true for that small of a sample size. Most players produce as they should, but there is always some outliers who either overperformed or underperformed during a season. In order to avoid being misled by unrepresentative numbers, outliers must be adjusted within a reasonable range.

For example, everyone knows that William Karlsson is probably not a consistent 40+ goals scorer even though he has been able to achieve that plateau this season. And that's where the evaluation of his overall offensive abilities, particularly the shooting category, and the data entered about the shooting percentage comes into play. The software would read that 23.3% is not a sustainable shooting ratio for such player, and subsequently pulls his goal total a bit lower to put him within a reasonable range of what you would expect out of that player. The reverse is also applicable, Lehkonen only had 2 goals in 36 games in the first half of this season, if on top of that you start factoring in that he was having quality ice time he would rank worse than basically any player in the league, but by looking at his shooting percentage that must have been around 3-5% it becomes evident that his 2 goals total is abnormal and unrepresentative. It feels necessary to me to make those slight adjustments in order to have the most representative results possible, but the reality is that most players produce as they should, so little adjustments are actually being made.

So that covers statistics adjustments, then there is also the projection dimension; over the years it became apparent to me that the game of faster skaters translates better at the next level. No matter how great of playmaker or shooter you are those strengths evaporates if you are unable to create time and space for yourself which becomes harder to obtain as you are moving up the leagues with players being faster and bigger. You have to greatly value the skating abilities of players in order to make good projections, otherwise stats watchers are greatly at risk of having players like Rychel and Shinkaruk in the top15 in the 2013 draft, a mistake I made in the past, but that I have learned from.

I have been speaking a lot about how I have been using the subjective evaluations in this thread, but what I would like people not to misinterpret is that I am mostly valuing statistics, I value statistics probably more than any other list, but I am also of the belief that you need to make small adjustments based on the viewing of players in order to obtain the best scouting results.

 

[vendetta]

Man this is good stuff.. thank you

 

[vendetta]

Does this software do overagers that aren't draft eligable for free agent purposes?

 

[alko]

Dahlin has 114,15 points. How it looks like with Jack Hughes (2019) or Alexis Laferniere (2020)?

And just curious. How was Connor McJesus on this list?

 

[emptyNedder]

I have been speaking a lot about how I have been using the subjective evaluations in this thread, but what I would like people not to misinterpret is that I am mostly valuing statistics, I value statistics probably more than any other list, but I am also of the belief that you need to make small adjustments based on the viewing of the players in order to obtain the best scouting results.[/QUOTE]


Thanks for the great response. The funny thing is I don't come at this from a hockey perspective. I have only become deeply interested in hockey the past two seasons--after spending almost a decade trying to understand the economics of the Great Recession. What was interesting about the economic debate is that some pretty basic models (IS-LM for instance) did a much better job of explaining what happened and, much more importantly, what needed to be done than serious economists with subjective opinions--like the need for everyone to "live within their budget."

Anyway, I agree that there is a need for some adjustments. I would think one adjustment based on the ability to translate overall skills to the next level (perhaps a factor that weighted skating {though I find it unusual that even the combine doesn't test for on-ice speed} and size combined). So that a player who had an overall "low" adjustment might not be included in the first two rounds.

Again, great stuff. For me at least, it makes hockey that much more interesting.

 

[TheGoldenJet]

I can see how it can be difficult to grasp how I am using the subjective evaluations so here are further explanations; I rate players attributes which is converted into an overall score, this score is sort of used as an expected value to remove anomalies in statistical performance (if the player overperformed or underperformed base on what you would expect). Those evaluations are also seen by the software sort as a projection of the player skills, affecting the final score again at the end, you can often see players with good stats but bad skating fall in the draft, so I particularly value skating so the software can match that. So it matters that I watch the players in order to have the appropriate subjective evaluations. If you were to be using the software you might obtain different results, but it should normally tend to look like mine.

So are you using scouts’ opinions for things like ‘hockey IQ’ and ‘skating’, or your own? How do you quantify (give numerical value to) what a hockey scout has said or written?

Like your list a lot, Tkachuk and especially Denisenko seem significantly too low to me but that’s a small nitpick in a list of 100 players.

 

[robochicken36]

Incredible work! I remember you once made it available to download so we could enter our own players and test it. Any chance you could make it available again?

 

[ProspectsFanatic]

So are you using scouts’ opinions for things like ‘hockey IQ’ and ‘skating’, or your own? How do you quantify (give numerical value to) what a hockey scout has said or written?

Like your list a lot, Tkachuk and especially Denisenko seem significantly too low to me but that’s a small nitpick in a list of 100 players.

Well it depends, on players I have watched a lot I am mostly using my own opinion over the scouts opinions and vice versa. I rate the players from a scale of very bad to exceptional, then according to the algorithm this is converted into a value.

 

[ProspectsFanatic]

Incredible work! I remember you once made it available to download so we could enter our own players and test it. Any chance you could make it available again?

Building a website with the software embedded in it where people can experiment with the software is what I am aiming for at the moment, not sure when it will be available. Looking for a programmer who can help me.

 

[ProspectsFanatic]

Dahlin has 114,15 points. How it looks like with Jack Hughes (2019) or Alexis Laferniere (2020)?

And just curious. How was Connor McJesus on this list?

Your the 2nd one asking me so I did McDavid, he scores around 135. Maybe one day I'll do a ranking of top prospects on their draft day could be fun.

 

[Aerrol]

Not to spam, and maybe I've missed the answer, but I'd also like to know:

Do your numbers compare year on year? If so, how do other year's top prospects come out? McDavid, Eichel, Barkov, Laine, Puljujarvi, etc.

I imagine this might not be doable as you might not have compiled datasets for prior years but I did see you go back a while for Vancouver's picks, so...

Edit: I see you've answered this already lol, sorry. I would love a top prospects year on year comparison.

 

[emptyNedder]

See that you have Linus Karlsson at 75--haven't seen him much on other lists. In this case, I think numbers (being 2nd highest scorer in SuperElit) indicate competence. He has decent size and about an even number of goals and assists.

I would be interested in those scouts/informed fans who don't think he is draft worthy what are their reasons.

 

[emptyNedder]

The software would read that 23.3% is not a sustainable shooting ratio for such player, and subsequently pulls his goal total a bit lower to put him within a reasonable range of what you would expect out of that player.

I like how this quote relates to my last post about Linus Karlsson (will get to William Karlsson in a few sentences). Linus K has been a productive goal scorer at multiple levels. So despite not having any "wow factor," he deserves consideration. I like how you factor in multiple years because at some point repeated success is more than just stats.

As far as William Karlsson--you stated "the software" would adjust. That was my point from the earlier discussion. No need to subjectively state he is not an exceptional skater or that he is just an average puck-handler. The software understands regression to the mean and based on past performance will correct. Experts don't always do that: think about how many times players have been give big/long contracts based on one season. For those who haven't read it, "Thinking Fast and Slow" opens with an anecdote that should make us all lower our own expert opinions.

Don't get me wrong, I think you are correct to make adjustments. From my standpoint with a limited understanding of formal statistics, you are using consensus opinion as a form of Bayesian inference. Since what you are analyzing are prospects who are developing it definitely has its place.

Again, I really think you are on to something. Keep it up!

 

[Mallard]

Fantastic job man. Really puts a nice perspective on these prospects.

Nice to see Bokk high up there.

 

[ProspectsFanatic]

Fantastic job man. Really puts a nice perspective on these prospects.

Nice to see Bokk high up there.

Bokk is a tricky one because he only has one year of relevant stats, making the sample size of data to analyze small, could likely be higher or lower in the rankings.

 

[Kevin Musto]

Where does Simon Johansson rank?

 

[ProspectsFanatic]

Where does Simon Johansson rank?

I am not sure, I will do a complete list with overagers when playoff ends.

 

[VIPettersson]

fantastic read thanks

 

[PAZ]

This is amazing, I love it! I do have a few questions:

- I noticed you're adding scouting reports and tweaking the numbers for certain players, but what happens when scouting report have different/mixed opinions on say, skating?

- Also, (sorry if this was already asked) how much weight is put into the D-1 and D-2 years in the case of risers? For example, i'm curious how your software came to the conclusion Makar was the #2 defenseman based on playing in a non-existent league (for your software) with no international play?

- What do you think the biggest flaw in your software is?

 

[ProspectsFanatic]

This is amazing, I love it! I do have a few questions:

- I noticed you're adding scouting reports and tweaking the numbers for certain players, but what happens when scouting report have different/mixed opinions on say, skating?

- Also, (sorry if this was already asked) how much weight is put into the D-1 and D-2 years in the case of risers? For example, i'm curious how your software came to the conclusion Makar was the #2 defenseman based on playing in a non-existent league (for your software) with no international play?

- What do you think the biggest flaw in your software is?

- I try to watch every player myself and come to my own judgement, but if I haven't seen a certain player I go for a middle ground or stick to the opinion of the report I trust the most.

- It is calculated automatically according to the algorithm. If game played and tournament play is all equal it is 67%, 23%, 10%, if the game played varies it is automatically adjusted.

- One of the strengths of the software is that it is incredibly consistent in evaluating/comparing players, it values the different metrics equally for all players, for example, you can't adjust the point production with the age of all players perfectly off the top of your head like the software does. But on the flip side, if the software isn't well programmed it can also be his biggest flaw, if I overvalue the positive impact of being of younger age, all younger players will benefit while all the older players will suffer, it is very challenging to pinpoint everything accurately. So this is potentially a pretty huge flaw, but at the same time software or not we are all bias, for example, some value size more than other while some value a certain league more than others. That being said, by doing those type of lists every year, I can look back on my results, tweak the algorithm and come closer to perfection.

___
On a side note, I watch a lot of WJC18 games and I now have a better scouting evaluation on many draft eligible players. So expect changes in the final rankings (with playoff stats and WJC18 performance also being added and affecting the results).

 

[Kevin Musto]

How's the overager list coming along? Also when should we expect final rankings?

 

[ProspectsFanatic]

How's the overager list coming along? Also when should we expect final rankings?

My final list is pretty much ready, I will be posting it after updating player height/size with this week combine. Unfortunately, I didn't have time to work on the overager list though, I spent a lot of time revamping the software so analytics take an even greater proportion over the scouting evaluations. I originally thought people wouldn't respect my methodology to a certain extent because I was looking too much into data, but feedbacks I received was actually opposite which I am glad because I share the same perspective. So expect the list to be a bit more out of line with the consensus, but very telling of you believe in analytics.

 

[Kevin Musto]

My final list is pretty much ready, I will be posting it after updating player height/size with this week combine. Unfortunately, I didn't have time to work on the overager list though, I spent a lot of time revamping the software so analytics take an even greater proportion over the scouting evaluations. I originally thought people wouldn't respect my methodology to a certain extent because I was looking too much into data, but feedbacks I received was actually opposite which I am glad because I share the same perspective. So expect the list to be a bit more out of line with the consensus, but very telling of you believe in analytics.

Perfect. Thank you for all your hard work!