Say What
Building a Legacy 4/28/96 Never again!!
- Jan 18, 2015
- 817
- 78
There's no monopoly or lone wolves. Scouts aren't making any decisions on their own.
Prospect Info: Logan Stanley - Part III
- Thread starter Romang67
- Start date
There's no monopoly or lone wolves. Scouts aren't making any decisions on their own.
Found it:
https://canucksarmy.com/2014/06/26/sham-sharron-takes-over-all-30-draft-tables/
Im sure you've seen it. It's far from perfect and I'd rather see a weighted ranking of final value of each player after their "fully developed", and then test your order vs that ranking. This what we are doing.
We also include dmen, include euros, and do not restrict to the next 30 on the css list (though I'd be fine doing this as again we're talking about combining analysis with actual scouts).
The cham charron 2.0 redux where the bucketed the players. To teams and assigned scores vs the team and the team that drafted.
Once we get our shit fixed I'd like to post our lists to.
There are teams that would definitely be taking a step back using a system like this, but there are many that would benefit.
My stance is largely not that these systems should replace scouting in any capacity, but that we should be using them as the absolute baseline/starting point.
Do whatever we're doing now, but starting from here.
Last edited:
Basic model was worse than the large majority of teams despite only drafting forwards ( no goalies or D). Not sure that's a realistic approach, and it is lower risk than a balanced drafting mix.
Only performed better than the majority of teams if they restricted to prospects that scouts had already prioritized.
yeah. there's progress. but i still don't see what the finished work will look like (and i don't hate him like other posters do) and how it will get there. lots of odd little gaps in his game.
He has increased his chances of success as a player due to his cohort.
Players that performed similarly to him in his draft year succeeded 25% of the time. "Expected development" would mean those probabilities remained fat, that his Dy+2 cohort was still a 25% success rate.
Due to his increased scoring, his cohort has improved to a 42% success rate. 42% of similar players in height and production in they Dy+2 season were successful.
Ex, 7 nhle and being 6'7 in Dy is a 25% success cohort. Similarly 12 nhle and 6'7 in Dy +2 is a 25% success cohort. 32 nhle and being 6'7 (his current pace) in Dy+2 is 42%.
(The second number is made up, I don't know off hand what stats would pull a 25% Dy+2 cohort.)
Which im fine with. That reasonable facsimile for the combined approach.
The point being is that first iteration shouldn't out draft anyone. Beating a single team means there's a terrible ineffeciency in their scouting department. Think of how much a scouting department costs. Couple million a year? That's like me paying for gasoline only to find out my car runs just as good off tap water.
Who knows where those guys models are at now that they've actually got backed by a team, but those numbers paint a pretty solid POC, as do the initial return s we've hit (better rsquared values then pure draft order and that's without a cis ranking check, though cis lists are the starting pool)
garret9
AKA#VitoCorrelationi
Sham Sharron was a model to show how bad drafting is because it’s a bad model and does comparably well.
It’s not supposed to be a good model.
Heck it excludes all non CHL players!
It’s not supposed to be a good model.
Heck it excludes all non CHL players!
So another big defenseman drafted in the middle of the first round a few years ago had PPG each season in junior looking like this:
Draft -1 - .35 PPG
Draft +1 - .57 PPG
Draft +2 - .84 PPG
His points per game improved by a similar trajectory as Stanley over a 3-year period. He followed that up with the following PPG in the AHL:
Draft +3 - .25 PPG
Draft +4 - .22 PPG
Obviously, there is so much more in determining a player's worth than looking at PPG, especially a defenseman...like how does he play defensively? And the PPG is rather simplistic in itself.
However, it is rather troubling that this player did not see any time with the parent club in his draft +4 year, particularly since this team was one of the worst clubs in goals against last season and was known for playing awful defense. You would think he would have received a shot with the big club to see what he could bring to the table but no.
He is in his draft + 5 year and does not appear any closer to reaching the show. And this is despite showing some very promising improvement in his offensive contributions in his draft +1 and +2 seasons. Something to think about when looking at Stanley's stat line and perhaps temper one's expectations with regards to these improvements.
Draft -1 - .35 PPG
Draft +1 - .57 PPG
Draft +2 - .84 PPG
His points per game improved by a similar trajectory as Stanley over a 3-year period. He followed that up with the following PPG in the AHL:
Draft +3 - .25 PPG
Draft +4 - .22 PPG
Obviously, there is so much more in determining a player's worth than looking at PPG, especially a defenseman...like how does he play defensively? And the PPG is rather simplistic in itself.
However, it is rather troubling that this player did not see any time with the parent club in his draft +4 year, particularly since this team was one of the worst clubs in goals against last season and was known for playing awful defense. You would think he would have received a shot with the big club to see what he could bring to the table but no.
He is in his draft + 5 year and does not appear any closer to reaching the show. And this is despite showing some very promising improvement in his offensive contributions in his draft +1 and +2 seasons. Something to think about when looking at Stanley's stat line and perhaps temper one's expectations with regards to these improvements.
Aavco Cup
"I can make you cry in this room"
- Sep 5, 2013
- 37,630
- 10,440
That does not mean Stanley will follow a similar path. A first Rd pick will be given much more opportunity to succeed. I suspect Stanley will play games in the NHL at some point. How effective he becomes is another question altogether
I agree. I just don't understand why people keep on saying that out-performed most NHL teams, because it didn't.
That's very simplistic. Drafting just high scoring CHL forwards might give decent outcomes if you are looking primarily at games played and point production in the NHL, but not a realistic strategy.
I don't doubt that there are inefficiencies in NHL scouting, but the comparative analysis is woefully simplistic and assumes that NHL teams ignore those statistical metrics. I highly doubt that.
Why? Why do you doubt that? We see teams consistently make decisions you would only make if you were completely ignoring this information.
We can say for sure the Florida panthers weren't, considering they hired the PCs crew.
Colorado hired a similar crew as well.
I know first hand that teams are now hiring and exploring this area (meaning at minimum many weren't before, or they wouldn't be hiring people, they'd have them).
Are these models good? f*** no. Are many teams drafting better? Hell yes. That's not the issue. The issue is they should be. They NEED to be. They're spending millions of dollars a year to be better.
The fact that a painfully simple model DID outdraft several teams (outdrafting two teams qualifies as outdrafting NHL teams, I never said all teams) is no different then if I showed up at work today and told my boss the entire accounting department is providing no ROI and a simple spreadsheet will perform just as good. THATS INSANITY. But you've heard all this before.... So what's the issue?
Sham Sharron is not our model. Our model is a more primitive version of PCS.
Our model does work for and includes dmen (nhl value was determined by gp, avgtoi and pts, using NHL coaches "trust"(toi) as a facsimile for NHL ability) our initial runs identified dmen much better (relative to league aggregate) then it did forwards (it was still bad, but was close to 3x better then draft order). To be clear it identified dmen worse then forwards, but as badly as the league in aggregate does.
Anyways, im not trying to peddle the cure all for drafting. This isn't some snake oil that's going to replace every scouting department.
This is a simple analysis that prior to sham Sharron I don't think anyone did, an actual measurable look at how effective teams are at drafting and is it providing the ROI they spend vs free resource (like basic math!)
It's not a surprise, given the timelines and turnover in scouting department, player development, and GMs, it'd be extremely difficult to performance manage and measure your internal/scouting drafting ability.
All im getting at is, hey, this simple model actually did a half decent job reletive to teams. Clearly then, it has at least similar value/worth/credibility as the scouting departments it beat. It also clearly has more value then the straight up third party css scouting list. Maybe then this should be the starting point if discussion and scouting instead of a CIS list.
Additionally, if these inputs and factors are able to build a better weighting on draft day, and have gone on to even identify overage players well (not sure if sham did this but PCs did and so does ours) then it can probably give a reasonable facsimile for player progression post draft.
You literally asked how do big dmen that score a point per game in d+2 pan out reletive to small scorers. All I did was pull all that information and communicate it to you.
That's not even a model, that's just the actual history.
So at the end of the day, what's the issue? What do you need to see? I don't claim this is better then every NHL teams drafting. I don't claim scouts are useless and need to be replaced. I don't claim this is a feasible practical alternative to every NHL teams scouting department.
All im saying is, these methods (comparing performance and career paths of similarly sized and scoring players) are already proven to be more predictive then many scouting lists. They clearly have value.
Everyone's welcome to trust their own opinion or what the css list said or the latest talking head on TV, but it'd be great to anti up a track record see how accurate their insight has actually been rather then just saying "well that's not good enough" without having an actual alternative that you can prove is better (unless you've got a line on the draft lists of all those teams that do outperform simple models, in that case im all ears)
We're not professional statisticians. If we ever manage to get this to the level of the initial PCs in our spare time I'll be stoked. We're much more focused on getting something "good enough" and making it publicly available to try and change the discourse.
It's not a feasible alternative, is that the issue? Great, I don't recall saying it was....even if it actually would be for the worst drafting teams in the NHL.
First, I didn't say that NHL teams don't undervalue point-based metrics, I just claimed that I highly doubt that any of them ignore it. If a player is putting up good numbers, teams will focus on scouting that player, and then perhaps over-use other methods to come to a final ranking. Conversely, I expect that teams that are considering a low-producing player, they will need to see a lot more in other areas to compensate (size, raw tools, character, etc.). They may have the relative weights wrong (i.e. too much emphasis on size or character), but I would be shocked if any NHL team doesn't use productivity as one of their evaluative criteria.
I don't have an issue with statistical models for ranking players at all. In fact, if there is a model that consistently out-performs scouts I would use it as my gold standard and largely ignore the prospect scouting organizations, polls of NHL scouts, etc.
My issue is that I would like to see more comparative evidence, and better testing of the models before I know how much weight to place on them. Whenever I see a claim that simple statistical analysis is better than NHL scouts and I ask for the evidence, I get referred to the same simplistic and flawed comparison (Sham Sharron). If someone has better evidence, I would be grateful to be pointed in that direction. But the analysis should be realistic, not a cartoon comparison. For example, the comparison shouldn't split forwards and defensemen, because teams don't make draft selections that way. Create a "big board" and then select the next best prospect (forward, D or goalie) on the board and see what emerges.
So, it should be relatively easy to find a better model that compares even better. I'd be happy to see it (and I don't mean that facetiously).
I don't disagree that more testing is better, but it hasn't been done because the end goal (at least for us) hasn't been what your after (a pick for pick draft board that's better then every NHL team)
Despite that the claim that A simple model is better then some NHL scouts remains true regardless of how cartoony the comparison. The terrible model beat the worst drafting teams. There were NHL scouts on that team.
I will hopefully be sharing our initial r value vs draft position lists soon, though they have some issues (primarily with the guys that were too good to have comps)
Again im not espousing any of the things you are after (something that beats all scouts and is a replacement for NHL departments on mass and draft board building).
Someone will always need to interpret the data (much like in 2015 when our model thought Gabe bast was Jesus.)
It's been repeated on these boards in the past. Your statement in this thread was a bit more generic, saying that it performed "better than scouts".
My point is that it would be better to use more consistent methodology so that the discussion / comparison moves beyond a polemic regarding NHL scouts per se (or particular team's scouts), to an actual assessment of where analytical approaches can (or can't) improve on other methods.
- Jun 10, 2014
- 57,393
- 29,209
Following this conversation I think there is something that is being glossed over. Earlier on Grind said he is promoting the idea of using his model (or similar) as a starting point rather than a 'be all, end all'. Please correct me if I am misstating your intent Grind.
So, produce a list based on this model, then fine tune it with the input of the scouts assessing the players eyes-on in-game and face to face. Starting with the players roughly sorted by this probability of success measure seems likely to produce a better result.
Yes this is accurate and the main issue. An autopilot list that's better then all NHL teams hasn't been the goal.
Even if it we're I doubt we could get there.
But to whilees point I think he's mostly concerned about further validation.
We've gotten good rs for NA forwards and dmen (separate not together) but have struggled in tuning the "priority" to build an actual pick by pick draft list.
And euros continue to confound us
Interesting. Quite a bit of my research is in the field of machine learning, and a few of my colleagues are looking into black-box models of sports performance (not hockey atm) that might help target and resolve inefficiencies using techniques adapted from stock market algos. Since these can have multiple inputs, eye tes, stats, psych profiling etc could be incorporated and weighted, with fly by wire adjustments.
Anyway, I’m learning a lot. So thanks.
- Jun 10, 2014
- 57,393
- 29,209
I assume the Euro problem stems from the lack of good NHLE's. At least in part. A draft eligible Euro player might play in 2 or even 3 different leagues plus international competitions in his draft year. That is going to continue to be difficult.
We have a method for dealing with that. The issue is for what OUR purpose is, euro nhles are bad. They get bogged down by the guys who don't jump over till their 26+ and end up playing bit roles.
With the new changes to the NHL site we might be able to do something about that. TOI in non NHL leagues has been a roadblock would go along way for us.
I've toyed with fairly arbitrarily tuning euros but I'm sure that's a not a good system.
If we can't figure it out we may even shelf euros for the time being and focus entirely on just getting NA's REALLY good (not what we want to do as it hurts our long term goal of essentially building what HF was but rather then everything just based off opinion it's all done mathematically (including team rankings)
Ad
Upcoming events
-
-
-
-
Game 3 Florida Panthers @ Tampa Bay Lightning - Florida Panthers leads series 2-0Wagers: 8Staked: $5,115.00Event closes- Updated:
-