What was your method of evaluation?
A few years ago, I did a similar evaluation, breaking them up into groups - IE, bang on, 1-5 points over, 1-5 under, 6-10 over, 6-10 under, etc. I don't remember the exact results, but the scary thing is that all magazines massively over-predict, it was something like 90%+ of players scored fewer points than predicted.
Evaluating the different mags against each other was hard, because mag A might be: 1-5 pts: 5% and 25+: 9% while magazine B was 1-5 pts: 7% and 25+: 6%. So, A had a few closer to reality, but they also had more busts who were way off the prediction. Draft a guy who's 25+ points from predicted and you're screwed. Is A really better than B here? I have a higher chance of getting that bust.
Similarly, evaluating under vs over-predicting can be hard. Is under-predicting by 10 points better or worse than over-predicting by 7? I'd much prefer getting guys who were under-predicted, if I drafted a guy expecting 40 points and he got 50, then my toes are tapping. If he was predicted for 60 and got me that same 50, then that sucks, because I drafted him some number of rounds sooner than he should have been.
The other thing I notice is that they predict a ridiculous number of people to go 21-21, 28-28, with goals exactly matching assists. It's often multiple players in a row, Smith 20-20, Smythe 24-24... They'll predict something like 30 guys to do this, when reality has only 3 or 4 players do it in a season.
That just screams 'guy in a room pulling numbers out of his ass' to me. The number foremost on your mind is the number you just typed for Goals, so you type the same for Assists. There's also lots of 5s and 10s, IE 20-40, 15-30, 25-30 which again seems hand generated because humans think in nice round numbers. I would hope they're using mechanical processes but tweaked by hand, which would eliminate this.
