Problem with the eye test #69 trillion: I believe that just existing on the ice more than your peers, even if you don't contribute, improves your perception due to the availability heuristic.
Players are actually rated by ice time. And I don't mean using TOI as an argument and appealing to coaches. I mean players are literally perceived according to TOI to a significant extent.
Some Ds really need a little more time to play their game. Guys like Suter, McD, Yandle and the likes just seem to find a grove being patient enough if they play a bit over 20. If they get down to 17-18 minutes they try to do too much.
However in general, hockey players are different from each other. Hockey is a multifaceted sport so to speak. You need to be able to accomplish a lot of different things on the ice, and your ability very much directly impact the result. With baseball, there may be an X factor. The devil may be in the details. But in the broad picture that is just not the case with hockey. Neither with like soccer. Sure there can be upsets and determination and will can overcome a lot of things against a team that don't execute, but a good hockey team with good players will dominate a less able team.
In that sense, there really isn't even remotely a problem with the "eye test" in hockey. Take this example. Lets say Mike Babcock, Peter Laviolette and Gary Gallant doesn't know anything about Swedish hockey. Let them follow the SHL one season. Then let them create the best possible team they can of players from the SHL. 18+2. I would bet that all these three coaches would pick 12 identical names. I would bet that 14 names would be on 2 of 3 teams. Then 6 names would differ. At least 2-4 of these names would have been considered by all three coaches.
Then pick another team from the SHL being completely blindfolded and only using analytics and let each coach try to coach that team for a month. I can guarantee that neither coach even remotely would like this team better. Or do you think otherwise?
Analytics can be of great help for a coach/scout though, just as watching video can be.