Interesting concept for a statistic - we've discussed the need (and the methodology) for subjective statistics over on the By The Numbers forum.
Fugu brings up a great point earlier in the thread - there are "good" plays and then there are "GOOD" plays; similarly with "bad" plays and "BAD" plays. If you wanted to calibrate the levels for different plays, you could look (objectively) at the relationship between those plays and goals being score (or win probabilities changing; a mistake in a tied game situation isn't the same as the identical mistake in an 8-2 game).
Ideally, you would have multiple people (and at least one non-Detroit fan) doing these evaluations and credibility weighting the results - having a single viewer will enhance the likelihood that narrative and/or confirmation biases will sneak into the results.
For that matter, how are confirmation biases (for instance - I think Datsyuk is the awesomest thing since awesome, so when I watch games I view Datsyuk in that light, and lo and behold! He scores well in my evalatuations) controlled for? Or narrative biases, for that matter? Daniel Kahneman's book, "Thinking Fast and Slow", is something that I can't recommend enough for fans of behavioral topics. The most compelling thing about the book (my opinion) is that he'll tell you in advance that "this is EXACTLY how I'm going to trick your brain". You say, "No, that's ridiculous", and then you go and do exactly what he said that you would.
One of the bias problems stems from watching games on television - even if you're unbiased, the people running the broadcast decide exactly what angle you're going to see, how often you'll see replays, and (if you have the sound on) how important every play is.
Anyhow, I like the concept, but it could be worth tightening a few of the levers and dials. These issues are likely what's causing a few of the head-scratching results.
