Of course you do, because it matches your agenda. Go with the subjectivity when it goes your way. Don't worry, I would be the same way most of the time, if subjectivity was the only way for something to be determined.
However if the basis of the award, and the current and past results clearly reflect this, has nothing to do with how good a GM does in that particular year; only his team. If NYI lost Game 7 against the Flyers last round, does Lou win? Based on those results, and the results of the past 10 years, that is almost certainly a 'no', and the only difference between that scenario and reality is one game. If Colorado beat Dallas, does Sakic make the top 3 instead of Nill (or even win over Lou)? Probably.
For an award like this, I don't care who is voting or how much of an expert they are, or even who won and lost, I would expect everyone to think twice about the legitimacy of the process. I may not be on that panel, or a GM in the league, but I know enough about processes to know when one is flawed.