Colon did suffer somewhat in that his best season (2002) was split between seasons. Had he remained in the AL all year (with the same performance), he would have been right in the middle of Cy Young voting.
It's a fair point that he wouldn't likely have pitched as long had he not suffered injuries in his prime.Even if we assume he did (at the same level of effectiveness), and given him the average of his 2004-5/2011-12 seasons in place of his 2006-2010 period, he'd have just about 60 career WAR without a really strong peak, which isn't a very good case.
Throw those counting stats on, though, and we're looking at a pitcher with a 305-215 record in about 4,100 career innings with a 4.00 ERA (roughly 108 ERA+), one Cy Young (undeserved, but he might have been close to deserving one he didn't win, call it a wash), four (maybe five) All Star selections, a decent postseason record but probably no rings (the 2006-2007 Angels were good teams, maybe a better Colon helps push them over the top. Maybe that's a Hall of Famer in an Early Wynn/Don Sutton sort of way, but it'd take him a few tries to get in. I don't think I'd vote for this hypothetical player, but I don't think I'd be too miffed by his election. Without the lengthy post-40 career, though, Colon doesn't come close to 300 wins or 4000 innings and so won't get elected.
Colon really needed not just health in his mid-30's, he needed another really effective season (like 2002). Take away forty runs in 2004 and he has a pretty nice prime, probably a third 20-win season, maybe another Cy Young (or maybe he wins a deserved award and not an undeserved award), another 5 WAR, and maybe that, in conjunction with better health, makes him a deserved Hall of Famer or at least not an embarrassing choice. But you could say the same for a lot of players.
Colon just didn't come close - he wasn't a really good pitcher for long enough.