I think that's fair. I'm also not a Byram fanboy - look at my post history and you'll see I was hot for Turcotte and Zegras last year. I'm not claiming he's the next great defenseman, or anything, but I think he's being underestimated due to his lack of offensive production this year. He also had some major illness during the WJC and I think that only adds to the impressiveness of what he was able to do there.
Admittedly, I haven't seen him in Vancouver this year, but what I've heard from someone close to the Avs org is that they tasked him to work on his defense this year, and that they are happy with the progress that they've seen so far this season. I'm sure if the Avs org were displeased with how he was being developed, they would step in and ask the Giants to change things up. He might be playing in the same situations as last year, but is he playing the exact same way? Again, I haven't seen him this year, so I wouldn't know. I'm just going off the assumption that he's looking better as a defender since the Avs are happy with the strides he's made this year.
I'm fully on board with your frustration with regards to the CHL/AHL agreement, by the way. A player like Byram should be playing against men, like you said, not against other kids that aren't anything like the competition he's going to see once he makes the jump to the NHL. As for offense being his bread and butter, that might be the case as of now, but if the Avs develop him how they want and don't screw it up (lol fat chance), then he will be a more rounded player than a pure OFD. The Avs want good transition play from their defensemen with how the team is structured and how Bednar's system works, and if Byram can make that happen, then I know the Avs will be happier than if he turned into a purely offensive defenseman.