Referees and Linesmen are a part of the game and have every right to be on the ice. They are there to maintain order when players with your mindset can't control themselves or break the rules. Sometimes, when a player shoots the puck without looking, they hit the official. Sometimes when they shoot the puck to try and score a goal, they hit the goal post. By your reasoning, there shouldn't be goal posts.
Refs are not playing the game.
I can have a game without Refs. You cant ref a game without players.
Your job is to control the game. That doesnt mean being in the way. WOuld a player on the opposing team go over to the bench and yell at player for hitting him with the puck? Or, by your reasoning, would the post go over to the bench and cus at a player for doing that?