I had to play a game in a Penguins sweater once. I forgot my $10 game sweater in the dryer (yeah yeah, washing gear, I know, it was the first time I'd washed it in like three months). Our colors were black and yellow, so it wasn't a big deal. It held up fine. I looked kinda goofy, though. Didn't have any numbers on it, but I still felt like a superstar out there.
I think you'll be fine, but don't be suprised when your typical wear and tear occurs. If it's a sweater you're not attched too, knock yourself out. And like 35 and 14 said, you'll definitely get some looks.