As far as visiting jerseys, I've never witnessed shenanigans, but there's always someone. Rule of thumb: If you don't mouth off at someone, they won't mouth off at you. If they escalate, don't escalate back. If someone sees you with a Tavares jersey, yeah, they might say, "Hey, pajama boy!", but they're probably not gonna start randomly throwing punches. Unless, of course, you insult St. Patrice.
Yes, you'll get brownie points (and quizzical looks) for the Patriots hat. But...considering Toronto has no NFL team, I guess that makes sense. Definitely a wise choice.
I'm not a huge bar person. I like The Fours - it's sort of a bar/restaurant combo. People seem to like Sullivan's Tap - I've actually *gasp* never been. I think it's cash only? I also think it's changed hands recently, or there was some business about it closing and opening...which kind of surprised me, because with all the word-of-mouth, I figured it'd be popular. Those are both within very easy walking distance. Halftime Pizza is a quick slice-and-beer place that's nearby.
I hope you make it here for the Game 7 of the series!