2 had better music, better overall Robot Masters, better weapons (albeit with some duds in the form of the one-shot, no-gun Flash Stopper and the immobility restriction on the Leaf Shield) and for my money was a little bit harder in the Wily stages.
3 was longer, had the cool feature of the Doc Robot stages revisiting the 2 RMs, better bosses in the Wily stages, added the slide feature which helped the game a lot, added Rush (even if you could abuse the Rush Jet), added Protoman, and originated the "It's not Wily this time, psych! it totally is Wily" twist with the angle of Wily going good as a ploy.
2 did more to define the series and I think is the more iconic game, but 3 I feel was better overall.
To be honest though, as much as I love 3, I think that 4 might be a little underappreciated and belong in the same tier. It had some creative stuff in it and with the charge shot and a little more polish it really stands up as well as 2 and 3. I also feel like it had a good, solid diverse cast of RMs. It wasn't until 5 and 6 that things started going off the rails.
For the record, I own 3 on the NES, have the PS2 anniversary collection, and bought 2-6 from the 3DS virtual shop. And I played them again as recently as last year, though I haven't finished 6 from that playthrough yet.
