You have to be pretty darn good to win 44 fights (26 wins by KO) only fight I have seen him push aside is Pacquaio, this is because they cannot agree on how to divide the enormous amount of revenue generated by the fight. The fight was projected to attract 3 million customers through pay per view which would bring in about 250 million dollars. Add Sponsorships and Ticket revenue and your looking at MASSIVE lump of cash! I have no idea which other fights he has turned down but your crazy if you think Mayweather is not an elite fighter
Mayweather is a great fighter. His defense is amongst the best in history. But a dominant fighter? No. He does not belong in this conversation.
As far as ducking fighters goes, Mayweather used every excuse in the book to avoid Pacquiao, from drug testing (which Pacuiao agreed to) to dates and venues that Mayweather was "unavailable" for to then refusing to split the revenue with Pacquiao. As in, he told Manny "this is as much as you'll get, no share of revenue beyond that" which is just an absurd demand.
And then he tried ducking Alvarez. First, he refused to sign a contract to fight Canelo, saying "We'll fight, I just don't want to put my name on it right now". Then he tried to demand that Alvarez drop in weight (despite Mayweather holding a title in Canelo's weightclass).
He dodged Margarito again and again while Margarito's camp was on their knees begging Money for the fight, he dodged Cotto (which led to the Margarito-Cotto bout).
His entire career is made on waiting to fight guys until they are past their prime, jumping on guys who are just moving up in weight class, and adding ridiculous stipulations to any fights to either weaken his opponent or make them back out completely. He himself has stated he tries to avoid taking on strong opponents because he's "getting older".
Mayweather is a brilliant, gifted boxer. All the talent in the world. But he never backed up the talk of him being the greatest of all time. When people look back fifty years from now, they'll remember Mayweather as talk and broken promises. Can you imagine how Ali would be viewed if he refused to fight Frazier? That's what people think of Mayweather not fighting Manny.