i understand ur point and i think here is where we have some kind of a projection. u seem to think that camps without the negan deal would get wiped out by other raiders. i think that the raiding thread still exist with negan, because we have yet to see how negan provides safety... his guards dont defend other camps, he takes the guns, he kills people off. he makes it harder for you to survive. you can argue that negan does not really wipe u out at once but kills u slowly instead. for example the wolfes attacked alexandria... (pre negan) but that could happen to hiltop or the kingdom, it was just that they found pictures from morgan of that place that gave them information about the location.
another argument is that negans main guys didnt bought in... which makes his idea to "safe" people a fail right from the start. but its realistic to me, negans plan requires order, trust, leadership and brutality, hitman/criminal mind to succeed which is very contradicted. u cant build a community with a main population as criminals... at some point they will turn/snitch on each other... its a wonder how negan build his personal army in the first place... the same goes for the governor, at least we got some background story on him, he lied and manipulated himself to the top and that only goes so far.