try giving the settlements to another faction. its good a idea to give them to an ally faction that you have military access to. that way u can retreat to that settlement (just placing your army stack near that settlement) and fight battles with your allies. having military access will also speed up your retreat if you have to.