Take note that when you receive the quest, it states : You can offer them a less valuable settlement in exchange"

Also, one of the many tactics, is your at war with them, is to offer a ceasefire for it (works sometimes, usually when they come offering a ceasefire to you)

Just remember though, if you trade off a settlement for it, theres some sort of cooldown time from when you gave it to them to when you can attack it. If you attack too soon, you lose rep big time for it