Just ask yourself, Do you need this town to be populated/ do you need pop boosts in other towns/ do you have the troop power to occupy this settlement/ Do you need the cash badly/ do I need this city to grow to the next level.

when you answer those, the answer that has the highest priority should tell you what you need to do.