If you have none or very few settlements, you will have more family members than settlements. That 1:1 ratio only really takes effect if you have somewhere around more than five settlements.