Very slowly out of a town and slowly in a town from what I can tell. It also seems that if you lose almost all bodyguards you get more the next turn.