Mercenaries will only reach a certain amount (usually 1 or 2), then stop increasing in number. But once they have appeared, they do not disappear until someone hires them, then they will take a while to replenish up to their max amount.