Hmm, what are some traits you're getting from having too many people? I could see having 50,000+ treasury, or some of the more decadant buildings.