And earlier, others made note that you could recruit peasant armies from large cities to move to small ones then disband them. When you disband the units add to the city population. I sometimes disband a few units just to get to the next city level early.

You can't build units that will drop population below 400.