Cities get their next upgrade level if they reach the next level of population:

400 village
1000 town
2000 large town
6000 minor city
12000 large city
24000 huge city

Similarly for castles:
400-4499 motte and bailey, wooden castle, castle
4500 fortress
9000 citadel

If you want to just up your population, you are best off with a chivalrous governor and building the farm upgrades. After that, you'd want buildings that grant happiness and trade.