I'm pretty sure that, once upgraded, a huge city stays a huge city regardless of how low the population goes after that