it depends on your game. on a GA game your score has a lot to do with diplomacy, as ell as your power, tech level, and income.

on world conquest there's no point in becoming smaller else you'll lose the game.