One solution: Only ever use default (base) size and manually set the unit sizes to the sizes you want in the unit prod file (e.g. 80/120/200). This avoids the doubled training time (though you'll be missing out, as the huge unit size is quite arguably the best way to play the game).
Even with this approach, more men will still mean higher support costs so you will have to tweak those manually.
Bookmarks