Is it really that much greater?

At the moment, I am trialing Napoleonic: Total War - but unless the quality picks up I'll be playing XL again soon.
When I played the very first time, I played on huge for realism.
However I saw both the added cost and the doubled recruitment time, and thought I was simply doing myself and the AI a great disadvantage.

So I usually play on Large, and I notice most people in the AAR thread play on as little as Small..