Adding to this, raising an army in Lords of the Realm II had a negative impact on loyalty and it took some time to restore it. This always made good sense to me. It might not fit current game mechanics of course.

The elite pools should be rather small and they should take some time to replenish. Building elite units of the same type back to back should not be a common luxury.

Some of this gets back to the raising armies vs. recruiting units. Armies should still have to be more balanced. Upgraded structures should improve overall army size (or even build time) and add some better substitute units in the army, but the armies should still have a core characteristic. This would probably help the AI build better armies in spite of itself.

I wouldn't mind seeing each unit have home provinces, so that if the province is lost they might either rebel, turn mercenary, deplete, become non-retrainable, etc. Of course, the AI probably couldn't handle this very well...