I like Mazoch's idea, it would work well for the Romans, might need tweaking for other factions, but it would work as long as there is a defined troop tree (infantry, missiles, horses, etc.). Elite troops are elite for a reason (training, equipment AND experience), pulling elite troops from the general populace seems a little wrong. In the Roman case, velites could be recruited from the populace, and sent to war. Once they have gained some experience (say 2 chevrons), they could be retrained up to hastati in a town with the right building. Retraining would cost some money and some of the unit's experience. The experience lost is reflected in the better unit stats. If it wouldn't be too complex, vacancies in a hastati unit could only be filled by merging a experienced velite unit, not through retraining in cities.
This would continue up the troop tree to triarii. Romans players would be forced to create lots of velite units, and be forced to use them to get the better units. The building requirements for units could be reduced to reflect equipment requirements only. After the reforms, units could be merged into early cohorts, and fight up to legionary/urban. New cohorts could be trained, but the cohorts created from experienced hastati/principes/triarii would have more experience. Similiar paths could be made for cavalry, archers, horse archers, etc. Losing a elite unit would be much more painful...

This breaks down for special units like nobles, but nobles should probably be limited anyway (maybe by # of provinces). How many nobles can exist in a 10 province faction?