In my last RTW:BI campaign as the Sassanids, I had 4 Clibinarii with one gold chevron each that were really depleted (less than quarter strength). After retraining, they had only 3 bronze chevrons... If recruits were trained at the units average, this should not happen. I remember similar things in MTW2. I guess recruits are normally fairly well trained, but if a unit is less than half strength, its quality decreases when being retrained.
It seems very logical to me: If you mix some recruits in a veteran unit, they will take an example of the veterans and behave the same, act the same in combat... If you have a green unit with only a few veterans in it, they won't have enough influence and they cannot look after all of them.