Do not worry as far as i know:
as long as i know when you are really near the crusade tagret you will not get deserters, only watch that your general doesnt die of old age...