Make sure you are always heading in a direct line for the crusade target every turn or your army will start deserting.