You'll have to kill every last crusader. It won't end before that. Note that in the first 10 turns, the crusades you beat can be brought back by sending a new army. After those 10 turns, it's not possible to send more armies at you, and once you've killed the last stack, the crusade ends.