I can say two things on that

1-I am sure that happened to many of you. The crusades can get lost. There are several paths to the target province, and if you go from the wrong way, the crusade no longer advances and stucks in somewhere.

2-Depending on the piety and zeal, as you go through zealous provinces your crusade collects volunteers and as you go by less zealous (or including high percentage of non-catholic population) provinces, your men leaves the crusade.

Any two of those must have happened.