I've never found a limit to the size of a crusade army, either, but it is way way beyond 32 units! I'm amazed you didn't get a loyalty drop as soon as your king joined the crusade. Quite simply units, once they're in a crusade, cannot leave it, ie it would put your king beyond any ability to return to the homeland until the crusade was over. Also kings just going into (an attacking) battle can put them "beyond the known world" as well, and give a massive loyalty loss.

As for ports, there's no known way to be sure it won't get destroyed, it nearly always is, but I've never figured a way to determine which or how many facilities are lost in a conquered province. Murphy's Law dictates that if you invade a province for its facilities, it will get razed to the ground Beyond that there seems to be no rhyme or reason.

I've always heeded the warnings, and never sent my king on crusade, though have sometimes sent my heir apparent to boost his stats a bit. Generally speaking, crusades can be a great way to make a right royal dog's dinner of your empire if they go wrong, and a lot can happen along the way. (Ever lost a Chapter House to an earthquake? )