Besides the possibility of having a crusade against one of your cities, what happens?

Within a few turns, I'll have the catholic nations down to me (Venice), Purtugal, and Poland. If a crusade is called on me, I can take them down.

But in order to take down the remaining factions, I have to go against that stupid pope BS of refraining from attacking.

Do I lose money or anything?