You suffer a 20% public order drop on cities, 5% in castles, you can be the target of crusades, Catholics have no penalty for attacking you (not sure if your relations automatically drop) and all your current crusade armies will disband