Actually, the relations remain the same.

In my current campaign as the Moors, I was allied with Egypt and the Turks. My relations with both of them is stable at "very good", before and after they drop out of the alliance.

I can buy them back, but it seems like once they have accepted alliance and regular tribute, it's better to avoid any sort of diplomatic exchange with them.