Bribe him? But that will probably cost a lot.

I ran into this in RTW a couple times too, it's pretty annoying. CA needs to implement a "leave my territory" demand.