After all the point of a trophy mistress is to make you look good to your rich buddies, not get you made fun of. His motivation here is basically the same as all the sponsors pulling away from the clippers--simple concern for what people think.
Wow. Just, wow.