You don't want to be bullied and he doesn't want to be bullied either - if there's one right thing to do, it's to forget the whole thing.

Actually, if you're bent on revenge of sorts, do the same thing: talk behind his back.