In one of my campaign's I had two enemy Geisha after me.I managed to elude them by running from one end of japan to the other via the ports.I was able to kill one with my own Geisha and the other with a ninja.When you discover a Geisha running amok try to find the enemy province with the Geisha house
with an emissary and then attack that province to destroy it.
Tito