They're probably high-level, difficult to kill. You could try to train an assassin and using alt+right click to assassinate characters in your own cities. Otherwise, just wait until they die off.