The only solution I have found to port raiders in an allied port is to place a fleet just outside their intercept range and send in a single sloop. They bite almost every time.

The AI does actually kick them out of their ports, it is just if you have a trade agreement with them because of the player hating aspects they don't get around to it for 20 or 25 years...