Try blockading them with a single ship. When they attack you have to spend the battle running away. You must have timed battles set, best at the shortest setting, and evade them for the whole battle.

It is a bit of an exploit but it is the only way to get their fleet out of port.

Then you can move in with you own fleet.