In the couple of naval battles that I've done, one particular thing has earned me a few ships.

Hit a ship with a bit of chain and or grape shot, as usual to soften them up.
If they try to run, make sure to hit the "Continue Fight" button.
Board them with one of your ships
When the grapnels are flying from your ship and you're winching them in, let off a broadside of grape shot into them.

Three battles this has immediately brought up the "Decisive Victory" or whatever screen, and the ship has been in my "Prize" panel.