When playing as Egypt, you'll be filthy rich. Building a port (400 denarii) is hardly going to break your bank.

Get the port! Unless you are a cheapskate, very poor, or looking to mass up troops to knock TSE out as fast as possible.