Simple - if you can build a port convert to a town. Playing some factions where the distance to the capitol or religious unrest is high. Leave it as a castle for a while until thing settle down, then convert.
As the turks not many castles are needed. I usually pick Aleppo and one more in the east. Going into Greece and the Balkans I add Sofia to the castles. Going into Africa Dongola might become a castle too.