Also, don't forget good relations/Chivalrous leader affect your reputation as well. So having bad relations over a period of time (such as on harder difficulties where its always dropping towards the lowest level) could be a major cause of a bad reputation. You almost literally have to be losing the game in order to maintain a decent rep, since being well off tends to make the AI more aggressive to you (like with a single ship raiding a port for no apparent reason. They will do that if they consider you too well off, even if going to war with you is a completely stupid idea).