Quote Originally Posted by Herakleitos
Culture penalty is indeed based on the buildings in a city. Destroying existing buildings (or upgrading them with your own version) helps in lowering the penalty. I believe the governor's building has the most influence, but you're not allowed to pull that one down I'm afraid.
Correct, except that temples also give a big cultural penalty. Tearing down foreign buildings is not necessary: upgrading works just as well. You can even upgrade certain foreign temples to temples of your own, provided you have the right temple type (fun, trade, war, horse, etc.). But this does not always work: you cannot upgrade the temple of Nike to the temple of Mars or vica versa, even though they have identical effects (I assume temple type differs).

There is a Spanish/Gaulish temple than can be upgraded to a Pantheon that is not otherwise avaible by the Scipii and the Brutii, but getting the Spanish to build the Sacred circle of that temple is quite tricky.