Quote Originally Posted by Test112345
should i just start destroying and buildings that arent built by me if there's a culture penalty? will i be allowed to rebuild it afterwards?
I usually destroy the temples, but not immediately upon conquering it - when public order is really low. The culture penalty is AFAIK only removed the turn after you destroy a building, while you do lose the public order bonus immediately.
After I've destroyed it I build my own shrines there, the more disloyal provinces of course get shrines that give the most massive public order bonuses.
Temples and governors residences are the worst as far as culture penalties are concerned, and the latter you cannot destroy. Other buildings are much less significant.

I recall that possessing the Pyramid wonders in Egypt removes all Egyptian culture penalties across the map - something to consider.