I'm not 100% sure, but i think that upgrading to the next level of city will upgrade all buildings except temples to that of your culture.

Great info, thanks oaty. I'll have to stop my happy practice of extermination or slavery for every town I take
I woundn't go that far!