Unrest is caused by spies, just happens in newly conquered cities and slowly goes away, and sometimes unrest is just a perminent staple in a particular city.
Culturaly penalty is caused from owning a city of a different culture. You can get rid of it by tearing down or building over (upgrading) buildings in the town that were built by the previous owners. Upgrading the town a level will get rid of most of the culture penalty. Temples also seem to effect it more than other buildings.
Bookmarks