Governments

[Mediolanicus]

Yes, you should destroy it and rebuild it.

The settlement may have your culture, but your admin needs to be build up from nothing after being destroyed by the enemy.

(That and the fact that the RTW-engine doesn't recognize multiple gov buildings, so you destroy the enemy's and repair your own = CTD IIRC)