The way it is probably is meant to prevent the player from making cities worthless (by destroying core buildings) or easier to reconquer (by destroying walls) when their fall is unavoidable.