When you conquer a town, always destroy the government building, always. Other than that, you can destroy the MICs (barracks) that are worthless to you (keeping in mind that some factions can use other faction's MICs, so not all are worthless). Basically, all other buildings can be left intact, unless you want to do away with cultural penalty causing unrest.

As the Romans, you may also want to destroy any Gyms in cities you conquer so that you can build Arenas instead.