You did not say as what faction you were playing, because if you're a steppe faction (including Pahlava) you can recruit troops through that building, if it does not offer you any recruitment options you can of course destroy it.