Fair enough. While I do think that sometimes its absurd, overall its not a deal-breaker for me. I tend to recruit a few spies anyways because of the poison garrison ability which is fantastic.

Also, if you didnt notice already, theres a new beta patch out that supposedly really helps the siege AI. Havent tried it myself but people seem to like it.