The enemy never gets to assaulting my walls anyway except in custom battles. They either try to wait me out or I sally or attack them with reinforcements.
Kind of sad I know. But then I'm glad because my comp can't handle large siege battles :P