you can destroy every sgement of teh wall, only it wont come crashing down and open a hole.
and historically, walls didnt collapse completely. mostly it were holes, and not even that big. atleast not with battering rams. to make a wall come crashign down completely, you'll be busy all years. not teh mention the crearing of teh rubble.

ancients walls were suprisingly good. they were boxe shaped (greeks, persians and romans) and empty inside, this boxe shaped wall will give good strength, were it not they threw the 'hole' full with rubble, earth and sand. that debris will absorb many hits ass well. so making a wall come crahsing down wasnt that easy. towers were even worse.