I've had a similar experience to the destroying all towers and punching holes in the walls, think I was saved from the dead cows due to my mortars taking out the trebuchets.

The only thing about siege equipment i've seen to be an issue is once my ballista towers had destroyed one ram, the other 2 were left alone. However their artillery still had time to knock 2 breaches in my walls before my mortars took them out, I've found the necessity to sally before their reinforcements arrive.