I can easily think of a reason to destroy a wall: you have a city with huge unrest that's going to rebel soon - you'd want to remove the wall so that when it does rebel, you have easier time getting it back.