I think you mean destroyed on camp map, not in battle... no matter how much time you spend trying to destroy all parts of the wall in battle, the only thing you get is it gets damaged to 0%. I don't think you can completely destroy the wall as a building.