I think it is because sapping (as it was in RTW) had no counter, even sallying out cavalry would have no use.Using ladders has no counter either. I wish halberdiers could push them off the wall, killing everybody on it and (possibly) destroying the ladder. Right now it's too easy: run to the wall (you could not run with ladders in RTW, IIRC) — climb — send more units to climb while the first is clearing the way for them.
By the way, is there any point in building Towers? Do they work or are they broken?