Pick a Ballista (I prefer NE since England is the default player faction in custom) and go play with it in the EDU. The secondary attack line details the attack of the ballista (that's stat_sec) while everything else is presumably about the men that use it. The original for NE is:

Code:
stat_sec         55, 3, ballista, 180, 50, siege_missile, artillery_mechanical, piercing, none, 25, 1
I'd suggest trying this:

Code:
stat_sec         55, 3, ballista, 180, 50, missile, missile_mechanical, piercing, none, 25, 1
That's me basically guessing that the missile type may be responsible for allowing it to attack walls - using the standard ones for archers might keep it from doing so. Of course it may just be the case that siege equipment attached to units can always target walls and towers.

In either case, you'll want to set ballista to the following in the descr_projectile.txt file:

Code:
projectile ballista

effect	    large_arrow_trail_set
end_effect   large_arrow_explode_set
end_man_effect			man_impact_small_set
end_package_effect    	wall_impact_small_set
end_shatter_effect		large_arrow_explode_set
end_shatter_man_effect		man_impact_small_set
end_shatter_package_effect	wall_impact_small_set


damage		0
damage_to_troops 5
radius		0
mass		0.8
accuracy_vs_units		0.005
;accuracy_vs_buildings	0.05
;accuracy_vs_towers      0.025 
min_angle	-30
max_angle	35
velocity	60
;bounce		0.5 0.6 0.5 0.4
body_piercing
display		aimed particle_trail invert_model_z
model		data/models_missile/missile_ballista_bolt_high.CAS,  40.0
model		data/models_missile/missile_ballista_bolt_med.CAS,  80.0
model		data/models_missile/missile_ballista_bolt_low.CAS,  max
Do the same with flaming ballista shot (it's the very next entry).

At the very least that change should keep them from harming walls at all... and I'm guessing the AI will avoid targeting walls if it knows it will do no damage. Could be wrong though...