I find that the AI when defending seems to just sit there letting you pelt them with fireballs until 50% of their army is gone. I don't use siege for normal fighting most times. On the way to a new settlement I might come across rebels or some defenders of the settlement and will get a chance to use it in a normal battle but they are definitely not needed.