I think the main problem is that you can't target the siege engine itself (or maybe that's just me ? Also, I haven't tried burning towers, only rams). Your archers really target the men lugging it around, not the engine itself, meaning only a bunch of stray arrows do hit whatever contraption you wanna cook up.