As an idea, what about stationing archers in forts? It is possible to destroy the rams with missile fire. A larger attacking army builds more rams and you need more archers.

Archers are generally cheap. They might be the perfect way to defend forts. I am not sure at the moment what happens when all enemy siege equipment is destroyed. I remember instances where the AI simply left the battlefield, but also situations where they just sat there forever. What happens when you leave the battle via ESC and they have no rams left? In EB I play without the battle time limit.