Hence the tiny number of castle assaults most players see in MTW.
It can be fun to goad the AI into assulting by leaving your province with a tiny but seigeworthy garrison. If the siege will last 3 or more years, usually the AI will assault, and when they do you can inflict massive losses on them if you're prepared for it. This can also be a way of defending a border province more cheaply than by having a full-stack defensive army -- useful if you're playing a small faction. Retreat to the castle, fight off the assault, and then counterattack their depleted force with reinforcements the following year to drive them out of your province for good. Although I wouldn't recommend this tactic with anything less than a castle, preferably with ring wall (but no catapult towers -- they tend to wreck your own walls). After building a keep, my next build is usually the curtain wall, which adds a layer of stone wall rather than flimsy wood that the AI can just hack down.
I *do* wish there was another option before a defensive battle wherein you could choose to withdraw everyone *except* those forces sitting in the castle. That way you could leave a suitable garrison in every castle and not worry about having a larger force outside the castle in the same province.
Bookmarks