A motte in Norway, Denmark and Sweden enables you to build Viking Landsmen. Not the best unit ever but building requirement are really, quite worthwile if you need axemen and do not play as the Danes.

Curtain Wall does boost the defense of a keep in significant way since it implies the replacement of the outter wood palissade by a stone wall so that the enemy can only enter through the gate if it has no siege equipement. Can be handy from time to time and only costs 200 florins and only takes two turns to build.

Other improvements are not really worth the trouble and the time even though assaulting a castle with catapult towers can be costly even if you have siege equipment (the catapult in the castle are much more accurate at least when the AI uses them so be careful with your general) ...