I have never heard of this problem, but if its happening on the strategy map, I wouldn't worry about it. If an enemy invades your province, you will still get the option to withdraw to your castle if you don't want to fight a field battle.

I guess there is a sort of back door way into the castle, come to think of it. When you retrain a unit, I think it reappears in the castle after the training is finished. Of course, you could only get one unit in per turn.