I understand that. I'd simply be happier if the "mysterious guards who cannot be killed but switch sides real easy" would be replaced by real units. And there should be enough of them to make storming the castle much more costly than it is now.