I only "let it occur" because I was on holiday with no internet access...

And I could always break my oath to him so he can't have an army and then demand that the chancellor give those units to me, which I will do if I have to...