there will be (a) free unit(s) of militia for every settlement/city/castle. So they won't be unguarded unless the A.I. is unable to handle it, which I doubt.