Negative security traits can reduce the number from the normal size. I found BG to be too powerful, so i cut them down to half size. This means 10 men is normal, but as you noted, the King and heir often have twice the man power.
I think this is ok. I reduced the size so that ordinary cavalry and Knights still has a use..