From what i remember (havent played EB in almost 6 months) that is an actual game mechanic. I used to fix those probs by recruiting a lot of mercs and then slowly changing them with other troops when the public order was better and that let me continue with the conquering.
But i could have missunderstud what you meen