Never been able to walk right in. It does not surprise me though. As another has observed, the AI likes field armies and not large garrisons. If you want to role play it a bit to give the game more of a chance, you can do the honorable thing and meet the field armies first before taking the weakly defended cities. It can make things more interesting.