Well, no reason to panic. Maybe I understood this the wrong way.

I'm not sure, if it's enough to move near the gate. I suppose you must send them inside actually. Have you already tried like ordering them to move beyond the gate in the road that leads to the town square?

If that doesn't work, I'd say it's a bug.