They aren't supposed to. I think it is a bug.

If you actually moved your army away from siege, then attacked again, and the mobs came back it would still be reasonable. But the game sometimes glitches and acts like you broke the siege when the defenders sally.