I'll get a lot of lag sometimes during sieges. And not mega-huge cities/armies either. The couple of times I remember it happening was when each army was 1,200 or so. The AI's army just wouldn't stand still and was running and wouldn't stop. This caused major lag. But it only happens occasionally.