What exactly causes lag when you have a ton of troops?

I usually start noticing lag with ~6000 troops on normal speed, around 4000-5000 on 2x speed, and around 4000 on 3x speed.

I once fought a battle where both sides had only around 2000 troops, but when all the soldiers clumped onto the bride, the game started to lag horribly even at regular speed.

My system is an i7 Quad (CPU usage is less than 25%), 6GB of RAM (RAM usage is less than 40%), 4870 1GB (overkill for RTW), and I installed RTW/EB on my SSD, so I know it's not a slow HDD access problem.

So why does the game still lag? Engine/coding limitations? O_o