So How has the AI improved ??

[R'as al Ghul]

My guess is that in the 3rd test, the AI doesn't wish to commit its general for fear of losing him to a bad die roll. After all, it doesn't need to to win : either its spears win the 1-on-1 fight and it wins, or the player wins the fight with a severely depleted unit that the general unit is sure to beat. Whereas should both units attack, and the player kills the AI captain, both AI spears could end up routing...

Mmmh. If the enemy spear unit is engaged and the General walks around to launch an attack in the rear how can the General get killed? I think waiting for the melee to be decided offers a greater chance to loose the general because the AI has to launch a second frontal attack, which poses a greater risk to the Gen than attacking the rear.

If such close in tactical decisions are determined by random triggers then that might also explain why attacks by the AI are often so fragmented. Its has certainly puzzled me in the past why the AI would calculate that an attack was a good strategy and then only send one unit to be slaughtered at a time instead of its entire battleline. Likewise the fact that it doesn;t always do it, suggests some sort of random factor is involved.

Random factors would certainly explain a lot but I shudder to think that's purely random. I also think that any manouver should be available to all units if the situation is appropriate. Not having spears doing flanking manouvers would certainly be a programmed disadvantage or a simple omission.
I'm certain that the tactical AI in STW is better in that respect and that the same test would have played out differently. I may post some results if I find the time to test that theory.