My Dream AI Opponent

[BeeSting]

I know many of you are by now sick of hearing me *itching about the AI, which is by any level "intelligent”. I don’t know if it is due to technical limitation or what have you, but the thing never learn. And maybe this is where the problem lies it just isn’t aware of what it does because much of awareness involves the ability to lean from ones own mistakes, or to see the future through past concepts. But you could school the AI by programming, teaching it to react and formulate its path to achieve goals. Yet these become patterns of behavior for a human player to easily remember and react accordingly, and permitting he/she is creative enough, trappings could be set along the possible route that the AI will take. What then? Could we ever have a worthy artificial opponent? Could it ever learn after so many failures and formulate its paths accordingly in systematic manner? Could it ever be cautious and not fool hearty?

I would like to one day not have to micro manage each units but have reliable AI generals in charge of other parts of the army, in style according to their personalities…. and these may be real historical figures. Perhaps you could actually sit with them before battle in a camp and interact with them for a battle plan. Damn, this would totally suck me into the game and I would literally forget that I now live in the 21st century.

Got a little deep there…. My apologies for any headaches caused by reading this.

What would you like to see the AI do in general, or in specific?
Any comments on the present AI and possible fixes for the future?

Any comments from programmers or just dreamers like myself would be much appreciated.

I just thought it would be interesting to hear your opinions and thoughts on this.

[Colovion]

I remember a long time ago (years) I saw this game on the back of a comic book. I forget the name, but it was something to the effect of Robot Wars with large machines fighting. The biggest thing they were selling was their AI. Their comments were something along the lines "Computer Opponents learn from mistakes, and counter your usual fighting styles for more gameplay fun!" That game I never played it, but felt the idea a grand concept. Another year or so past, games like Starcraft were played and I saw another game in a Gaming mag which was an RTS game similar in style to Command and Conquer. It said basically the same thing about it's AI: "Computer AI searches your defenses for weak spots and exploits them. Stay on your toes!" That game, I also didn't play.

The point is that what I've always been impressed with was a computer opponent that could continually give me new challenges to overcome and new ways to try to defeat me which I'd have to, in turn, counter myself.

My thought of a truly remarkable AI is one which can see what you do poorly and exploit those weaknesses. If you start doing something well - such as building loads of archers to mow down their phalanxes then perhaps the next time they'll bring a bunch of light Cav to run down your pesky archers. Even something as simple as that would make my day, and anything more would floor me.

[A.Saturnus]

The problem is that all computer games use rule-based AI. The AI has a number of rules which it applies to any situation. Even those that exploit weaknesses of the player do that. They just have rules concerning possible weaknesses and when they see those they apply these rules. I suspect learning is implemented into rule-based AI by analysing the behaviour of the player and categorizing it according to preprogrammed rules. So the AI doesn´t really learn, it just has a set of different assumptions about the behaviour of the player and uses those assumption that have been correct most of the time before.

A revolutionary strategy game could make use of a connectionist network instead of rule-based AI. A mulit-layer perceptron, for example. The programmers could expose the network to strategic problems and let it assign "winning actions" (chosen by experts) to these until its performance is very high. It can then learn which actions are "winning" in which situation and generalize it to all strategic problems. The network would still not learn "on-line", - that is during playing with the player - but it would have a high chance of picking right moves in all situations not too different from those trained. The advantage would be that the network would not stick to rules that might be extremely off in some situations and would show a less monotone playing style.
Though the demands on the network for a game like RTW might be much too high.

[L`zard]

The funny thing about all this is that if the 'AI' where truely heuristic, everyone would be complaining about the cpu hit on thier top-o-the-line brand new 'puters, ~:eek:

tanstafl........

I wouldn't expect this until the entire gaming world has next generation boxes to play on, eh?

Pity......

[hung41584]

okay, you guys don't know how hard it is to create an ai. if you could produce one of those kind, the game could be like civ 2 and still would be a best-seller today. Have you guys ever looked at the scripts for an ai?? I remember a while back for aoe, all the ai had for a script was build buildings, build a few units, build bigger buildings, build more units, build more units, then build a wonder. I'm not so sure how they have evolved from that nowadays, but i would hardly think it would change from some sort of script form. You still get the same old "build this unit then that unit", if human builds this unit, build counter unit to that". And that's all that happens. If you hadn't already noticed, on normal difficulty, the ai just build regular units with no resource bonus, which means they can run out of money. But on harder settings, they get like 200% or greater bonuses and build gigantic armies out of a couple of cities. So this so "better ai" that you guys wanted doesn't exist, it seems the only improvement that can be presently made is a resource bonus. I'ved noticed this in many other games like civilization too.

[L`zard]

okay, you guys don't know how hard it is to create an ai. if you could produce one of those kind, the game could be like civ 2 and still would be a best-seller today. Have you guys ever looked at the scripts for an ai?? I remember a while back for aoe, all the ai had for a script was build buildings, build a few units, build bigger buildings, build more units, build more units, then build a wonder. I'm not so sure how they have evolved from that nowadays, but i would hardly think it would change from some sort of script form. You still get the same old "build this unit then that unit", if human builds this unit, build counter unit to that". And that's all that happens. If you hadn't already noticed, on normal difficulty, the ai just build regular units with no resource bonus, which means they can run out of money. But on harder settings, they get like 200% or greater bonuses and build gigantic armies out of a couple of cities. So this so "better ai" that you guys wanted doesn't exist, it seems the only improvement that can be presently made is a resource bonus. I'ved noticed this in many other games like civilization too.

hung: That's the point that A.Sat and I were trying to make, eh? The resources required for a heuristic 'AI' would be extremely large, and would either slow your puter to a crawl or lock you up entirely depending on your personal 'puters specs.

As far as any of the Total War series goes, we're not talking about A(rtificial)
I(ntelligence), we're talking about 'Programed Opponent'.

This is most likely true for about all of the games any of us is likely to see/play.

When 'big blue' goes back to winning, then we'll see, eh?

[Pode]

Umm, hate to burst your bubble, fans of the rule based AI, but a nerual network is capable of farily complicated tasks on relatively low end machines. A guy I work with has a couple of PC-104 single board computers, like 166Mhz processors, forming an adaptive autopilot that continually relearns how to fly an unstable helicopter 20 times a second. I *think* my 1.6 Ghz gaming machine, with 10 times the power, should be able to learn the ideas of rock paper scissors and flanking manuevers over the course of a few hours. It's not the machines that are lacking in knowhow and brainpower, it's the developers. I'm not saying they're dumb. I'm saying that they're not given enough time/funding/support to go learn about adaptive autopilot programming on the off chance that they could apply it to AI design. Corporate types aren't willing to gamble like that. The limitation is human, not machine

[L`zard]

Umm, hate to burst your bubble, fans of the rule based AI, but a nerual network is capable of farily complicated tasks on relatively low end machines. A guy I work with has a couple of PC-104 single board computers, like 166Mhz processors, forming an adaptive autopilot that continually relearns how to fly an unstable helicopter 20 times a second. I *think* my 1.6 Ghz gaming machine, with 10 times the power, should be able to learn the ideas of rock paper scissors and flanking manuevers over the course of a few hours. It's not the machines that are lacking in knowhow and brainpower, it's the developers. I'm not saying they're dumb. I'm saying that they're not given enough time/funding/support to go learn about adaptive autopilot programming on the off chance that they could apply it to AI design. Corporate types aren't willing to gamble like that. The limitation is human, not machine

A.Sat: this what you where talking about?

Pode: NEAT! Where can we get the proggy, someone has to ship this off to CA asap, eh? Not to mention all the other game devs, eh? Bugger a bunch o' corp suits, let the truth come out!

'sides, I'm sure a LOT of modders would like to see the insides of a program that could do things like this, no lie!

Of course, as long as R:tw is 'hard-code' for most of it's content, guess this won't be a lot of use at the minute, but WTF,over, It's a wonderful idea, eh?

Neural Networks.........hmmm...

(and 'they' created Seti@home to do what? Hmmmm)

[hung41584]

Umm, hate to burst your bubble, fans of the rule based AI, but a nerual network is capable of farily complicated tasks on relatively low end machines. A guy I work with has a couple of PC-104 single board computers, like 166Mhz processors, forming an adaptive autopilot that continually relearns how to fly an unstable helicopter 20 times a second. I *think* my 1.6 Ghz gaming machine, with 10 times the power, should be able to learn the ideas of rock paper scissors and flanking manuevers over the course of a few hours. It's not the machines that are lacking in knowhow and brainpower, it's the developers. I'm not saying they're dumb. I'm saying that they're not given enough time/funding/support to go learn about adaptive autopilot programming on the off chance that they could apply it to AI design. Corporate types aren't willing to gamble like that. The limitation is human, not machine


Yeah but how do you apply that to the game??? The ai can't relearn how fight and build all the time. What could you learn about fighting???

What I thought the ai would have is MORE RANDOM tactics. I just thought maybe they finally found a way how to make things like this simpler. I mean, haven't the games gotten so much better in terms of graphics and features???? Like you could do something like just use a certain tactic that won't leave any flanks open and just choose to charge in different direction everytime for the ai. it makes it more random than seeing the ai general charge in a direction that is prone to breaking and then chasing after him with your own cavalry and killing the general, thus winning a large part of the battle.

I have at least one good ai strat that will always work. Simply have them put more archers in their armies. I mean, those things do wonders in any army. 4 archers, protected by triari or spearmen, and you got yourself a difficult battle.

[The Stranger]

Umm, hate to burst your bubble, fans of the rule based AI, but a nerual network is capable of farily complicated tasks on relatively low end machines. A guy I work with has a couple of PC-104 single board computers, like 166Mhz processors, forming an adaptive autopilot that continually relearns how to fly an unstable helicopter 20 times a second. I *think* my 1.6 Ghz gaming machine, with 10 times the power, should be able to learn the ideas of rock paper scissors and flanking manuevers over the course of a few hours. It's not the machines that are lacking in knowhow and brainpower, it's the developers. I'm not saying they're dumb. I'm saying that they're not given enough time/funding/support to go learn about adaptive autopilot programming on the off chance that they could apply it to AI design. Corporate types aren't willing to gamble like that. The limitation is human, not machine

it will come, it's just we're talking about games here. they are willing to do this kind of things for national protection but not for national fun

[BeeSting]

Yeah but how do you apply that to the game??? The AI can't relearn how fight and build all the time. What could you learn about fighting???

What I thought the ai would have is MORE RANDOM tactics. I just thought maybe they finally found a way how to make things like this simpler. I mean, haven't the games gotten so much better in terms of graphics and features???? Like you could do something like just use a certain tactic that won't leave any flanks open and just choose to charge in different direction everytime for the ai. it makes it more random than seeing the ai general charge in a direction that is prone to breaking and then chasing after him with your own cavalry and killing the general, thus winning a large part of the battle.

I have at least one good ai strat that will always work. Simply have them put more archers in their armies. I mean, those things do wonders in any army. 4 archers, protected by triari or spearmen, and you got yourself a difficult battle.

Archers are overpowered for this period. I reduced mine to 5-8 attack for arrows, mostly 5, and it did wonders for the game play.

The random factor should be counted into multiple characters/personalities for the AI. Multiple attributes should each have a linear scale reflecting the likelihood of that attribute surfacing, i.e. "cautious to chancing/conservative to aggressive": A chancing general will leave things more to luck and engage the enemy general although the percentage/conditions for success is not favorable. An older or experienced AI general is more likely to be cautious.

And the impression of AI "learning" could be given to a player by simply having the AI recognize and count its opponents past battle statistics--wins/losses. It would not therefore face an army led by a commander reputed to win 9 out of 10 battles without superior numbers and better troops, unless it is forced to do so. All these could be translated into numerical values and weighed according to applicable rules via “if, then”. You get the idea....

Also, AI learning on the battlefield level does not have to be so complicated as figuring out new counter tactics. The problem I see with the RTW AI is that it is impulsive and does not plan out its course. If, the AI wants to attack its neighbor, it should send out spies to figure out the kind of soldiers it has, for these are rock-scissors-paper units. If the armies of enemy faction are composed of strong archer units, then it should counter that by composing a large part of its force with missile units. If this doesn’t succeed in wins, then it should slowly increase the number of cavalry units for the sole purpose of running down the archers. My tactics rarely win me battles, my units do. I win on battle fields, mostly because I manage to make the right units while the AI fails to do so.

[Red Harvest]

Neural nets don't excite me...my experience with them at work was that they couldn't tell the difference between cause and effect (I could give some humorous anecdotes.) First principles were better than the neural nets. Where I saw the nets having some impact was in helping to identify the actual problem, and in coming up with some controls of some things where direct measures were not technically possible. Also, the net fell apart when something fundamental to the system changed that the net was not trained for. So all you would have to do to befuddle it utterly is do something illogical, then beat the heck out of it with logic after that initial move. So yes, with some repetition it can learn not to "fall for that trick" again...but it won't appear very intelligent while it is learning. And with that you open it up to new types of exploitation.

The idea of not being able to exploit a neural net seems highly implausible to me. I have had games that appeared to "remember" my last response (it seemed to increase their probability of responding predictably the next time.) I could exploit that heavily.

Rules based AI makes sense along with some numerical analysis of the situation. The problem is the depth of the rule set, and the pattern recognition required for triggering the branch and the rule. That is how we as humans are doing things. And in some chess computers the branch choice is somewhat randomized while in the "opening book." One could argue that a sufficiently detailed and accurate rule set would be much stronger than a human, because it would never make an outright mistake.

You must have well established game mechanics (game rules) before you can write good AI rules however. If you change/reverse the relative efficacy of something without updating the AI, you are unlikely to get satisfactory results.

The AI rule set should have trouble with gambits where it can't see the "trick" being played by the human. Exploits will be found. However, a good rule set with some ability not to always choose the exact same path can utterly foil a gambit. If the AI sometimes chooses the "2nd best" response, the gambit player risks using the wrong tactics vs. the wrong opponent.

[hung41584]

well, hopefully there will be more TW like games that will have this so called great ai. right now, this genre is sorta just beggining

[Pode]

My point was not so much to endorse the use of neural nets instead of a rule based system, because in the helicopter example the neural net is only learning one thing: how the helicopter responds to the control inputs, and there are very well defined equations that cover that response to 99% accuracy. My intent was merely to emphasize that the very complex task of learning to fly a helicopter was being carried out at 20 hz by a single-board computer. I don't see why a typical gaming rig can't be expected to process an AI capable of resisting a mediocre player.

[The Storyteller]

Two things...

Firstly, I think what the STW and MTW AIs did quite well was to have a reasonable strategy each time. It looked as if someone had played a few battles and said "okay, in general, it's best to pop your archers here, pop your infantry here and advance in a line. Meanwhile, the cavalry tries to outflank."

The AI WAS predicatable. You could EXPECT flanking manouvers. But predictable doesn't always mean easy. Basic tactics done well can still result in wins. Even iIf you KNOW the AI going to flank, you still have to bolster your flanks, which takes away strength from your front, which makes it easier for the enemy to break through anyway.

Secondly, I know AI that learns from past mistakes is already present in Chess. But you know, having an AI that would get progressively smarter, and maybe even take on personality traits according to V&Vs is... scary! It's only a few steps away from true blue AI that would be exactly like a child when it started, and grow up to become like an adult with personality and all.

I guess RTW TRIED to do the next best thing by having their units analyse whatever units were in front of them to see if they could be beaten. I guess the idea was to exploit wekanesses in the line on the spur of the moment. Sounds great in theory...

[A.Saturnus]

In the example with the helicopter the number of parameters was probably smaller than in reality. Otherwise we would see lots of robot-pilots flying around already ~;)
Playing Rome is undoubtly more complicated. Consider that some neural networks to simulate weather require Cray computers. BUT, CA still could use bigger networks. What requires most calculating power is the learning. CA could train the network on a workstation and then implement the trained network as AI in the game. The AI would be unable to learn, but it would have the knowledge gained in an extensive training phase without being demanding to much.
Of course, the AI would then react confused to strange situations, but rule-based AI has exploits too.

That neural networks can play strategy games is certain. There has been a network that learned to play backgammon. After sufficent training, it could beat any available rule-based backgammon-software.

[HicRic]

I think the biggest problem with AI is the humans, not the computers. As someone said, computers are capable of processing quickly now. The main problem is how to program a learning AI! The sheer number of factors a human mind can take in, instantly process, and then take action upon is staggering...somehow, you have to apply this all into a program, something that is pre-written but can still learn and adapt. With something that has so many different variables and actions like RTW, this is extremely difficult. I imagine even producing a learning AI for something like chess is hard, and that has very strict, small, well-defined rules and actions.

Just look at it this way: does anyone really understand how the human brain works, how it learns, stores, and then applies information to further subjects and then repeats this cycle? And if you do, how could you apply that to something like a computer?

'Tis a perplexing problem. ~;) Don't forget that the amount of time, money and resources involved in developing such an amazing thing would no doubt be huge-and in the end, you have to make a profit on your game.

[A.Saturnus]

I think the biggest problem with AI is the humans, not the computers. As someone said, computers are capable of processing quickly now. The main problem is how to program a learning AI! The sheer number of factors a human mind can take in, instantly process, and then take action upon is staggering...somehow, you have to apply this all into a program, something that is pre-written but can still learn and adapt. With something that has so many different variables and actions like RTW, this is extremely difficult. I imagine even producing a learning AI for something like chess is hard, and that has very strict, small, well-defined rules and actions.

It depends on what you mean. Considered a rule-based AI you´re right to say that it´s difficult to let it learn. If we would have an algorythme that lets the machine learn which parameters mean what for RTW, we would have a powerful and quick AI, though the trouble is to find that algorythme. However, this problem does not exist for neural networks. Networks learn automatically, that´s more or less all they can do. They constantly change the association weights in order to optimize their results. By doing that, they will focus more and more on parameters that are important. The network just has to be big enough to represent all parameters. For example, the network decribed above that learned backgammon didn´t have any inventive thought included in its design, certainly not about good strategies of backgammon. It was a rather normal multi-layer-perceptron.
Neural networks do not use instructions, only input.