The AI ... some wishful thinking

[Revenant69]

This is just me thinking out loud about a topic of AI, or more precisely the PO (Programmed Opponent as someone put it).

To start off I'll ask this question:
Has anyone here played the game called "Galactic Civilisation" by StarDock?
If not then you definitely should, because it offers unparalleled challenges. I've played it long time ago, but I have only good memories about it. So, what can we say about the PO in Galactic Civ?

Before releasing the game, Stardock has held a betatest with lots of people participating. They carefully recorded the whole spectrum of different strategies that these betatesters exhibited, and ranked them from really shite ones to really good ones. Then, here comes the kicker, they programmed the PO according to the strategies that they got from this betatest, i.e. the PO on Easy would act as a dimwitted human, PO on Normal would act as an average human, ... and the PO on Very Hard would act as the top 5% of humans did in the beta test. Can you feel what this means? :san_shocked:
It means that you end up playing not as much against the computer AI as against a human AI, albeit at a reduced level. Nonetheless, even this "little" implementation has turned Galactic Civ into the game that everyone said was better than the long anticipated MOO3 (Master of Orion 3). If you doubt my words go out and buy Galactic Civ to see what I mean.

What am I leading to? Well, why couldn't CA do the same for RTW/MTW/STW? Instead of giving the PO static bonuses on Hard and Very Hard difficulties (which I find unfair), they could've implemented the same thing as Stardock. On Easy, the PO would act dimwitted, on Normal it would act as an average human, on hard it would act as top 15% humans and on very hard as top 5% humans in the betatest.
CA care to comment on this? :san_grin:

Well, maybe CA will do it for the next Total War title. I hope.

Now let's talk about the AI. AI stands for Artificial Intelligence. I'm pretty sure most of you will agree that humanity has a long way to go to develop the true AI. So far, the opponent in computer games is just that - the programmed opponent, PO. One way to improve the PO is genetic algorithms. You are gonna ask, what is that? Well, I'm giving you an example:

Imagine that the PO's general attacks your army and you go to the battlefield screen. the PO's general mutates it's curent battle algorithm (say algorithm A) and makes some changes to its battle plan. It fights you with this new battle plan and...wins. The genetic algorithm says "Hey, you've won. Maybe your tactic wasn't bad after all. Remember this from now on as algorithm B" The next battle the same general uses the algorithm B as its base algorithm and mutates it, i.e. makes some changes and wins. etc. etc. etc.

What if the PO loses a battle, i.e. army gets detroyed? Well, obviously the algorithm wasn't good enough but it is no longer an issue as the general is dead, i.e. removed from the genetic algorithm pool. Can you see what I'm getting at?

This would mean that only the strongest algorithms would survive during the game, moreover they'd improve (and this can be manipulated by the probability of mutation of the given algorithm) thus improving the overall difficulty of the game. Also, the gamedeveloper could evolve these algorithms inhouse before releasing the game so that way the PO wouldn't have to evolve from scratch when the end consumer starts playing the game. Developers could also set different starting algorithms depending on difficulties, i.e for Easy the PO starts with 0-base algorithm, on Normal it already has somewhat evolved algorithms and on very hard PO has highly evolved algorithms.

Genetic algorithms would thus have another added benefit: the replayability of the game would increase insanely as each game the PO might evolve in a different direction. Well, this at least somewhat approximates what the true AI should be.

What is my point from all of this? Well, being a great TW fan I just want for this game series to become better. Ideally I'd want all of the current TW titles reprogrammed with a new PO, either based off of betatesting or based on genetic algorithms.

Maybe it's just my wishful thinking.......

[AquaLurker]

Wow interesting read. Lets hope that your suggestions would be implemented.

[Mooks]

CA needs to do something about the extremely dumb AI, the only real strategy the AI has is stretching his line so far he flanks you, but then again all it takes is calvary turn that around.

[Butcher]

Play HOI2 if you want to see how an a.i can get better through subsequent patches.

[Zild]

I know I'm often beating on about the fact that games should be released complete, not requiring patches (or, worse, paid expansions) to make them work properly, but I must admit I'd rather see CA release a supplement that improves upon the existing PO than adds more units that will themselves require further correction in the future. The thought of poor PO is really the only thing keeping me from buying RTW (well, that and the fact I've never heard a good comment regarding sieges), but as far as I'm concerned that's reason enough - who cares how aesthetically pleasing a game is, when you get bored of it after a few weeks of play?

The thought of using genetic algorithms in games opens up a whole host of possibilities in my mind! Communities could share their GA-based artificial opponents, allowing players to fight battles against opponents who have already learnt the hard way from myriad other players, or the strength of the opposition could grow as a game progresses - imagine how tactics might evolve (historically and in-game) after the first battles using a new unit type. And were you suggesting that specific generals within a single game could have their own state of learning, distinct from that of other generals, that grows as the general does and dies with them... That beats the current star rating system, if you ask me!

Of course, the problem with that is the technical aspects of the GA system. The artificial opponent will only be as good as the system that grows it, and that will only be as good as the people who program it!

As for Galactic Civilisations - I saw this in Game a few weeks back and it piqued my interest, though I set it aside as 'just another strategy game' (I have so many on the backlog that I'm not sure I'll ever complete them - I'm not even sure I'll ever 'complete' MTW!) But I may reconsider it at some point, from what you've said... Thanks!

[Revenant69]

...who cares how aesthetically pleasing a game is, when you get bored of it after a few weeks of play?

Yeah, I feel exactly the same way. Although the graphics element in a game is very important to me I can get mindlessly bored from a dumb AI. I haven't got bored of RTW yet cuz I just started playing it :san_grin:


The thought of using genetic algorithms in games opens up a whole host of possibilities in my mind! Communities could share their GA-based artificial opponents, allowing players to fight battles against opponents who have already learnt the hard way from myriad other players, or the strength of the opposition could grow as a game progresses - imagine how tactics might evolve (historically and in-game) after the first battles using a new unit type.

This is great. You've suggested something that I haven't thought about. Of course the community could share their GA-based opponents, i.e. share savegames, thus adding to the game tremendously. Great stuff.


And were you suggesting that specific generals within a single game could have their own state of learning, distinct from that of other generals, that grows as the general does and dies with them... That beats the current star rating system, if you ask me!

Yes, that is exactly what I was suggesting. In fact I'll elaborate more on that. Firstly, the PO would have two components - the strategic and tactical. Here is the interesting bit: the strategic part of GA-based PO would be inherent to each faction; in such a way, the strategic PO of say Egyptians might evolve differently than the strategic PO of Gaul because they faced different enemies and learned different things etc. !!!
Now, each faction's generals, or army leaders (maybe even captains) would have their own tactical PO !!! Thus, the tactical PO of say Julius Ceasar would conists of highly evolved algorithms when compared to the tactical PO of some Green Shmuck general. Yes, this would probably eliminate the need for the current star rating system.


Of course, the problem with that is the technical aspects of the GA system. The artificial opponent will only be as good as the system that grows it, and that will only be as good as the people who program it!

Absolutely true. Thus, this would be a critical step during game design. The developers would have to ensure that their algorithsm can indeed evolve, and that the stronger algorithms survive, etc.


As for Galactic Civilisations - I saw this in Game a few weeks back and it piqued my interest, though I set it aside as 'just another strategy game' (I have so many on the backlog that I'm not sure I'll ever complete them - I'm not even sure I'll ever 'complete' MTW!) But I may reconsider it at some point, from what you've said... Thanks!

Great stuff :san_cheesy: Definitely give that game a go when you get a chance.
Cheers.

[DensterNY]

These are certainly good ideas that we would hope one day could be implemented to have an opponent that was thinking and learning and evolving. (Hehe, Terminator - anyone?) Not to be a nitpicker but such a potential enemy would probably be smarter than many of the real generals that existed in the time portrayed in TW's games.

The one thing about it is trying to program something of this complexity and magnitude in trying to control and react to countless variables in combat. Its not like programming a computer to play chess whereas there are set rules and patterns of movement - in TW's battles, as in war, anything goes.

However, computerized chess succeeds because you can create a gigantic, growing database of patterns so I suppose in some ways we'd have to try and mimic this.

[Ciaran]

Much as I agree with you, I don´t see it happen.


Absolutely true. Thus, this would be a critical step during game design. The developers would have to ensure that their algorithsm can indeed evolve, and that the stronger algorithms survive, etc.

That´s the breaking point. To assure that, it would require massive, time-intensive testing which is simply too expensive to conduct.

[Zild]

The one thing about it is trying to program something of this complexity and magnitude in trying to control and react to countless variables in combat. Its not like programming a computer to play chess whereas there are set rules and patterns of movement - in TW's battles, as in war, anything goes.

However, computerized chess succeeds because you can create a gigantic, growing database of patterns so I suppose in some ways we'd have to try and mimic this.
Actually, I was thinking about this earlier, and I have to disagree - I think the limitless possibilities in a TW battle make Genetic Algorithms more suited to it than a game with a much smaller number of possibilities, such as chess. In fact, I'd think that chess is actually too small for GA to work effectively. Also, whilst abstracting elements of TW would be required to allow GA to be used, I think that chess is too abstract to really benefit from it.

Ciaran, I agree with you it's not likely to happen. Any form of PO requires a vast amount of effort, and it is clear that CA have not been willing to invest that in RTW (for entirely understandable business reasons, I'd say). But there must come a point where the use of GA becomes more efficient than more traditional techniques. Given the potential complexity that the battles in TW games allow, I'd say this would be an ideal place to start using them. I can certainly envisage them being used in the future.

[hellenes]

I have a question for the Original Poster and everyone:
What is easier to programm a Multiplayer Campaign or an EQUALLY challenging Programmed Oponnent?

Hellenes

[Revenant69]

I have a question for the Original Poster and everyone:
What is easier to programm a Multiplayer Campaign or an EQUALLY challenging Programmed Oponnent?

I' not quite sure what is easier to programm, that is a thing to be debated. Although, all TW games have a singleplayer side, hence the need for PO, and the multiplayer side. It would be great to programm a Multiplayer campaign, but even the Multiplayer campaign would have the need for a quality PO as it may happen that not all factions are taken up by humans so some would be left for the computer to control, thus needing at least decent PO. So, I think, that the Multiplayer Campaign includes in itself the need for a good PO.


Much as I agree with you, I don´t see it happen.
That´s the breaking point. To assure that, it would require massive, time-intensive testing which is simply too expensive to conduct.

Well, that is a difficult thing for sure. But the whole idea of genetic or evolutionary programming isn't something terrbily new. There are already genetic and evolutionary algorithms out there made for specific tasks. It's just that they haven't been used in gaming industry. Yet, I hope. Do not forget, I suggested two possible ways to improve the PO. And while I agree that genetic / evolutionary algorithms are waaaaaay cool :san_grin: there also remains the option to programm the PO based on the results of the beta tests.
Think about this, CA made 1.5/1.6 patches based on the input that the gaming community, including the ORG, made. Now, imagine that CA has done a betatest, where the betatesters revealed the various bugs that plagued the game; CA corrects the game and the PO, and only then releases the game. The result would have been a well-polished game, yes it may have some bugs, but at least they won't be as bad as the Memory Leak or other nasties.

Maybe I'm asking too much???

Well, I've started this thread to show my thoughts on this subject and to see what other people think about it and how they react to these ideas. So far so good.

Cheers

[Geoffrey S]

If so much effort can go into programming an excellent looking game, I'd be surprised if that much time couldn't be invested in improving AI, at least to adjust the innate dumbness of the one in RTW.

What would be a nice thought is if a generic strategy AI could be developed which could learn in the way you said, by some seperate company, like the Havok physics. That combined with an extensive amount of options for the AI to use (say, suggested tactics for large small hoplite force against cavalry heavy enemy) would hopefully improve the game significantly.

[DensterNY]

Well, aside from undertaking the tremendous task that would be necessary for a genetic algorithm there is already evidence that the battle AI can be fine tuned. I don't know who on this board has been fortunate enough to catch some of the time commander shows on the BBC but basically the producers on the show had redesigned the battle thinking of the AI. Specifically the programmed the AI to fight like Romans and they do a pretty good job of it and usually beat the human players so I wonder why CA doesn't take a look at their example and learn.

First, it has been proven that the hardcoded aspect of the game can be improved upon so its not like they can't recode their engine. Second, have AI nations fight like their actual, historical counterparts. For instance, Hannibal used the double envelope tactics quite successfully on the Romans so have the Carthagians form lines with weak centers and strong flanks. Gallic fighters were supposedly one-trick-pony fighters... they'd charge at your lines until it broke or until they broke - not very ingeniuous but its what they did. The Mongols and the steppe horsemen very successfully used the I can hit you but you can't hit me approach with their horse archers.

Otherwise, as we have it now when you fight the AI its all the same... The Egyptians fight like the Spaniards, like the Greeks, like the Dacians, etc...

[GFX707]

Well, anything would beat the current experience. I just won a siege where the enemy, with 2 armies with 3 towers, 2 sap points and 2 rams, upon losing all the men destined for one particular sap point to arrows, just stood there for the entirety of the siege....and did absolutely nothing despite having all that siege equipment left. I had to sit there and wait for the time to run out.That's 7 minutes of sitting there staring at the screen. Great gameplay there.

[hellenes]

I' not quite sure what is easier to programm, that is a thing to be debated. Although, all TW games have a singleplayer side, hence the need for PO, and the multiplayer side. It would be great to programm a Multiplayer campaign, but even the Multiplayer campaign would have the need for a quality PO as it may happen that not all factions are taken up by humans so some would be left for the computer to control, thus needing at least decent PO. So, I think, that the Multiplayer Campaign includes in itself the need for a good PO.

Cheers

My question was what is easier to create without any PO...which isnt needed in case of a slow pased balanced realtime campaign map...

Hellinas

[Geoffrey S]

DensterNY, what you're suggesting wouldn't work outside historical or setpiece battles. Time Commanders can program the AI to emulate the historical tactics because they only have to do so for particular battles, whereas in RTW proper the AI has to be able to cope with an in theory unlimited number of battle scenarios due to varying terrain and army compositions. The best that can be done is allowing the AI to recognize which units are vulnerable to certain other units, and when to commit reserves or attempt to flank with cavalry, but it remains a complex job to get a computer to be flexible or recognize and utilise the necessary tactics.

[Zild]

Hellenes,

I would say that the multiplayer campaign would be far far easier to program in itself. But there are two major problems with it:

1: As Revenant69 said, you'd most likely still need to have a programmed opponent within the campaign mode (probably a lot easier to do than in battle mode, I'd imagine).

2: Most people tend to shy away from multiplayer campaigns, and it's hard to get a system that people like. My experiences of Civ3 were that different people liked different kinds of campaign style, hence why that game had so many different multiplayer modes. Each mode also had a number of different settings. So adding (and testing) multiple styles of campaign mode with multiple options each increases the challenge considerably. I'd also say that the generally slower pace of these does not appeal to the more casual gamer, at whom RTW was aimed - I think that may have played a big factor in their decision as to whether or not to include a multiplayer campaign option.

[DensterNY]

DensterNY, what you're suggesting wouldn't work outside historical or setpiece battles. Time Commanders can program the AI to emulate the historical tactics because they only have to do so for particular battles, whereas in RTW proper the AI has to be able to cope with an in theory unlimited number of battle scenarios due to varying terrain and army compositions. The best that can be done is allowing the AI to recognize which units are vulnerable to certain other units, and when to commit reserves or attempt to flank with cavalry, but it remains a complex job to get a computer to be flexible or recognize and utilise the necessary tactics.

I agree that it would be terribly complex to try and individualize particular armies to fight a certain style but I guess its really wishful thinking... however, if you put into place stylized fighting as opposed to a general battle engine that fails to work anyhow at least there would be some rhyme or reason to the AI's battle tactics.

Perhaps, that is the reason that the AI behaves as it does now because they've tried to program for as great a number of battle scenarios as possible as opposed to setting strict guidelines. I hate to keep comparing RTW to MTW but in MTW the AI mostly followed solid battle strategies. Of course, after two dozen battles you'd learn different tactics to beat the AI but you'd have to fight like an army because you faced a functional and somewhat organized force. Unfortunately, in Rome now the AI doesn't seem to know how to control or utilize its units much.

[ShellShock]

I agree that it would be terribly complex to try and individualize particular armies to fight a certain style but I guess its really wishful thinking... however, if you put into place stylized fighting as opposed to a general battle engine that fails to work anyhow at least there would be some rhyme or reason to the AI's battle tactics.

Perhaps, that is the reason that the AI behaves as it does now because they've tried to program for as great a number of battle scenarios as possible as opposed to setting strict guidelines. I hate to keep comparing RTW to MTW but in MTW the AI mostly followed solid battle strategies. Of course, after two dozen battles you'd learn different tactics to beat the AI but you'd have to fight like an army because you faced a functional and somewhat organized force. Unfortunately, in Rome now the AI doesn't seem to know how to control or utilize its units much.

I thought everyone knew that Time Commanders was against a (hidden) human opponent? (despite what you may have heard to the contrary).

[gardibolt]

No, that's the first I've heard that. Do you have a link? Or is this just gossip? It would make sense, though.