Investigation of AI reassessment upon reload

Over the past month or so, a new game mechanics issue has arisen: does the AI reassess its actions after a reload of a saved game?

This thread is for investigating that issue, with the eventual aim of understanding the observations already made by many of the community. To be most effective, especially reading some posts from CA staff members, the thread must be as concise, relevant and conclusive as possible. Please try to be as complete and accurate as you can be before posting.

Also, take some time to familiarise yourself with the posting guidelines for this forum. Any post, or part of a post, which is not related to research of this issue will be removed from this thread.

Edit: In answer to Pode's first post, the sort of things we are looking for are: detailed observations, repeatable experiments, community attempts to gather large datasets, analysis of that data, and hopefully possible workarounds.

Which is a very long-winded way of saying, yes, that's fine! :thumbsup:

added by A.Saturnus

It seems the discussion around this issue has come to a halt now. Therefore I post the conclusion I have drawn later in the thread here. The participants of the discussion have more or less agreed in general to this conclusion. Only the wording has drawn objections. But for simplicity I will leave it as it is. The objections can be found later in the thread.

• It is certain that after loading a savegame, the AI will most of the time break up ongoing sieges in cases where it doesn't do that without save/load.
• It seems that the objective to take the previously sieged city is not lost entirely, as the AI will usually try to reengage the siege the turn after the load.
• As a consequence, saving and reloading will affect the development of AI factions, at least in the beginning of a campagne.
• It is not yet clear to what extent the course of a campagne is influenced in an undesirable way by this. AI factions do conquer new territories at a slow pace, but it is not clear whether this is due to the save/load issue or general weakness of the AI. There are three possibilities:
  
  • Above the weakness of the strategic AI, this issue is irrelevant as its effect is only noticeable under extreme conditions.
  • Saving and reloading often does noticable affect gameplay, but doesn't make the game unplayable.
  • The effect is so pronounced that it can be called a "game-breaker".
• A clearly noticable aspect is that, when saving and loading often, AI factions will not take the opportunity to capture rebel territory to the same extent as it does in continued playing.

It should be clear that point four is the most important aspect here, as it concerns the impact the issue has on gameplay. From reading the evidence in this topic everyone may come to his own conlusion on this.

Not sure if this is the type of post you're after here, therother, but this was what convinced me.

RTW 1.2, mods to build file only, M/M campaign as Britons, toggle_fow off. Zero player activity after first turn, simply hit end turn until an AI laid seige somewhere. When that happened, I hit ctrl-s, then ctrl-l, then end turn. Test ran to winter 265. 13 of 13 sieges were broken off by AI, zero provinces changed hands.

Have not tested the protectorate acceptance implications yet.

I'll quote my post from the thread in colosseum where this was discussed (minus the snide remark at the end):

Quote:

---

A suggestion for a test:

Start a number of campaigns always with a same nation (one that is regularly unlockable) in vanilla unmodded RTW 1.2. Play a number of turns without reloading and without doing anything. Toggle FoW off, and count the number of provinces that have changed hands. Now do this with saving and reloading every time, and after the same number of turns count provinces that have changed hands. If a few of us do the same thing, take the same nation, play the same number of turns, use the same counting method etc. everything should be controlled for, and we would get a bigger sample size in a less time-consuming fashion. So, we would have to agree on the faction, No. of turns, No. of tests each of us would do, etc., but we would have respectable dataset in a short time, and without any of us spending several nights just clicking end of turn button.

After we gather all the data I volunteer to do the statistical anaysis on it and then we'll have cold hard evidence that something is going on. And no more "probability theory" arguments either - as we would have solid basis to counter them (another reason why we need a large sample size).

---

If people are interested, I'd be happy to elaborate further. Personally, I'm not sure it will change anything before XP though (if at all), but I'm willing to lend my time to this investigation.

Reposting from the Colosseum thread in case not everybody who is interested is reading both threads;

__________________________________________________________________

Ok, so, for those who are interested, here is the setup I would recommend:

- use umodded RTW 1.2; this means no custom mods either; we want everything to be the same and pristine in every experimenter's case so as to make the data cross-comparable
- take the Britons (thanks rcp ), medium/medium imperial campaign
- for the control, play 15 turns without doing anything; this literally means don't touch a thing other than the end of turn button; 15 turns is enough to bring two consecutive sieges that ended in starving the garrison out, I believe, hence I think this is a good number - not too big, not too small
- once 15 turns have passed (you hit end of turn for 15 times, to be exact), toggle FoW off and count the number of provinces that are not owned by the original owners; this means rebel provinces as well; this requires you to know beyond any doubt who was the original owner of the province - write it down beforehand if you think it's going to be necessary, or something to that effect
- for the "treatment" group, do everything like you did it in the control group - Britons M/M, hit end of turn (EoT) 15 times, don't touch anything - with the exception that now each turn you hit quicksave(QS) and then quickload (QL); hence, start - QS -QL - EoT - QS - QL -EoT -... etc. until you have hit EoT 15 times; at that point, turn FoW off and again count number of provinces not owned by the original owners
- if at any point in either your control or treatment campaigns you get attacked, abort that campaign, and start anew; we don't want to interfere at all, even if it's only autoresolve, and hence discard that campaign and instead do a new one fromt he begginging until you have the 5 controls (without reloads) and 5 treatments (with reloads)
- make 5 separate control campaigns (each running from fresh start to 15th EoT), and 5 separate treatment camapigns (again each running from fresh start to 15th EoT); save the campaigns at that point to a regular save, zip all those saves (you should have 10 of them, and this shuldn't take up much space) and keep them on your HD for the time being inc ase we need to send these to someone or verify something
- PM me the results, but keep in mind that I can only have 5 PMs at a time, so if I don't write back saying I got it in a day, PM me again; I'll do the analysis of the results, and post them here; I will not soup up the results, or do anything else to them other than analyze them - I do this kind of stuff for a living, and I know how to keep my personal feelings out of it, believe me


It is paramount that we all do it the same way, so please, if you think that you won't be able to do it in the way we all agree to do it, do not send me your data. I really do not want to deal with falsifications, souped up data for whatever reason, results that were collected through deviation of the basic rules, and similar things, as this would invalidate everybody else's effort as well. Let's do it right, or not do it at all.

This should tell us whether there is something fishy going on or not with saves.If we can't get 10 people (well, 9 other than myself), maybe each of us could do more runs. Say, 8 or 10 per control/treatment. Added bonus to this is that it cannot be said that it is due to a single PC configuration, or having FoW off all the time (which is a cheat and hence not an out-of-the-box game functionality). Please feel free to comment on the guidelines I posted above. If we agree, and there are no further comments until about 6pm GMT Friday, we can get to work, and by the end of the weekend, we should already be able to discuss the results. ~:cheers:

Cheers,

Another avenue for testing: can we predict when the "periodic re-assessment by the AI" will happen?

If we could, then we could remember to save only at those points (say, every 5 turns, or whatever). This would minimise the effect of the re-assessment on re-load (since the assessment would have happenned even if we didn't reload).

Problem 1: how do we tell whether there has been a re-assessment? Using broken sieges as our best indicator for re-evaluation, how about: turn FoW off and watch the map like a hawk for broken sieges on every end-turn, log such events, including the AI involved, over a large number of reasonably long (50 turns?) campaigns, and look at statistical patterns (it will have to be statistical, because sometimes there won't be a siege in progress when the re-eval happens, and sometimes the siege won't be broken).

Problem 2: it is quite possible be that (absent re-loads) AI re-assessment is staggered through the turns (AI#1 assesses on turns 1, 11, 21, etc; AI#2 on 2, 12, 22; ... AI#10 on 10, 20, 30 etc.) so there is never a "good" time to save. The test above should show this as separate repeating patterns for the various AIs.

Thoughts, anyone?

Also, we should standardise unit size as well: Britons/M/M/Normal?

Quote:

---

Originally Posted by mishad

Another avenue for testing: can we predict when the "periodic re-assessment by the AI" will happen?

---

I'm not convinced the "re-assessment" only happens once in a while. Surely the AI would perform exactly the same processes between each turn. One of the most obvious examples of this issue is that AI leaves sieges and then goes immediately back to them. That kind of behaviour is not really indicative of the AI reassessing its strategy on reload (per the Shogun post) and then again at the beginning of its second turn. It's more indicative that for the first turn the AI doesn't know what it's doing and then it does a proper reassessment for the second turn.

If this is the case and the AI reassesses its position each turn, it would be useful to know at what point this happened. Of the viable times (beginning of its turn, end of its turn, end/beginning of year) it seems most likely that it occurs at the end/beginning of the year - any other time would not fit in with the available evidence.

A useful exercise would be to use Myrddraal's hot seat mod to halt the turn in the middle of the slave faction, save the game there and reload. The game should then reload from the slave faction's go under AI control, go over the year end, perform the proper re-assessment (hopefully!) and then go to the player's turn. It should be clear enough whether the AI has maintained a siege that on a normal save/load it would have abandoned.

Reposting from the original Loadgame bug thread at the .com, now in the graveyard. I posted this on 27/02/05 on page 5 of that thread

Quote:

---

Na, I am sure the problem wasn't as severe in v1.1. I continued an old campaign from 1.1 after installing 1.2. The AI was doing just fine, specially the Romans. They had made substantial gains, specially Brutii and Julii (I being Parhia never met them in combat till v1.2). After installing 1.2 I couldn't understand why Brutii which was so strong couldn't even take the two rebel settlements it would lay siege to every other turn and then break away. It went on for 50 or so turns. I play very slowly and micro-manage everything, hence quick saving/loading is a norm for me. This was true for 1.1 and never hurted the AI like this.

I did extensive testing after reading this thread. I never jump on the band wagon without first testing something myself. I have too much faith in CA to flame them without good reason (well I still can't flame them, they hold a special place in my heart... hehe). My tests confirmed that AI armies not only lift sieges, it always tends to completely walk away from the said settlement as if it has completely forgotten what it had set out to accomplish. Only to repeat the process ad-infinitum.

I thus played yesterday for 8 hour at a stretch without loading once and because i am quite further down my existing Parthian campaign, I only got some 5/6 turns done. And guess what, the Brutii finally took those two rebel settlements that they couldn't for the last 50 turns. They also laid siege to two other towns. Same with Scipii and Julii who each took one settlement and laid siege and maintained it, to atleast three more. However, after 8 hours, i had to save and go to sleep, and when i reloaded later, the AI had forgiven/forgotten its enemies/goals and walked away from all its previous sieges. When I say walked away, they had moved their sieging armies a whole turn length away from the previously sieged settlements.

Hope this helps.

---

here is the link

http://p223.ezboard.com/fshoguntotal...rt=81&stop=100

EDIT: Reposting another test i did and posted on 3-01-05 on page 7 of the above mentioned thread, this was while trying if there was any difference with autosaves/quicksaves/regular saves and difficulty settings.

Quote:

---

Sorry for all that fuss guys. I just play tested autosaves on VH, and the bad news is that the bug is very much present in autosaves just like quicksaves etc.

I played my existing Parhia campaign for atleast 6 turns without loading and tested on a turn which had atleast 5 settlements undersiege, I checked them all repeatedly (by re-using the same autosave; manually copy pasting it back to check), and every single time the AI lifted all the 5 sieges as i pressed 'end turn', without any nearby threat.

So thats it for VH. Lets see if you guys have any luck on any different difficulty levels.

---

link is
http://p223.ezboard.com/fshoguntotal...t=121&stop=140

hrvojej, your control/test concept is perfect imo. With enough numbers, we can do a simple statistical analysis (t-test?) to confirm that the AI's reassessment routinely results in much fewer provinces changing hands.

Might I suggest an improvement to the test though? Instead of manually noting the province allegiances, maybe a screenshot of the campaign-map mini-window on the bottom left at the beginning of the test and end of the 15 turns would improve accuracy. Furthermore, perhaps proper random selection of who tests with saves and who doesn't is worthwhile to remove any doubts of bias?

Finally, is there thread for participants to sign-up and post their results somewhere on the forum?

[edit]I should read better. 5 and 5 per tester and PMs it is.[/edit]

Quote:

---

- PM me the results, but keep in mind that I can only have 5 PMs at a time, so if I don't write back saying I got it in a day, PM me again; I'll do the analysis of the results, and post them here; I will not soup up the results, or do anything else to them other than analyze them - I do this kind of stuff for a living, and I know how to keep my personal feelings out of it, believe me

---

You can PM your results to me. My mailbox is bigger.

It would be ideal if FoW is toggled off all the time (including the controls), so every single reassessment can be spotted. However, please add to your results a description whether or not your trials have been exactly as hrvojej has layed out.

Probably a lot of you have seen this, but it certainly constitutes evidence that there's a problem: Protectorate: Total Peace.

-Simetrical

The problem I see with Brits is that very soon will Gauls or Germans start seiging your mainland province.

Go out and make an alliance with them both using your lone diplomat, that should hold them off long enough to complete the test, and I don't *think* CA will argue that the approach of a diplomat with no money is enough of a threat to make the AI break every siege on the map.

Or you could simple give province as a gift to one of those two factions...

Quote:

---

Originally Posted by A.Saturnus

It would be ideal if FoW is toggled off all the time (including the controls), so every single reassessment can be spotted. However, please add to your results a description whether or not your trials have been exactly as hrvojej has layed out.

---

True, but then you would have to scan the map all the time after every turn, instead of just doing it once at the end. Also, you would have to have FoW on all the time, and you wouldn't be able to escape arguments that the game was not intended to run without FoW and that the AI is lifting sieges because it's shy or something... ~;) In sum and a priori, I don't think that the information you would get from countinuous sampling outweighs the number of trials you can do by scaning at the end. I know what you're saying, but I predict the effect size to be so large that it won't really matter.

Do people have a lot of problems as the Britons from being besieged? I don't have time right now to do a lot of trials, it ended up being a busy end of the week, but I ran the control twice and was not besieged in 15 turns. Maybe I was just lucky? In any case, I thought that the suggestion to take Britons was a good one, since you're away from the action, and in addition to not being involved yourself you won't stand in a way of expansion of others. Another faction I can think of that also might be good is Parthia, but I have to admit that I have not played the regular campaign with either of the two, so I don't know how likely you are to get attacked.

And to answer therother's point about modding the game: I think that moving a single faction's name from nonplayable to playable factions list is really nothing we should concern ourselves about. I do however think that it is important we pick a faction that is unlockable through regular means, as it would seem that those which are not have some unfinished quirks about them. The only thing that remains to be seen is if with Britons you cannot avoid being attacked all the time, then we should consider other factions like Parthia. I'll test it as soon as I have time (tomorrow night it seems).

Cheers, and thanks to all who intend to participate ~:cheers:

Quote:

---

True, but then you would have to scan the map all the time after every turn, instead of just doing it once at the end. Also, you would have to have FoW on all the time, and you wouldn't be able to escape arguments that the game was not intended to run without FoW and that the AI is lifting sieges because it's shy or something... In sum and a priori, I don't think that the information you would get from countinuous sampling outweighs the number of trials you can do by scaning at the end. I know what you're saying, but I predict the effect size to be so large that it won't really matter.

---

If not, the hypothesis becomes questionable anyway. I suppose we do both. For now, we should do it as you suggested to test our hypothesis. If it is proven beyond the shadow of a doubt that the bug exists, we can explore it to more detail by investigating individual AI actions.

Quote:

---

Originally Posted by Simetrical

Probably a lot of you have seen this, but it certainly constitutes evidence that there's a problem: Protectorate: Total Peace.

-Simetrical

---

I see that Simetrical has already posted the link, but I wanted to elaborate on the exact procedure I followed to gain the protectorates documented in the screenshots.

I started a medium/medium game as the Julii. I split my armies into individual units and built nothing but diplomats, ships and calvary. Each diplomat would travel with one of the army units. When the diplomat encountered a faction that we were not at war with, I would use the army unit to besiege the town and immediately lift the siege (or attack the faction's army) just to get the protectorate option on the negotiation screen. The diplomat would then ask the faction to be a protectorate, offering nothing in exchange. The faction would refuse. I would then use control-s to save and then control-l to load. I would have the same diplomat talk to the same unit or town and make the same offer of a protectorate for nothing. In every case, the other faction accepted.

Occasionally, I would already be on a turn that I had just loaded. When I encountered another faction, I could simply besiege, lift siege and then offer protectorate status and they would accept.

Quote:

---

Originally Posted by hrvojej

And to answer therother's point about modding the game: I think that moving a single faction's name from nonplayable to playable factions list is really nothing we should concern ourselves about.

---

Okay, just checking to see if you thought it was okay. I think the Brutii are perhaps the best choice, given their sheltered location, the fact that they are unlocked to begin with, and that they are a Roman faction (it is Rome: Total War after all). Of course, that last bit works against choosing them to some extent as you also have the Senate AI hardcode possibly interfering with things. Perhaps we can do both, and see if there are any significant differences?

Just a basic test along the lines hrvojej mentions. I freely admit that I haven't counted the provinces that changed, but compare the world maps. The start one looks the same as the save/load one, whereas there's been a lot of movement on the no-save one.

Also, for a smaller close-up view, see Sicily. No loads meant that the Scipii went crazy and kicked everyone off. Save/load each turn and all three factions trade love notes.

1.2 Vanilla. Julii. Both tests over 10 years (20 turns).
Move armies into town. No other actions. Respond "no" to everything.

Starting Position
http://img174.exs.cx/img174/5283/270bcstart0kh.th.jpg

Save/load every turn
http://img174.exs.cx/img174/1363/260...turn8om.th.jpg

No loads
http://img174.exs.cx/img174/7643/260bcnoloads3dg.th.jpg

There's been some talk about a similar issue in GalCiv over in the thread in .strategic. Starting post is the Gerry Quinn response to my OP, currently no. 30.

Just reproducing Arphahat's and Camp Freddie 1969 protectorate posts from the .com forum, just in case it disappears. I know that Sim already linked to the screenies, but some of this text might be useful.

Please note that I have not personally confirmed any of Arphahat's work, although beezer says that he has.

Quote:

---

Arphahat.
There is a simple technique for winning the game. Simply siege a city of a rival faction, immediately lift the siege, save the game, reload the game and then have a diplomat ask the rival to become a protectorate. This works every time, without exception! Look at my glorious conquest in picture form below.
www.stampor.com/Rome/exploit.html
EDIT: here is a link to a list of the images, instead of loading all at once, if it helps www.stampor.com/Rome/exploit_links.html

Question.
One question I have is what happens when 2 of your protectorates go to war with each other. Are you forced to choose 1 and thus you loose the other???
You lose one of the protectorates. But, no problem; just make sure you keep a diplomat next to each of the factions. If you ever lose a protectorate, simply save, reload and ask again.

Camp Freddie 1969.
I found that if I declared war then immediately asked for a protectorate (before hitting end turn), I would always get one due to the much lauded re-evaluation feature.
However, I also tried it on the Thracians, with whom I've been fighting a long war. I outnumber them by a huge margin and regularly spank the armies they send my way with losses that are >10:1 in my favour. I've always been suprised that they don't accept being a protectorate in normal gameplay.
So I asked them after a reload - and they refused! Could it be that the auto-protectorate 'feature' is broken?! Then I looked at the refusal message:
"We cannot accept such generosity, since we have nothing to offer in return".
That's right folks. They refused because the AI re-evaluation decided that my demand of making them a slave nation was 'too generous'. They found themselves not worthy of my benevolent (er, yes, benevolent) rule!
After moving on a turn, I asked again and got the normal response of, "We are proud of our freedom and do not give it up lightly...."
So the re-evaluation feature seems to be totally underestimating the strength of the AI nations.

---

could we workaround this issue by changing all the faction's AI personalities to aggressive ones?

I can also confirm Araphat's protectorate issue. I also agree that using the Brutii would likely be the best choice for this test if we are to ideally do nothing but press end turn during our testing.

At the forum in twcenter (http://www.twcenter.net/forums/index...c=27076&st=108) Pode wrote an interesting discovery:

"The AI will almost always, upon a reload, abandon a siege and run as far away as possible. Unless you interrupt it again with another load, however, it almost always returns and lays siege again the next turn."

Has any of you had the same experience? It could be a workaround for the bug to minimize the negative effects, allthough it will not eliminate it.

I never had any issues with the orginally and the first patch on this sieging of cities bug. Now that I have put in the last patch this issue shows it ugly head. I think it is pretty simple: they fixed one issue and created this issue. Now they say that the AI has changed its mind and wants to move the resources some place else. I have news for them I did the save game I was Seleucid and two of my cities were being sieged by Egypt. After I restarted the game the Egyptian left my cities and came right back in the same move. Now did they change there mind and than said no go back to the city!!! I can't see Alexander doing that at the sieging of tyre. :embarassed:

Last night I ran a test using v1.1 to see if this was introduced with v1.2. The no-expansion issue is just as prevalent (didn't try the protectorate issue, that was bugged in 1.1 anyway). I played 20 turns for each, Med/Med, as the Britons (unlocked, not mod), FOG_OF_WAR:FALSE in preferences.txt.

With no saving/loading, the factions expanded normally, eating up the rebel provinces and fighting it out with each other.

With a save and load each turn, not much happened. Diplomats and stacks moved around, rebel stacks got pounded, but only one province changed hands, I think (Brutii took Thermon, no siege, just walked right in). It seemed like after a load, the AI would do whatever it could short-term within it's interests (mainly crushing rebels). I don't think I saw any sieges lifted, mainly because the AI never got around to sieging cities in the first place. This would probably be present if I waited 3-4 turns between saves/loads.

I have screenshots at each turn for both test cases, the end result is pretty much the same as Bromley's. These results lead me to think that the reassessment is an integral part of the game, just as the .com post said, not a new "feature".

On the FoW issue, it would be interesting if CA tries to use Heisenberg's Uncertainty Principle as an excuse. ~D

Quote:

---

Originally Posted by MajorFreak

could we workaround this issue by changing all the faction's AI personalities to aggressive ones?

---

The evidence collected so far indicates that this issue applies to all factions, not just those with a less aggressive personality.

Quote:

---

Originally Posted by ramboost

"The AI will almost always, upon a reload, abandon a siege and run as far away as possible. Unless you interrupt it again with another load, however, it almost always returns and lays siege again the next turn."

Has any of you had the same experience? It could be a workaround for the bug to minimize the negative effects, allthough it will not eliminate it.

---

Yes, I noted the same experience in my post above. But even if the AI immediately returns to the siege it would be at least 3 turns before it could assault. This issue particularly effects users who can only play for one or two turns at a time. Users who play for longer stretches are less affected by this issue in any case.

Quote:

---

Originally Posted by drone

On the FoW issue, it would be interesting if CA tries to use Heisenberg's Uncertainty Principle as an excuse.

---

:laugh4:

Quote:

---

Originally Posted by Epistolary Richard

Yes, I noted the same experience in my post above. But even if the AI immediately returns to the siege it would be at least 3 turns before it could assault. This issue particularly effects users who can only play for one or two turns at a time. Users who play for longer stretches are less affected by this issue in any case.

---

I agree that it certainly does not solve the issue. Personally when I play I play an average of minimum 10-15 turns. If this "workaround" can minimize the amount of turns the AI waste, it is definetely worth pursuing from my point of view.

Unfortunately, I do not have the time to test it until sunday, so I cannot verify the AI behavior descriped by Pode.

I don't quite understand what you're suggesting. What would people do differently to try and work around the issue? Just play for longer?

Don't use the quick save at all! Use the normal saves, and it should be okay! Are you guys planning on testing that as well?

ie, 15 turns of doing nothing, versus
15 turns of saving and reloading using normal saves?

Maybe quicksave was MEANT not to save the army status (standing or laying siege) when on the campaign map, therefore you lose the siege, but you gain the benefit of quicksaving and loading.

Um, I hate to say this, but the proposed test does not seem to impose any controls which eliminate other possibilities.

For instance, is it possible that as part of game theory that the AI factions are approximating the aggressiveness exhibited by the human player? This is fairly commonplace in computer games. For one thing, it allows players of all different types to stand a chance at winning the game. For another, it offers gameplay which is compatible with the style of the human player. If the human player merely "turtles in", why shouldn't the AI do the same? There is no percieved threat as the human player is not yet being aggressive.

Second, the AI might use a "delta factor" to determine it's reassessment of moves -- in other words, the degree of change from the previous move prompts the AI to make decisions regarding the current move. Maintaining the status quo, as the human player seems to be doing, is not actually a losing scenario. With a delta of zero, the AI factions have an expansion factor of nearly zero.

The fact of the matter is, I have performed the proposed test and came up with identical results -- none of the territories change hands. However, in a REAL game, in which I actually do save and load every single turn -- usually multiple times per turn -- and which I play aggressively, each AI faction actually does seek to expand its empire.

So I don't think this test is a valid proposal. It needs more thought.

I'm a seasoned programmer myself and I've read through some threads by other programmers theorizing on the nature of this problem and one of them was pretty knowledgable. He stated that in order for the AI to remember its long term goals that its planned actions would need to be "serialized" to disk (meaning, stored in the save file along with the positions of units, etc.) which is a nontrivial programming issue because the information is stored in memory using objects that are linked together by pointers. He's probably absolutely correct about the data representation, but the truth is that serialization of such objects is actually not a complex task. I have done it numerous times with numerous software products -- and new languages like C++ make it a little easier since it's sort of a built-in feature of the language. It's actually a rather mundane and simple task when compared with some of the more complex programming that is tackled in a computer game, as another programmer pointed out to him. So I don't think the explanation that the programmers thought it was too difficult or too much work to save information in the save file is a good one. It would only seem the obvious thing to do so that a save game starts where it left off and merely requires a little grunt work.

One issue which even most of the programmers were a little confused about seemed to be the idea that the game looks ahead a given number of turns to make long range plans -- some of them were stating possibly as much as 20 turns. Have you ever played chess? One of the measures of a good chess player is how many half-moves he can look forward into the game. Five is considered good and I believe about seven or eight is the record for a human being. Chess programs do the same thing. So looking ahead 20 full turns is a little unrealistic on a map this size and with this many pieces and factions ("players"). The AI factions might have general goals but they cannot plan ahead for specific movements for too many turns because they need to react to the human player and the other AI factions. So it's not an issue of serializing detailed, long-term movement plans for a projected 20 moves; simply a matter of storing generalized objectives in the save file.

Taking all of this into consideration, if the AI is reevaluatiing its position as CA claims, then it must have elected a NEW set of campaign objectives when it relieves its sieges. These new objectives must take into consideration the fact the besieging faction is at war with the besieged faction and that trade is no longer possible with them. It must be able to find a more desirable alternative in light of that problematic issue. Does it attempt to negotiate a ceasefire and/or trade rights in the same turn? It would be interesting to analyze a move in which the AI has abandoned it's previous objectives and attempt to determine which objectives it has identified as more desirable.

By the same token, if the AI is reevaluating its military objectives it must also reevaluate its economic objectives, since the two go hand in hand. Is it canceling or changing building and economic development to match the military changes?

Also, does anyone know where I can read CA's official response regarding this issue and other bug reports?

rouge all CA currently stands in this issue is found here: http://p223.ezboard.com/fshoguntotal...ID=24377.topic

I don't think we need a control for player passivity, as the player will be inactive in both tests, i.e. it's not a variable. It may well be interesting to try and devise a test for AI response to player actions, but I think it is unnecessary at this stage.

Well, then, that entirely discards the relevance of a real game, in which the player is actually making moves, but also loading and saving each turn, which is the way most people probably play. To simply refuse to move at ALL for 20 turns is to simply allow it to examine the starting position over and over again and decide amongst the many possible opening moves, but not to allow it to follow up on those openings because you are giving it nothing to "react to".

Most AI tends to be reactive to the human player -- sometimes called "triggers" in the gaming world. If you don't give it anything to react to, any kind of predictor on how you intend to play, it could very well just bide its time until you do. If a "load game" behavior triggers that particular AI behavior, it doesn't really mean much to me when I'm playing a real game and making moves. Because each turn I give it new information to react to when I make my moves.

You may well have a point, but IMO it does not undermine the fundamental reason for this test. We are attempting to test if the AI acts differently in the two cases, where the player is triggering AI in exactly the same way, with the only variable being the saving/loading of the game in one case.

rouge the AI factions already react to each other... At least that is obvious from the non-save/load games. That we remove a single faction is of little consequence as the game is supposed to actually feature non-reactionary regions (rebels) and dead factions. One faction out of 21 is not much, and in general only means another 2-3 'rebel' provinces.

From all my games it doesn't seem that the AI factions react any differently to us than to all the other AI factions. We are 'just' another faction. And the AI factions have plenty to react to, each other.

Unfortunately, I think you're seriously wrong about that and it was one of the main issues I pondered over when I first played MTW. The faction AIs do indeed seem to act as seperate entities, but there seems to be an overriding AI governing the game in a certain direction for the benefit of the human player. I've always thought this is one of the reasons they've skirted around the prospect of a multi-player version of the campaign game. As long as it is not designed to be multi-player, balancing the relative strength of different factions on the campaign map is unnecessary. It allows the game developers to take liberties to impose situations on the human player. For instance, it would be simple for ANY bordering faction to simply crush the Polish faction in MTW early in the game. But they don't. Otherwise, it wouldn't make much sense to try to play the Polish as a human player, right? So this AI "overseer" makes sure that doesn't happen. I think we get a small glimpse of him through the Pope and the Senate, wherein he tries to direct your actions as well.

After you reach a certain skill level in the campaign game, the AI is never really that challenging, but what it can do (if you allow or encourage it to) is set up some really interesting scenarios.

For the benefit of newcomers to this issue, let me try to distill the past history of testing that lead us to this point. Not all of these points have been established to the same level of proof.
-Standard test methodology has been as described below. Run for a bit just ending turn and note the number of provinces taken or sieges prosecuted if you had FOW off. Run the same campaign saving and loading every turn and watch things fail to be conquered and the AIs move defensively or aimlessly.
-Quicksaves, autosaves, and named saves have all been tested and found to display the same behavior.
-Loading from the manual interface and loading using continue campaign have been found to produce the same behavior.
-Loading every other turn has been found to catch the AIs in a loop of alternately laying siege on the turn not preceeded by a load and lifting siege on the turn preceeded by a load, indicating that the decisions produced after a load are materially different that normal even in very similar situations.
-Timing of the load within the player's turn if the player is active has been shown not to have an effect.
-Player inactivity with loading produces AI inactivity. Player inactivity without loading produces AI activity.
-Two loads per turn without player activity does not appear to affect the AI's pacifism after loading in my VERY limited test last night. The fact that Roguebolo loads multiple times during a turn and has yet to see a single AI siege lifted in his campaigns indicates that this one needs testing badly.
-AFAIK, RB is the only person who has experienced the bug in testing but not in campaign.

That is the body of fact as I see it. My understanding of the purpose of this thread is to gather statistical evidence of these facts instead of the semi-anecdotal evidence currently available. Hypotheses that can explain these observed facts are probably welcome as well (especially testable ones), although that is the moderator's call. Personally, I'd be particularly interested in anyone that can explain or duplicate RB's results. What makes him special that his AI's work? ~:confused:

Quote:

---

Originally Posted by therother

You may well have a point, but IMO it does not undermine the fundamental reason for this test. We are attempting to test if the AI acts differently in the two cases, where the player is triggering AI in exactly the same way, with the only variable being the saving/loading of the game in one case.

---

The goal seems to be to confirm statistically that there is in fact a difference, and only then evaluate whether that difference has a gameplay impact, correct?

Well, something must trigger the AI to be aggressive again. Possibly activity on the part of the player?

To be fair, I must admit that I never "sit out sieges". I always rally, usually on the turn after being besieged and never later than the turn after that.

I should mention that in my current campaign, which I'm playing as the Julii because I reinstalled RTW and Patch 1.2 from scratch, I still have the save games from just about every turn in the game. I think I will go back and take a look at them and turn off FOW and see what's happening with the OTHER factions -- the stuff I can't see.

*edit*

Actually, I just thought of another load game issue which I've always thought was unrelated but might not be. You know how random events can happen when you select End Turn? Well, if you don't like the outcome of those events, you can do a load game. If you just hit End Turn again, you usually get the same random events. But if you make ANY change -- say, move a noncrucial diplomat or something -- it will re-roll the dice for all of those random events, and you will get a completely different set of results.

Perhaps this is related, perhaps it is not. But the change definitely occurs only if there is player activity AFTER the load game and BEFORE the End Turn.

Looking back through my save games, I have found that I saved each and every one of my first twenty moves immediately prior to hitting "End Turn", so I can track each faction's activities for the first 20 turns. However, FOW doesn't seem to want to be disabled if it was originally enabled.

By turn 20, the majority of the nations were still in control of only their starting territories and had not engaged in warfare with other factions, with the following exceptions:

1.) The Seleucids and Egypt were at war, Egypt gaining a territory on turn 11 and losing a territory on turn 18; the Seleucids gained a territory on turn 18.
2.) Parthia and I (Julii) both gained a territory on turn 2.
3.) The Brutii gained a territory on turn 4 and another on turn 14.
4.) Pontus gained a territory on turn 14 and another on turn 16.
5.) The Greek Cities were at war with Macedon and Pontus.
6.) The Seleucids were at war with Egypt and Armenia.
7.) Gaul and I were at war.
8.) Brittainia and Germania were at war.
9.) Scythia and Parthia were at war.

Now, that's not exactly inert inactivity, and there were a considerable number of load games during those first twenty moves. I think the fact that I can upload each and every save game to whomever wants to look at them, as well as numerous variations I tried throughout the course of the campaign, is proof of that.

So far, I managed to find only one save game where I was under siege, in Thapsus, later in the game. There was another Numidian army approaching Carthage, obviously intent on laying siege, at the same time. As usual, I sallied when I was actually playing the game. However, if I load the save game and simply hit "End Turn" without sallying forth, the Numidians do indeed abandon the siege -- to block the bridge between Thapsus and Carthage, the only way I can get reinforcements there, while the other army layed siege on Carthage. Note that the second army did not forget that it's plans were to lay siege, and that the first army did move to a logical strategic point! (Except for the fact that I had a trireme in dock at Thapsus.) However, here's the bummer:

If you do a save/load at that point and hit "End Turn", Numidia also abandons the siege of Carthage and both armies walk away. If you just hit "End Turn" without the save/load, then they maintain the siege. So there's apparently some validity to this observation and it is some type of player activity which is defeating it. I will look at some additional save games and see if I can identify it. Anyone else who wants to look at them is welcome.

Hi all,
I apologize for not being around in the past few days, life got in the way of testing. ~:)

Quote:

---

Originally Posted by therother

Okay, just checking to see if you thought it was okay. I think the Brutii are perhaps the best choice, given their sheltered location, the fact that they are unlocked to begin with, and that they are a Roman faction (it is Rome: Total War after all). Of course, that last bit works against choosing them to some extent as you also have the Senate AI hardcode possibly interfering with things. Perhaps we can do both, and see if there are any significant differences?

---

I don't think the Brutii would be great because of the senate, as you have mentioned, and because they are supposed to be an active faction due to the senate as well. Which therefore means that they are more likely to conquer something when controlled by AI and produce results.

As to the player's activity influencing AI moves, that's a separate variable that the test I originally proposed does not test for. However, it's kept constant in it, and hence does not influence its results. The only thing that is not constant, provided that everybody does the test in the exactly same way, is saving/reloading. This very basic test will tell us whether this single variable is doing something to the game, provided everything else is kept constant. Nothing more, nothing less. I won't go in whether the AI responds to player's moves or not right now, as this test is not designed to test for that in the first place. If really nothing happens when the player just stands still, then we'll go from there and redesign the test. But I can say right now that even though I'm highly aggressive in my campaigns, with sufficient reloads everybody else is static no matter what I do. Also, the preliminary result by Bromley are promising (thanks for posting them btw), and hence I think the save/reload test is worth doing in any case. ~:)

As far as my testing goes, I'll do my runs by tomorrow. Feel free to PM me your results if you already did your tests.

Cheers,

MORE TESTING

I decided to use the same, identical test that was proposed on this thread except starting from my save game at turn 19 rather than at turn 1. The Brutii laid siege to Segestica on the second turn and immediately released it after a save/load and "End Year". However, the following turn it laid siege again and I discovered that the city had suffered from attrition just as if the siege had never been relieved -- it could hold out for 3 turns now instead of 4. I followed this with three save/loads immediately followed by End Year during which time the Brutii maintained the siege. In addition, other factions were still declaring war on each other. I guess this kinda blows away the idea the save/load always forces the AI to relieve sieges, or paralyzes the AI into some kind of inactivity. I think it just needs for some kind of opening moves to be initiated before it can follow up on them a little more decisively. Again, I have the save game if anyone wants to see it. I've done it more than once and the Brutii always besiege Segestica.

OK, I'm coming to the conclusion that CA is right -- as I would expect them to be, since their information comes straight from the developers. "Load game" does indeed seem to cause random events and AI behavior to change, but not in any way that is game-breaking. In the Thapsus/Carthage-Lemonum/Condate Redonum example I'm still forced to ostracize them from my territories -- whether I sally from the siege or attack them in the open terrain. Otherwise I must succumb to repeated attrition and devastation. The change in the behavior of the AI caused by a save/load procedure allows me to select which of those two options I prefer, sallying or open terrain, but it does not otherwise dramatically change the course of the game -- the main problem, as I see it, is that it denies me the privilege of being the defender in a castle assault, which happens to be a lot of fun.

One can hardly come to the conclusion that the AI's behaviour is indicative of losing track of long-term goals in the light of these results. It took a total of five turns for the Brutii to arrive at Segestica by boat, initiate a siege, relieve the siege, then reinstate it, hold it for three turns, and finally occupy the territory.

Furthermore, in the test involving Thapsus/Carthage and Lemonum/Condate Redonum, although the existing sieges were temporarily released, those units that were approaching a settlement to initiate a siege continued toward their objectives. In other words, they did not lose track of those objectives. Apparently in these cases the AI deemed that instigating conflict through a siege or through desecrating the territory were equally acceptable and the load game caused them to choose the opposite one that was in effect at the time the game was loaded. This is something which can be exploited if I feel I'd like to endure the siege for a turn with the likelihood of being able to meet them in the field on the next turn by doing a save/load, but it's definitely not a game-breaker.

Currently, the tests Ive seen are too broad, given the large amount of factions in the game. I propose that someone mod a game to be setup with 2 factions, plus senate. Use an isolated area and put both factions there, i.e. Britain. Give AI large army and yourself a minimal garrison. Now let him siege you and perform the save/load system. Make sure there are no rebels in his/your area for accuracy. Report results.

I feel, given the CA description, that you need to limit the variables much more than just starting a campaign with all factions on board. I feel this would definitely give a much better example.

As well as testing the effect of reloading every turn, I suggest we also test at other frequencies of reload, e.g., every 2, 4, 8, 16 turns (I know this is a lot of testing!). I know some people have done this elsewhere and they show evidence that the expansion of the AI is directly proportional to the frequency of reloads - this should produce a very convincing graph (number of provinces changing hands by rate of reloads).

I can imagine that CA may respond to a test of reloading every turn as being unrealistic (although I for one often only play one turn at a sitting). We can counter this by testing at other frequencies of reload.

Also I suggest we use the faction rankings graph (rather than removing FOW) to assess the impact on the AI. To simply things we could use the Top 5 ranking on e.g., Territory - that is, make a note of the number of territories each of the top 5 factions has. The beauty of the graph is it shows historical data too, so at the end of the test you can see how the AI expanded for each turn.

I think it is a confirmed fact that loading a saved game causes RTW to regenerate (or recalculate) random events and often changes the behavior of the AI, especially regarding sieges, and that it is supported by empirical and repeatable data.

My own tests were identical to the one that was proposed here, except that I started at various midgame points wherein alliances, declarations of war, and faction targets and strategies were clearly established to see what kind of impact this actually had on the game. I would have reverted to an earlier patch if it had turned out to be significant. However, I'm pretty convinced that this is not the case.

The problem with testing the phenomenon when no moves have been made by the player at the opening of the game is that it is testing an extrema which generally only happens once in a game -- at the first move. That is the point in the game where no alliances have been established, no warfare, no movement, and no building. Many computer algorithms exhibit awkward behavior with various extrema -- for instance, the quicksort (which is the fastest sorting algorithm on the average for most data types) exhibits its worst-case (slowest) behavior with data which is already sorted. That's not really a concern when the programmer expects the program to be working with unsorted data. Extrema can be very useful in determining actual bugs in a program such as integer overflow or divide-by-zero errors. But what's being analyzed here, the AI, is a complex network of heurisms and algorithms. The truth be told, the AI is probably playing a better game than the human player during this test, because it's undoubtedly building an economic infrastructure for each faction and moving its diplomats. It might be possible to force some military stagnation at the most extreme moment in the game through save/loads and nonactivity, but under more realistic conditions I found that it was not as easy to accomplish. Even in the first 20 moves, with multiple save/loads per turn I was unable to recreate that kind of stagnation. In midgame the proposition is even more difficult.

Knowledge of the fact that this reassessment logic is tied to the load game feature is surely something that I can exploit if I wish, but I am summarily unconcerned with any other ramifications of gameplay.

Finally, I'm not sure that it's really such a good idea to have the objective of just a few large AI factions in endgame, as was always the case with MTW. In the early stages of MTW, the AI was geared to force the player into situations modeled after historical realities. The historical reality is that Rome conquered each of these rivals individually, and ultimately become entangled in civil war.

roguebolo, just to clearify, you said you followed the research design developed in this thread. Did you compare a experimental condition (with save/load) to a control condition (the same without save/load)? If yes, did you find differences? I didn't get that clear from what you posted.

Quote:

---

Originally Posted by hrvojej

I don't think the Brutii would be great because of the senate, as you have mentioned, and because they are supposed to be an active faction due to the senate as well. Which therefore means that they are more likely to conquer something when controlled by AI and produce results.

---

I can't see how that should matter. If the effect is there for the Brutii, it exists and no alibi can lead around that.

Quote:

---

Originally Posted by Colt45

I feel, given the CA description, that you need to limit the variables much more than just starting a campaign with all factions on board. I feel this would definitely give a much better example.

---

Why? All other variables are controlled. It doesn't matter how many factions there are, only whether save/load influences their behaviour. In fact, we want to test a vanilla version of the game. A mod with only two factions wouldn't be representative for the normal game.

I did not actually test twenty turns without the save/loads from Turn 1 because I have faith that I will get the same results that have been posted by others; the reason why is that I have a glimmering of the kinds of algorithms that were used and why and how the load game behavior is resetting the AI's behavior. It essentially feels free to select a new opening move. The available options are probably somewhat similar to a software random-number generator, which does not truly generate a random sequence of numbers but eventually will cycle through the sequence and start at the beginning again. While the test results are interesting, they seem to be specific to a given set of extrema.

The control, or the factor that was eliminated in my tests, was player inactivity. A review of the first 20 moves of my campaign -- each and every one of which was saved, because as I said I do multiple saves and loads each turn -- shows clearly that the AI factions are also seeking to expand. I listed the exact details of their expansion and other interactions with each other and with the human player (except alliances) in a previous post.

To test further, I introduced player inactivity at midgame save points, wherein alliances, declarations of war, sieges and other policies had already been initiated, and tried hitting "End Year" repeatedly both with and without save/loads. I posted those results as well, but I will summarize here. Although the AI did have a general tendency to relieve sieges after a save/load, I found cases both where it did so and where it did not; and in those cases where it did relieve a siege, it did not abandon its objective but merely changed the way in which it manifest itself as the aggressor. Units in route to a siege consistently continued toward their target, even with save/loads; units that relieved a siege because of a load game consistently continued to threaten the territory and sometimes blocked access to other besieging units for reinforcements; in some cases they abandoned their siege to add support to another siege in which they were undermanned. I found no cases in which they yielded an inferior strategic position as the result of relieving a siege -- and quite honestly, they were more annoying when they were relieving and reinstating the sieges, almost as if they were taunting me!

I have some really interesting save games that will allow you to observe the phenomenon yourself. It's actually kinda funny, watching the British in the north and the Numidians in the south alternately engaging and disengaging sieges each turn, which gets triggered by the save/load, but it also becomes pretty apparent that they are no less of a threat to your Empire than if they maintained the sieges as they do when you hit "End Year" without a save/load. That's when I realized that if I were willing to endure a siege for just one turn I could use a save/load to exploit this AI behavior to meet them in the field rather than sallying forth from the castle. However, that's probably not a very useful exploit since they will probably end up with the terrain advantage and I will lose the castle which can protect my flanks as I approach them. My guess is that this is one of the factors in the AI's assessment that relieving the siege is equally acceptable to maintiaining it; they can choose which square (or which realtime map) they wish to defend, giving them the advantage of position if I decide to attack them. That's always been an advantage of the AI, that it has a database of which realtime battle map will be used for any given square of the campaign map.

The reason I chose Turn 19 for my other test was because it was still early enough in the game that there had been some interaction between the different factions, but not extensively. I wanted to find out precisely how close to a position of extrema you needed to be in order to observe more stagnant behavior on the part of the AI as the result of player inactivity and load games. I expected results fairly similar to starting the test from Turn 1, but perhaps not quite as pronounced. However, my testing was rudely interrupted after just a few turns when the Brutii laid siege to the rebel town of Segistica. After the first save/load they relieved the siege, then after another save/load they reinstated it and after subsequent save/loads they maintained the seige and occupied the territory. They do that consistently, so in the reassessment logic there must be no "equally viable" alternative to maintaining the siege. The one interesting thing that I noticed is that when the AI relieved the siege after the first save/load, then reinstated it, it had no effect on the attrition of the settlement! The settlement could only hold out during the siege for 3 turns rather than 4, just as if the siege had never been relieved!

For players that play only on turn per sitting, I suppose this behavior could have a small advantage in that sieges on their settlements will be audibly announced every other turn.

Anyway, as you've probably guessed by now, I'm a programmer myself. After reading the response from "The Shogun" at http://p223.ezboard.com/fshoguntotal...ID=24377.topic it seemed to me as if he had brought up the topic to his programmers, and as if they fully understand the phenomenon and explained it to him, but as if he did a poor job of explaining how a load game causes the AI to reset the list of equally viable options -- selecting the "seed value", to describe things in terms of a random number generator. He also did not explain the fact that although this behavior can produce some awkward gameplay at the moment of greatest extrema (which happens to be the first turn of the game in the presence of player inactivity), those side-effects diminish as the game moves forward -- and that it can, in fact, enhance the gameplay experience because the computer cannot be relied upon to make the same move after a load game as it did when you simply pressed "End Turn", thereby making it less predictable.

My guess is that the developers showed him some concrete examples, like the ones in the save games from my campaign that I've experimented with. That's why I've offered to send them to anyone who wishes to review them. They were not specifically constructed to create a point; they are just save points from my campaign that I thought would be good points to test the effects of this phenomen during midgame -- specifically, those points at which sieges were in effect. Combined with a little explanation, they are sufficient evidence of the viability of strategic reassessment after a load game. Unfortunately, I think something was lost in the translation from developer to customer relations to end user. Have you ever played the game "Secret"?

Interesting... Anybody about to duplicate it?

So if I understand you correctly, you only differed by actually going out and attacking and basically playing the game for a while (while saving/loading every turn) then you sat back and watched what else would happen if you became stagnant (still saving/loading every turn)?

Right. Both with and without save/loads every turn during the period of stagnancy.

So, Roguebolo, what would be the point of the extra re-assessment after the save/load as opposed to just letting the game go on as if there had been no save/load? Especially as the no load/save seems to bring fine results as is.

I guess we'll need someone to recreate the Sicily run of turns, but start after one player move...or do we need to play more than that Roguebolo?

If the person who did that test could simply start on Turn 20 and make one move as his faction, and then continue the test, we'd know for sure.

Also, with the Total Protectorate feature, how can a player know if the AI protectorates were ill-gotten or not? In fact, I had several protectorates that seemed too easy. Is this part of the "re-assessment" function to accept all protectorate requests after a seige/lift save/load?

I labeled each of my save games before "End Turn" as "1st Turn", "2nd Turn", etc...all the way to the 20th turn. I was quite surprised when I looked back and saw this because I had no idea how useful it would become a few weeks later for these tests.

I had just reinstalled RTW and installed Patch 1.2 for the first time. (I had been playing a different game for a couple of months.) I played as the plain-vanilla Julii (again) since all the factions were locked and I decided to unlock them with gameplay rather than by modifying the text file, as before. Assuming that I did not accidently skip a turn, I started the tests just before hitting "End Year" of the 19th turn. At any rate, it was the Winter of 261 B.C.

At the end of the turn in Winter 261 B.C., despite frequent saves and loads, here was the political climate:

1.) The Seleucids and Egypt were at war, Egypt gaining a territory on turn 14 and losing a territory on turn 18; the Seleucids gained a territory on turn 18.
2.) Parthia and I (Julii) both gained a territory on turn 2.
3.) The Brutii gained a territory on turn 4 and another on turn 14.
4.) Pontus gained a territory on turn 14 and another on turn 16.
5.) The Greek Cities were at war with Macedon and Pontus.
6.) Egypt and the Seleucids were at war.
7.) Brittainia and Germania were at war.
8.) Scythia and Parthia were at war.

Alliances:

1.) Scythia was allied with Armenia and the Seleucids.
2.) Carthage and Spain were allies.
3.) Parthia and Thrace were allies.
4.) As the Julii, I was allied with Carthage, Macedonia, Gaul, Germania, Dacia, the Greek Cities, Numidia, Scythia, Spain and Thrace. I believe I initiated all of those alliances with the possible exception of Gaul.

On the following turn, I began an invasion of Gaul and we declared war, which is why I chose this turn to start testing stagnation; up to this point my only military conflict had been with Segesta and random rebel stacks.

I think I will go back a little further toward the point of greatest extrema -- say, turn 12 since most territorial acquisitions began around turn 14 -- and see if the byproduct of a stagnant AI is more pronounced. At that point only Parthia, the Brutii, and myself had gained a territory.

Did Parthia gain the rebel territory?

I think Julii and Bruti do that as well.

Could it be AI versus rebels as opposed to other AI is different?

Note Pontus and Egypt could also be taking Rebel only territory.

To be quite honest with you, I had never succeeded in getting a faction to submit as protectorate until this most recent campaign. I had beaten Gaul back to just it's capital in Celtiberia, and laid siege to it, and they patently refused Protectorate status. I read about the save/load exploit on a forum, and I was absolutely determined to have them become a protectorate even if I had to use an exploit. So I tried it, and it had absolutely no effect. They only finally caved in after the siege had been held for several more years -- for a price, of course.

I ran into the same situation with Spain a few turns later at their capital in Lusitania. They had no other territories and their capital was under siege. Again, the save/load had no effect. However, after a few turns of being besieged $14000 convinced them.

I should have save games of both of these situations as well.

Quote:

---

4.) As the Julii, I was allied with Carthage, Macedonia, Gaul, Germania, Dacia, the Greek Cities, Numidia, Scythia, Spain and Thrace. I believe I initiated all of those alliances with the possible exception of Gaul.

---

LOL. That's pretty amazing for 20 turns. If you had asked them all to become protectorates, you'd have sewn the game up. When I play without save/load, I don't think I can achieve so many alliances.

Well, I guess that's just proof that I do save/loads every turn, in support of the other data I've provided.

I always send out my diplomat by boat from the first turn of the game. He moves from faction to faction trying to sell trade rights, usually unsuccessfully after which he gives them away. Then he tries to sell an alliance, which he also gives away if unsuccessful. As the Romans, it's easier to get alliances because it inhibits other Roman factions from warring with the proposed ally and often causes a ceasefire if hostilities are already underway. The AI is aware of this and likes that insurance policy. The wealthy Greeks are particularly suckers for this technique. If trade rights have already been established, an alliance seems easier to sell or give away. The last thing he tries to sell is map information.

I also build a diplomat in my territories so that I can approach foriegn diplomats traveling nearby. Sometimes I reject their diplomatic offers and come back to them with the same offer on the following turn, so that my diplomat will get the influence bonus.

I will check which territories were taken over, and I'm also going to try the same test from Turn 12.

EDIT: Apologies, but I have missed out Rhodes (Greeks) in all of my tests. It's immaterial to the conclusions though.

As far as I know, only DimeBagHo has tried to quantify the effect on the AI of different intervals of save/load. I've just finished a series of tests to provide some similar evidence. Unlike Dime, I just ran one test for each interval, so I don't have averages. That weakness is offset by running the test for all intervals between every 2 turns to every 10 turns (inclusive).

Just to be clear, when I say an interval of, say, 4 turns I mean that I click End Turn 4 times and then save/load.

As intervals of 4-6 turns seemed to be key, I did perform a second run for each of those, so I have basic averages there. I also ran a no-load game. The first column in the results, labeled "Start" is the starting position in 270 BC. I started all the runs from this same saved position.

So:
RTW 1.2, no mods, M/M, Brutii, 20 turns, various intervals, everything moved to cities at start, no other player action, answer "no" to any diplo/marriage.

I recorded the number of territories owned by all of the factions (except rebels). The total of these show the total AI expansion, which is the key result for this test. The data on the individual factions is still there though, so others can draw conclusions from those if they'd like.

I also specifically recorded the state of Sicily.

Conclusion

Observations of Sicily show that the AI has real trouble expanding if loaded every 4 turns. The cusp seems to be 5 turns, where the Scipii in one run took one city yet in the other remained static. With 6 or more turns, the Scipii managed to overrun Sicily every time (although it took them longer at 6 turns than at 10).

Total AI expansion seems to really suffer until you hit intervals of 4 turns. It seems to require an interval of 7 turns to maximise (or get close enough for there to be little difference). Obviously, more runs would generate averages that would better pinpoint this, but I'm happy that the general trend is pretty obvious with what we've already got.

Note how that last paragraph disagrees with the first. i.e. in the first I'm saying that 6 turns will allow the Scipii to conquer Sicily happily and yet in the second I'm saying that intervals of 4 and 7 are the key ones. I believe this has to do with things like the distance between the cities and the level of opposition in the area. So extending the interval beyond 6 makes no difference to the Scipii in Sicily because they only need 6 clear turns to reach and overrun a city. It does help other AI factions reach the cities that are further away though. Likewise, the Scipii are completely boned with an interval of 4, yet other factions manage to do okay. That's because the Scipii are facing walls and armies rather than rebel settlements.

http://img212.exs.cx/img212/248/brut...tstable4tp.jpg
http://img212.exs.cx/img212/9104/bru...tschart1dj.jpg

If I haven't explained something properly, pm me (to keep this thread as uncluttered as possible) and I'll fix it. Likewise, if people want any of my saves pm me and I'll upload them somewhere.

EDIT: Apologies, but I have missed out Rhodes (Greeks) in all of my tests. It's immaterial to the conclusions though.

I did a short extension, just to see what happens when alliances/wars etc are firmly in place.

Loaded the no-loads 260 file. Then (a) played to 250 with no loads and (b) played to 250 with 4 turn loads (i.e. long enough for the AI to recover from the previous load but still expected to impact AI behaviour).

Here are the results with a few notes. Basically, loading cut the Rebels some slack but also prevented the destruction of Gaul and held Egypt back. Also, the Brutii didn't try to establish a foothold in North Africa.

http://img63.exs.cx/img63/1997/bruti...sttable0fw.jpg

Interestingly enough, if you look at the absolute differences for the 250 no-save and 250 interval 4 compared to the 260 load used to start the test, you see some startling results.

The bottom line shows that the no-load has increased in total by 7 and the Int-4 by 3. A big difference, but not huge.

But then look at the territories by country though and compare them to the 260 start.
* Looking at the absolute changes (i.e. ignoring + and -, just the number)
* Removing the number of rebel provinces in that amount (which we know by looking at the total provinces owned before and after)
* Dividing the resulting number by 2 (as all provinces were taken from another faction, so are double counted)

That gives some stunning results. No provinces were taken from other factions in the interval 4 run, whereas 8 were taken in the no-load run.

That's huge. I know there might be a few logic errors in what I just did (as I haven't done maths for 15 years ~:) ), but not enough to explain away a variance that large.

Where it says "20", that's meant to read "start" or "260". Can't be bothered to fix the image ~:) .
http://img63.exs.cx/img63/5264/bruti...bleprovinc.jpg

[edit] Sorry, I misunderstood something. OK, the AI factions have all made alliances but the human player has not made any alliances nor expanded yet after 20 turns, right? That introduces an entirely different extrema that is a little hard to concieve of, since the AI definitely attempts to respond to the human player.

I'm finding that alliances are actually a key factor. If I load my game at turn 14, then play forward to turn 19 with or without save/loads, I get results that are identical to when I was playing the actual game. The same factions have acquired the same territories, except that when I did not use save/loads Egypt apparently beat the Seleucids in their war because on turn 18 the Seleucids did not gain a territory and the Egyptians did not lose one; so the game with save/loads was actually less static, but that's merely a matter of an autocalculated battle turning out differently.

HOWEVER, there IS a breaking point. If I move back to turn 12 and play the game forward to turn 19 with save/loads, none of the factions take their rebel territories. Whether I use save/loads or not, the alliances and declarations of war are considerably different than when I had established alliances on my own, as well as considerably different than each other. The save/loads seem to be affecting who they select as allies and who they select as enemies, much the same way that random events are affected by a load game.

I know exactly why different declarations of war are occuring. At the beginning of turn 12, my only alliances were with Gaul, Germania, the Greek Cities, Numidia and Spain. At the beginning of turn 14, most of my alliances had been established and included Carthage, Germania, and Thrace as well, thereby preventing many of the other factions from declarations of war in order to keep their alliance with my Roman faction. Without these alliances, which is the only thing I did while actually playing the game except for building my faction's infrastructure, the other factions were free to make war or love as they pleased, without any regard to my faction's alliances. This strongly reminds me of MTW.

When starting from turn 12, without my alliances to guide their decisions, the alliances and declarations of war that they chose were considerably different depending on whether I used save/loads each turn or not. Without save loads, several of the factions were less active (but not inactive) in taking rebel territories, and several other factions were more active. However, WITH save/loads, no faction took any rebel territories. If this had something to do with the different nature of the alliances and war declarations, because they thought they needed the forces elsewhere, or because of other reasons, I cannot say. In any case, it does seem to support your idea that the rebels get a break, and it also seems to be growing in prevalence as I move closer and closer to the point of greatest extrema. But let me point out that it also seems to grow less in prevelance as I move away from it, so that by midgame it seems to have no effect on gameplay at all.

I don't know if this is coincidence or if it has something to do with the anomalies you've pointed out, but turn 12 also happens to be an even multiple of 4 and 6.

HarunTaiwan:

The point of reassessment after a load game is that it makes the AI less predictable. You can't say, "Oh, I didn't know it was going to do that. I'll just load the last turn and play a little differently."

Thank you Bromley.

Your second analysis answers the question that occurred to me on reading your initial results - the number of territories that changes hands is a much better measure of the effectiveness of the AI compared to the number of territories each faction owns. I assume as players we generally prefer an AI that is aggressive, expansive and therefore more likely to attack (if not conquer) our territories.

The community owns you a big thanks for the time you've put into this. ~:grouphug:

I hope this data will give CA some insight into the game mechanics over time, and will help them in their plans for the future of the TW series, if not RTW itself.

In the test I did with a Julii campaign, the war declaration didn't change between playing continuously and reloading.

Julii imperial campaign, M/M difficulty, FOW false:

Start, 270 BC. Position the map so that you can see both Syracuse and Athens. Hit end turn.

Turn 1, 270 BC. Scipii sieges Syracuse which can hold 7 turns. This initial siege of Syracuse might be scripted. Hit end turn.

Turn 2, 270 BC. Decline the trade offer from Gaul. Syracuse can hold for 6 turns. Hit end turn.

Turn 3, 269 BC. Accept the marriage offer. Syracuse can hold for 5 turns. Hit end turn.

Turn 4, 269 BC. Syracuse can hold for 7 turns. (Perhaps an unsuccessful assault was made and the siege has been reinstated.) Macedonia sieges rebel owned Athens which can hold for 6 turns. Macedonia is at war with the rebels and no one else. Save the game. Hit end turn.

Turn 5, 268 BC. Syracuse can hold for 6 turns. Athens can hold for 5 turns. Hit end turn.

Turn 6, 268 BC. Syracuse can hold for 5 turns. Athens falls to Macedonia. Exit the game.

Load the savegame made on turn 4. The random number generator could introduce a difference at this point because after a reload the AI may not perform the exact same sequence of decisions requiring a random number. It's possible that a random number is used to weight the decision to break a siege, but I find this test to be very repeatable. Hit end turn.

Turn 5, 268 BC. Scipii lifts the siege on Syracuse and moves away. Macedonia lifts the siege on Athens and moves away. Macedonia is still at war with the rebles, and the Macedonian army which was sieging Athens moved north to an area where there are no rebel armies. There is in fact no observable threat to Macedonia anywhere on the map, and they are not at war with any other faction. Hit end turn.

Turn 6, 268 BC. Scipii sieges Syracuse which can hold for 7 turns. Macedonia sieges Athens which can hold for 6 turns with the same army as before. This is a futher indication that there are no other threats to Macedonia because the AI returned to the same strategy is was using before the save on turn 4.

End of test.

Conclusion:

Clearly, loading the game caused both sieges to be lifted because neither siege was lifted when play was not interrupted by the save/load cycle. The expansion by Scipii and Macedonian is set back by the lifting of these sieges. It's apparent that the AI reformulated it's siege strategy, and it took more than one turn to get the sieges back in place. It took the first turn (turn 5) to reassess the situation just as CA claims, and the second turn (turn 6) to reinstated the sieges. This reassessment was necessary only because the AI was not fully aware of the situation after the reload. The course of the campaign is altered by saving and loading because Macdonia should already be in possession of Athens on turn 5, and Scipii should be further along in it's conquest of Syracuse. This has a far reaching effect because a faction is set back economically for the rest of the campaign, it affects all AI factions and happens repeatedly as often as you reload.

It's true than not all sieges are necessarily lifted after a reload, but from observation with FOW off I would estimate that 90% are lifted. The AI may be moving these formerly sieging armies away from cities because it's programmed not to position them that close to cities that it isn't sieging.


Bromely's data is very nicely done, and it doesn't surprise me at all that it shows 4 continuous turns to be the level which allows expansion since the max siege length is typically 7 turns. I'll bet the average siege length is about 4 turns. I would say to get on a level footing with the AI you have to not only play at least 4 turns per sitting but also break off your own sieges and perform no diplomacy on the turn after a reload.

OK, after confirming that there was a breaking point at Turn 12 in my game, I began to wonder a few things. Why was it that if I did save/loads and "End Turn" consecutively after turn 12 that other factions would not take rebel provinces, when they seemed to have no qualms about doing it when I started from Turn 14 or Turn 19? The only thing I did other than building my internal infrastructure between turns 12 and 19 was establish alliances. But how could that have such a drastic effect on the AI?

So I went back to the original test scenario, starting at Turn 1. This time, I looked at the diplomacy panel each turn, both with save/loads and without them, to determine which factions were allies and which had declared war and -- more importantly, because it's not explicitely listed in that panel -- which ones had established a ceasefire from the previous turn.

As anyone who has played MTW is familiar, the course of the game was largely decided by the alliances that you established. The same is true of RTW.

It appears that in the reassessment process, the AI does not merely reassess its moves -- it reassesses who would make the best allies and the best enemies before making any other decisions. When "load game" is NOT used, it sticks with its original choices.

When a "load game" causes it to reassess the political landscape, and it sees multiple choices that are equally viable, it will often establish a ceasefire with hostile countries and select a new target -- I suppose all on the grounds of remaining "unpredictable". (It also has the byproduct that the human player can exploit load games to change the political landscape, allowing him to establish alliances that were previously not possible.)

The end result is that repeated "load games" on consecutive turns throws it into a floundering, confused state, with so many allies and enemies to select from, of switching sides over and again. You can confirm this by watching as ceasefires go into effect when sieges are released, and as they begin to ally with factions that they were besieging a few turns ago.

To further complicate things, the AI apparently percieves the Rebels as "just another faction" and sees the same benefits of nonhostile relations with them as with other factions. So the rebel factions make out like bandits (no pun intended) in this situation.

However, as I've pointed out, there seems to be a stabilizing factor that occurs as you move further and further away from this point of extrema, which is partially provided by the actions of the human player. As the human player begins to interact with the other factions, establishing declarations of war and ceasefires and alliances, and as other factions do so as well, each faction starts getting "locked in" to a situation in which it is less and less desirable to cease hostilities or establish or break alliances. This is the stabilizing factor that made it so much more difficult to recreate the phenomenon in midgame.

It does have some kind of an effect on the beginning of the game; primarily that if you load and save a lot, it will take longer for rebel territories to be occupied. The fact that the Scipii have two extra territories and the Greek Cities and Carthage each have one less territory before this stabilization process begins, assuming that you did not interfere with that course of events by using load games or by establishing alliances which prevent it, has very little effect on the progress of the game as a whole; in fact, as the Julii I'm more likely to be at war with Carthage or the Greeks before the Scipii, so it makes the game a little LESS challenging without the save/loads. Note that load games are not the only thing that will prevent the Scipii from conquering Sicily -- you can intentionally prevent it by manipulative use of alliances.

But the fact that save/loads causes rebel territories to be occupied later is more of a problem, because it sets back the development of those territories by several turns.

In my real game, the situation was becoming stabilized by turn 14; but as you can see, I was making a considerable number of alliances. It may take longer for players who wait for the AI to establish the political landscape.

Now that I understand the situation a little better, I actually do have a real gripe about the fact that rebels are being treated the same as other factions. Although it may not ultimately have a major impact on midgame or endgame, which is really the part that I'm looking forward to, it does have a minor impact. And furthermore, it just plain doesn't make sense.

I'm not exactly sure why you guys are studying this issue if CA has stated they do not intend to release another patch because of their two-patch limit. I think in the long run bugs and glitches and other phenomenon like this one will make themselves evident; what really needs to be addressed is the two-patch limit. Speaking as a programmer, I can honestly state that this is a ridiculous, arbitrary policy and needs to be changed. So go for it. I'm going back to my game. If there's any way I can help with your efforts, let me know.

And, btw, congratulations on whoever first discovered this phenomenon. Good work!

Wow... I'm impressed. You guys have done a gerat job.
It seems the player will have to play a good number of turns before doing a load, so that the political scene can work towards active AI factions. But even in that case many people are forced to reload pretty much every turn for a wide range of reasons. For them it is still impossible to get the AI to pose a threat, at least a fun one.

We know they read this forum, so this is more of a plea.

Thanks for the hug ShellShock ~:) .

Another extension of my test. I loaded up the 260 save where I'd not save/loaded during play. That gave me some nicely expanded AI factions, so they could be expected to turn most of their efforts towards other factions rather than Rebels.

I then played 20 turns with a 2 turn save/load interval. But this time, I was active and really played them. If nothing else, this proved to me that the Medium portion of the strategic game is ridiculously easy ~;) . Greece and Macedon both felt my wrath.

Check out the results. Although there's obviously a slight problem with my method of determining how many provinces were taken from other faction (as I know I took 6 yet the function spews out 5), the variance is pretty immaterial.

What isn't immaterial is that I was responsible for ALL of the faction on faction province swaps. So I think that it's fair to assume that, for the purposes of AI expansion, player activity is not a determining factor.

I'm not going to perform an interval-2 test with no activity by myself because I think it's pretty obvious from earlier tests that the AI won't expand. I've included the 250 no-load data where I was passive for comparison - you can see that the AI might be expected to have nabbed 8 provinces off other AIs in those 20 turns without the save/load feature (on top of an extra 5 Rebel provinces)..

http://img119.exs.cx/img119/9728/bru...bleactivep.jpg

Quote:

---

But even in that case many people are forced to reload pretty much every turn for a wide range of reasons. For them it is still impossible to get the AI to pose a threat, at least a fun one.

---

Not really. It just takes longer for the game to reach a more stable situation. The political climate will be DIFFERENT, not less challenging; the factions you are at war with may be different and they may be stronger or weaker, but eventually you need to face the majority of them and their wishy-washy alliance changes become less likely. However, the same logic that allows it to change alliances if they look equally viable could cause a long-time ally to suddenly break their alliance with you and form one with your primary rival. Or vice versa. Especially right after a load game, or any other event that triggers a reassessment.

The territories that started out as rebels, however, will always be underdeveloped if you were using a lot of load games in the earlier moves of the game. That's why my main gripe is regarding the fact that the AI is treating rebel factions as "just another faction." Eventually they will conquer the nearby rebel territories...just a little too late, and too slowly.

In my game I was saving and loading every turn, and the political climate *started* stabilizing by turn 14 -- but as someone mentioned, I formed an awful lot of alliances. It would take quite a bit longer if the human player is not as proactive about forming alliances. I like forming alliances early because it gives me more control over the political climate.

Something Puzz3D said got me thinking:

Quote:

---

Turn 1, 270 BC. Scipii sieges Syracuse which can hold 7 turns. This initial siege of Syracuse might be scripted. Hit end turn.

---

Does anyone know if the first AI moves are scripted? If they are, do the scripted decisions get saved and how do they fit into the reassessment? If they are scripted but not saved, this could be causing problems at the start of campaign games.

Nice work, Bromley! ~:cheers:

Quote:

---

Especially right after a load game, or any other event that triggers a reassessment.

---

I feel funny quoting myself, but thinking about it I began to wonder...ARE there any other known events that can trigger a reassessment? Has CA mentioned any, or has anyone else observed any? Or is it just the load game?

Quote:

---

I agree that it is the rather strange two patch policy that needs to be changed, but since it has been settled already (actually before the game was finished at all)

---

That's what I meant when I called the two-patch policy arbitrary. It's not a very realistic approach to software development. It's entirely arbitrary. I'd like to share a few anecdotes with you of similar situations but its too off-topic.

So, roguebolo, should I save load every turn, for even BETTER AI?

Notice Bromley confirms that's it's all rebel territory many without walls that you are seeing being taken, and not AI territory.

I doubt alliance does much at all, since the AI breaks those at will.

I suspect the reasssessment if it exists leans toward peace and friendship, which could explain the protectorate issue a bit. Maybe it's designed so the human player who is coming back has some breathing space. LOL

Also, Roguebolo, the last thing you want in a wargame is stability. You need to feel threatened as well as threatening. In MTW, you definitely felt that. In RTW, only if I play a small nation am I ever worried.

Nah, I rather think that the AI goes "Hey what is going on here... Ok I better get in a position where I can formulate a good strategy (hence sieges broken off) and I need to secure myself (hence acceptance of Protectorates)."
Basically the AI goes into 'start'-mode. It starts over again as if the game starts at that date, which seems to be a fitting explaination as the info to make the AI realize it was actually in a running game could possibly be rather substantial, and an obvious place to cut if size was demanded smaller.

I agree with you. I think the results show if you play 6-7 turns it's not so bad...so I will play less now since sometimes I only have 20-30 minutes...shouldn't have had that kid, should I?

Quote:

---

Originally Posted by Kraxis

Basically the AI goes into 'start'-mode. It starts over again as if the game starts at that date, which seems to be a fitting explaination as the info to make the AI realize it was actually in a running game could possibly be rather substantial, and an obvious place to cut if size was demanded smaller.

---

I think that start-mode is a good word to describe what's happening. The AI has to rebuild some memory resident data structures.

Quote:

---

Notice Bromley confirms that's it's all rebel territory many without walls that you are seeing being taken, and not AI territory.

---

Right. I've done the same tests. I've watched what happens in Sicily without load games. I've also watched what happens in midgame with them. Even then the AI will relieve sieges, but the net effect is different. It continues to threaten the same territories. Like I said, don't take my word for it. I'll send you a number of different save game positions that show it to you, from the same game even.

The truth is that the AI has never caught me with my pants down so that waiting out a siege would ever work anyway...so it boils down to a matter of, "Should I sally or meet them in the field?"

The fact of the matter is that in the first 20 moves, without load games, it's primarily rebel territories that fall.

Quote:

---

I doubt alliance does much at all, since the AI breaks those at will.

---

Here, I agree with The Shogun in that there is (in his words) "a misunderstanding of the AI." In midgame, the AI definitely does not easily break or form alliances. There's a very clear logic on what it considers acceptable. However, in the early phases of the game it will change them willy-nilly whenever there's a reload. It doesn't seem to merely reassess the situation and roll some dice for 50% odds -- it seems to intentionally reverse its previous decisions.

Play through the test scenario one more time with load games, but this time watch what happens in the diplomacy Pane with Egypt/Seleucids/Armenia or the factions in Sicily. Make note of ceasefires as well. You'll soon see a pattern of ... they removed the siege ... a ceasefire is in effect ... now they've formed an alliance ... etc. I should have recorded that information and posted it because it gives some good insight as to what is going on.

At any rate, if you doubt that alliances have an effect on the AIs willingness to follow through with sieges, I can illustrate with specific save games in which that is the one and only determining factor that modifies the behavior of the AI, despite load games. Isn't there an upload area for sharing data?

Quote:

---

the last thing you want in a wargame is stability. You need to feel threatened as well as threatening.

---

Now, that's taking everything I've said out of context. Stability is a very generic word which can mean a lot of different things. I think you know perfectly well that I was not referring to a "comfortable, easy-going game where everything is under my control." I was referring to a stable political landscape which causes the AI to behave in a more stable, reasonable manner. You certainly don't want the kind of instability that it exhibits early in the game with numerous save/loads to continue throughout the game.

Sometimes I wonder how many people here are actually playing games beyond the first twenty moves. I can definitely see that there has not been much testing beyond that period of extrema. But to truly understand the phenomenon, you need to come up with instances in which it is NOT exhibited, and determine why. You need both a positive and a negative.

While the statistical data supporting the load game behavior is very good, I have seen very few suggestions that explain this behavior -- nothing that illustrates a truly good understanding of it.

It's a hard proposition to make projections about the overall difficulty of the AI factions in midgame on the grounds of its early behavior resulting from load games. Generalized statements like "it's less aggressive" don't carry much weight in the light of actual savegames which illustrate the AI's midgame behavior to be substantially different. They tend to be semantical statements arguing a foregone conclusion of an empirically observed and repeatable fact, and that's simply not good enough. You not only need to be able to make projections about the future behavior of the game, but you need to back them up by observing and measuring the resultant behavior as well.

A similar semantical argument (and similarly illogical) would be, "It sounds to me like you want one of the AI factions to beat all of the others for you, so that you can take it on after it's overextended itself." When the Scipii, your allies at that stage of the game, grow stronger, Carthage and the Greek Cities grow weaker. But the sum total of the AI factions is the same, and must all eventually be defeated.

I really feel that some serious thought should be given to proven instances in which this phenomenon does not seem to exist or does not seem to exhibit the projected results. I don't think it will be fully understood unless those are understood as well. I have a number of such save games, which is why I've asked several times about sharing them with people. I'd like someone else to take a look at them and see what conclusions they come to. If anyone is interested, please PM me and I can send them by email.

How does that fit with my last test? That seemed to show, to me, that the AI broke off all sieges (note that I did not visually confirm this, it's just that they ceased to expand into each other and only took a few Rebels). That's even with a human playing and after the diplomatic scene had been given 20 turns to mature (although the human was passive in those first 20 turns).

Also, although without save/loads the Rebel territories are a larger part of the total territories taken in the first 20 turns, there were more provinces swapped in my 270-260, no save/load test than taken from Rebels. The proportion of Rebel territories to all territories taken approaches 100% as save/load interval is reduced. I personally believe that this is because the Rebel territories are easier to take (fewer defences, smaller defending armies, no supporting armies), but may be wrong there.

Not only easier to take, possibly hardcoded into diplomacy for the AI to ignore rebels as a negotiating partner, etc. If you can't negotiate with them, lifting a siege won't result in any diplomatic benefit, so you might as well see it through.

Also, Roguebolo, your point noted on "stability" but yes, I do think having AI's expand and compete along with you, to the point where you face factions of similar sizes makes a much better game.

Bromley, yours is some of the statistical data that I said was referring to as very good. I guess the point is that although you've been able to create scenarios which duplicate the phenomenon, I've been able to dig up some scenarios which duplicate explicitly contrary behavior. In some cases they illustrate the very kinds of AI behavior that The Shogun mentioned in his post, but that does not necessarily redeem them. Aren't both worth some study if you wish to understand that behavior?

I actually do visually confirm the results of the first 20 moves with an "End Year" without save/loads, and there are typically four territories under dispute that are not rebel. All of the remaining acquisitions are rebel, and if you ask me that's some pretty sensible playing. It's also easier to debate the stance and prove the projection that rebel territories being occupied at a later point in the game will be less developed than if they were occupied earlier. I hope no one disagrees on that point.

It's much more difficult to debate the idea that a vague concept like "the AI is less aggressive to other factions" has any real impact on the gameplay, particularly in midgame -- then you'll need to face the entire slew of possibilities that The Shogun mentioned in his post and show that each and every one was not a reasonable assumption.

I honestly do consider the inactivity on the part of the human player to be a weakness of these tests. I'm not going to debate the reasons why but eliminating that lack of inactivity is what yielded the contrary results that I consider worth studying. In one case, even something as simple as forming three alliances in the space of two turns completely changed the behavior of the AI and made it more aggressive.

I don't really agree that a single large faction makes a better opponent; I would rather see smaller factions allying against me as there is more diversity in the gameplay because of the different unit types they can create. The only case in which I would like to see other factions grow at an equal pace is when I'm playing a Roman faction and eventually have to face the other Roman factions after civil war begins. When I first played I thought this is the way things would turn out, and I was surprised to find my Urban Cohorts facing large bands of Hastati and Velites.

This is a little off-topic, but the thing that reduces the challenge the most for me is that the AI does a poor job of building the more powerful units and a balanced army, and that it does a poor job of upgrading the arms and armor for its units. (There was actually a minor improvement in the latest patch, but not enough to be significant.) If it did a better job at these issues, it would make a considerably superior opponent.

Apparently my posts on this forum have become somewhat controversial, but I guess I'm getting used to that. Recently on a different forum regarding a different game some of my posts were originally considered controversial, eventually got me flamed by a few individuals, and ultimately won me an award as their best strategy columnist. Bromley has asked me to post a summary of my position because apparently some people have been referring to my posts as evidence that there is no loadgame bug.

In some respects, they are correct. The problem here is the word "bug". To a programmer, a bug is a clear and undetected error that produces incorrect or unintended results or an interruptive abberation in the behavior of a program, such as divide-by-zero errors or integer overflows or a deadlock due to improper use of synchronization objects. I have encountered, tracked down, and eliminated many such bugs over the last 20+ years, and this does not truly qualify as a "bug." I assure you that if you were to refer to it in those terms to the developer(s) responsible for the AI that they would take at least a small amount of offense to it. In fact, as a fellow programmer, I take a small amount of offense to it myself on their behalf. I do wish everyone would adopt a more accurate description such as the "load game reassessment behavior" or "load game reassessment anomaly".

What we are dealing with here is a peculiar behavior of the reassessment strategy under conditions of polar extrema -- beginning from the opening move of the game, without player interaction except for the refusal of all alliances which serves to reinforce the extrema. I'm not undermining this test, because it quite possibly gives us some insight into the viability of the the algorithms and heurisms used by the AI throughout the course of the game, including midgame and endgame, in the absence of such extrema.

There is a clear tendency for the reassessment that occurs after a load game to often (not always) reverse diplomatic decisions and to relieve sieges. It's also clear that this behavior tends to diminish or manifest itself in a different way as the game moves away from a position of extrema. Unfortunately, I'm one of the few people who has apparently studied the phenomenon with that possibility in mind, although Bromley is moving in the same direction. It helps to have a large database of saved games to draw from.

I think there are some weaknesses with the research that's been undertaken here. It seems that many people have reached "obvious", foregone conclusions from their observations of this fairly limited test, projecting those "absolutes" to the conclusion of the game even when the extrema is removed. There is no support for such conclusions except inasmuch as they can be supported semantically. There is virtually no empirical evidence as to the resulting strength of the AI opposition as a whole due to the AI's reassessment behavior.

There are also a few conclusions that are entirely accurate, although unsupported empirically, such as the fact that the load game behavior will allow the human player more time to acquire rebel territories and more time to develop them than the AI factions, which definitely gives the human player an advantage. I think this could easily be supported empirically, although the mathematical analysis would be quite challenging.

There has been very little attempt to understand the reasoning behind the AI's behavior, which I consider to be a key factor in suggesting how it might be changed. To simply say, "The AI needs to stop relieving sieges" could very well introduce another undesirable behavior in the AI. In order to understand the decision-making process of the AI, and how it might be improved, it is important to analyze those scenarios where the AI is behaving in a different manner than that which was observed under conditions of extrema.

In other words, I think the research here is a good start, but incomplete. It is not conclusive because it does not exhibit empirical evidence of the projected conclusions.

RB, you've put forth a fairly solid case and my opinion of the impact of this issue is being revised accordingly. However, my primary concern is that the anomalous behavior at an extrema is at an *unavoidable* extrema. Every campaign must have a beginning, and if the AI takes 20 turns or so to really get its wits about it (more if you play as I do and never form alliances with people you intend to subjugate one day), that's a crippling handicap. Even if the behavior straightens out and the AIs begin to function better as you suggest, for a lot of players it may very well be too little too late for the AI to pose a challenge. Your evidence is *beginning* to convince me that it is WAD, but even so I stand by my assertion that it's a very bad WAD and needs serious attention for the XP.

Pode,

My idea about alliances is not 100% confirmed, but it's true that the human player can force ceasefires between AI factions with his alliances sometimes, which also means it would no longer be a "equally viable" option for the AI to change its mind about certain alliances, bringing about some degree of stabilization. An analysis of alliances and ceasefire agreements in the 20-move test with saveloads seems to support the idea that changing alliances are partially repsonsible for the loadgame behavior, as do the save games from turns 12 and 14 of my campaign in which the AI chose to agressively take territories (even distant rebel territories!) only because I had established three new alliances without any other movement, despite the fact that I was doing save/loads each turn. Furthermore, the exploit that has been reported (I've never tried it myself) which allows you to create alliances with an unwilling faction, or even to establish a protectorate (which I've tried only in midgame, to no avail) also seems to support the idea. I think it merits a little more investigation.

For example, let's take the first 20 moves, with no load games. A good player, who has a good grasp of the game and how to manipulate the other factions, will realize that he can use his starting diplomat to sail to Lilybaeum by the third turn and form trade rights and an alliance with Carthage, and then sail to Thermon in three more turns to form trade rights and an alliance with the Greeks, which causes the Scipii to relieve their siege at Syracuse. (It also establishes some fairly lucrative naval trade.)

Hence, the situation in Sicily is entirely under the control of the human player. If you consider the Scipii takeover of Sicily desirable, you can allow it; or, if you choose, you can prevent it. From that analysis alone I don't see the situation in Sicily as a strong contributing factor to the overall effectiveness of the AI later in the game. The Scipii siege of Syracuse was only as effective as I allowed it to be, so the AI was not any more effective in this case, with save/loads or without them.

However, with save/loads, it is a more difficult proposition to convince the Scipii to attack Syracuse. In this particular example, due to the unpredictability factor caused by save/loads, the AI is somewhat harder to manipulate, just as load games seem to make it easier to manipulate in other ways. Try making these moves yourself if you want -- you don't need to make any changes to the original test except for your diplomat and the boat he came in on.

But you are right about the fact that every single game will begin with a floundering period of uncertain alliances if there are a lot of load games, and that the period will last longer for different players depending on their style of play. One thing is for certain -- the AI won't be picking up a lot of rebel territories, giving the human player more time to invade unclaimed territories (assuming he can spare the resources and manpower) and to develop his own rebel acquisitions. Although I don't consider that to be a massive game imbalance introduced by the load game behavior, it certainly has some effect.

As to whether the AI begins to stabilize, I'm certain that it does as the result of some fairly extensive testing. But nevertheless its behavior is still worth observing because it gives me a clue about how the AI is reassessing the situation on a load game, a topic which continues to fascinate me.

Arphahat:

Chess programs are a great comparison in some respects. If you save between moves, they will reassess their next move each time you load and may not make the same move -- either if it's drawing the information from a database, as in the openings, or if it's in midgame and sees equally viable moves.

It's funny, because those same programmers on the other forum brought up the comparison between the AI in RTW and chess. One of them pointed out that chess is also substantially different because of the fact that it has only 64 squares, and a limited number of pieces with limited movements. So, yes, it's an unfair comparison in that respect.

Quote:

---

A chess program would have the same type of bug if after you loaded a saved game the AI would expose itself to checkmate regardless of the difficulty setting.

---

That is, again, merely an unsupported assessment of the degree of impact of the load game behavior. I have encountered too many midgame scenarios which discredit the viewpoint that it simply gives up the game because of load games.

I still don't see the reason that the AI needs to reassess on reload at all. roguebolo suggested that perhaps this was to add an element of unpredictability, but the easiest way to do that would surely be to use exactly the same code with a different pseudorandom seed, and that can't be what's happening—if it were, the AI might lift a siege or not, but it wouldn't do so disproportionately if a new seed were selected. Unless the decision of which action to pick isn't pseudorandom at all, but I don't see why that would be if you're aiming for unpredictability.

Then again, what do I know? I'm not a programmer. Probably none of what I said makes any sense.

-Simetrical

Edit: Removed semantics discussion.

Roguebolo,

Is there a way to test your hypothesis?

If I am playing Parthia, engaging in alliances and war, you are saying despite load/save every turn that the AI will figure things out and the Scipii will take all of Sicilly?

This seems like an easy test to run.

Please confim if this test would fit your hypothesis that human interaction is needed to jump start the aggressive behavior.

Or is it only working when the human player's faction is involved directly in the situation?