Investigation of AI reassessment upon reload

[Kraxis]

Interesting... Anybody about to duplicate it?

So if I understand you correctly, you only differed by actually going out and attacking and basically playing the game for a while (while saving/loading every turn) then you sat back and watched what else would happen if you became stagnant (still saving/loading every turn)?

[roguebolo]

Right. Both with and without save/loads every turn during the period of stagnancy.

[HarunTaiwan]

So, Roguebolo, what would be the point of the extra re-assessment after the save/load as opposed to just letting the game go on as if there had been no save/load? Especially as the no load/save seems to bring fine results as is.

I guess we'll need someone to recreate the Sicily run of turns, but start after one player move...or do we need to play more than that Roguebolo?

If the person who did that test could simply start on Turn 20 and make one move as his faction, and then continue the test, we'd know for sure.

Also, with the Total Protectorate feature, how can a player know if the AI protectorates were ill-gotten or not? In fact, I had several protectorates that seemed too easy. Is this part of the "re-assessment" function to accept all protectorate requests after a seige/lift save/load?

[Bromley]

EDIT: Apologies, but I have missed out Rhodes (Greeks) in all of my tests. It's immaterial to the conclusions though.

As far as I know, only DimeBagHo has tried to quantify the effect on the AI of different intervals of save/load. I've just finished a series of tests to provide some similar evidence. Unlike Dime, I just ran one test for each interval, so I don't have averages. That weakness is offset by running the test for all intervals between every 2 turns to every 10 turns (inclusive).

Just to be clear, when I say an interval of, say, 4 turns I mean that I click End Turn 4 times and then save/load.

As intervals of 4-6 turns seemed to be key, I did perform a second run for each of those, so I have basic averages there. I also ran a no-load game. The first column in the results, labeled "Start" is the starting position in 270 BC. I started all the runs from this same saved position.

So:
RTW 1.2, no mods, M/M, Brutii, 20 turns, various intervals, everything moved to cities at start, no other player action, answer "no" to any diplo/marriage.

I recorded the number of territories owned by all of the factions (except rebels). The total of these show the total AI expansion, which is the key result for this test. The data on the individual factions is still there though, so others can draw conclusions from those if they'd like.

I also specifically recorded the state of Sicily.

Conclusion

Observations of Sicily show that the AI has real trouble expanding if loaded every 4 turns. The cusp seems to be 5 turns, where the Scipii in one run took one city yet in the other remained static. With 6 or more turns, the Scipii managed to overrun Sicily every time (although it took them longer at 6 turns than at 10).

Total AI expansion seems to really suffer until you hit intervals of 4 turns. It seems to require an interval of 7 turns to maximise (or get close enough for there to be little difference). Obviously, more runs would generate averages that would better pinpoint this, but I'm happy that the general trend is pretty obvious with what we've already got.

Note how that last paragraph disagrees with the first. i.e. in the first I'm saying that 6 turns will allow the Scipii to conquer Sicily happily and yet in the second I'm saying that intervals of 4 and 7 are the key ones. I believe this has to do with things like the distance between the cities and the level of opposition in the area. So extending the interval beyond 6 makes no difference to the Scipii in Sicily because they only need 6 clear turns to reach and overrun a city. It does help other AI factions reach the cities that are further away though. Likewise, the Scipii are completely boned with an interval of 4, yet other factions manage to do okay. That's because the Scipii are facing walls and armies rather than rebel settlements.




If I haven't explained something properly, pm me (to keep this thread as uncluttered as possible) and I'll fix it. Likewise, if people want any of my saves pm me and I'll upload them somewhere.

[Bromley]

EDIT: Apologies, but I have missed out Rhodes (Greeks) in all of my tests. It's immaterial to the conclusions though.

I did a short extension, just to see what happens when alliances/wars etc are firmly in place.

Loaded the no-loads 260 file. Then (a) played to 250 with no loads and (b) played to 250 with 4 turn loads (i.e. long enough for the AI to recover from the previous load but still expected to impact AI behaviour).

Here are the results with a few notes. Basically, loading cut the Rebels some slack but also prevented the destruction of Gaul and held Egypt back. Also, the Brutii didn't try to establish a foothold in North Africa.



Interestingly enough, if you look at the absolute differences for the 250 no-save and 250 interval 4 compared to the 260 load used to start the test, you see some startling results.

The bottom line shows that the no-load has increased in total by 7 and the Int-4 by 3. A big difference, but not huge.

But then look at the territories by country though and compare them to the 260 start.
* Looking at the absolute changes (i.e. ignoring + and -, just the number)
* Removing the number of rebel provinces in that amount (which we know by looking at the total provinces owned before and after)
* Dividing the resulting number by 2 (as all provinces were taken from another faction, so are double counted)

That gives some stunning results. No provinces were taken from other factions in the interval 4 run, whereas 8 were taken in the no-load run.

That's huge. I know there might be a few logic errors in what I just did (as I haven't done maths for 15 years ), but not enough to explain away a variance that large.

Where it says "20", that's meant to read "start" or "260". Can't be bothered to fix the image .

[roguebolo]

[edit] Sorry, I misunderstood something. OK, the AI factions have all made alliances but the human player has not made any alliances nor expanded yet after 20 turns, right? That introduces an entirely different extrema that is a little hard to concieve of, since the AI definitely attempts to respond to the human player.

I'm finding that alliances are actually a key factor. If I load my game at turn 14, then play forward to turn 19 with or without save/loads, I get results that are identical to when I was playing the actual game. The same factions have acquired the same territories, except that when I did not use save/loads Egypt apparently beat the Seleucids in their war because on turn 18 the Seleucids did not gain a territory and the Egyptians did not lose one; so the game with save/loads was actually less static, but that's merely a matter of an autocalculated battle turning out differently.

HOWEVER, there IS a breaking point. If I move back to turn 12 and play the game forward to turn 19 with save/loads, none of the factions take their rebel territories. Whether I use save/loads or not, the alliances and declarations of war are considerably different than when I had established alliances on my own, as well as considerably different than each other. The save/loads seem to be affecting who they select as allies and who they select as enemies, much the same way that random events are affected by a load game.

I know exactly why different declarations of war are occuring. At the beginning of turn 12, my only alliances were with Gaul, Germania, the Greek Cities, Numidia and Spain. At the beginning of turn 14, most of my alliances had been established and included Carthage, Germania, and Thrace as well, thereby preventing many of the other factions from declarations of war in order to keep their alliance with my Roman faction. Without these alliances, which is the only thing I did while actually playing the game except for building my faction's infrastructure, the other factions were free to make war or love as they pleased, without any regard to my faction's alliances. This strongly reminds me of MTW.

When starting from turn 12, without my alliances to guide their decisions, the alliances and declarations of war that they chose were considerably different depending on whether I used save/loads each turn or not. Without save loads, several of the factions were less active (but not inactive) in taking rebel territories, and several other factions were more active. However, WITH save/loads, no faction took any rebel territories. If this had something to do with the different nature of the alliances and war declarations, because they thought they needed the forces elsewhere, or because of other reasons, I cannot say. In any case, it does seem to support your idea that the rebels get a break, and it also seems to be growing in prevalence as I move closer and closer to the point of greatest extrema. But let me point out that it also seems to grow less in prevelance as I move away from it, so that by midgame it seems to have no effect on gameplay at all.

I don't know if this is coincidence or if it has something to do with the anomalies you've pointed out, but turn 12 also happens to be an even multiple of 4 and 6.

HarunTaiwan:

The point of reassessment after a load game is that it makes the AI less predictable. You can't say, "Oh, I didn't know it was going to do that. I'll just load the last turn and play a little differently."

[ShellShock]

Thank you Bromley.

Your second analysis answers the question that occurred to me on reading your initial results - the number of territories that changes hands is a much better measure of the effectiveness of the AI compared to the number of territories each faction owns. I assume as players we generally prefer an AI that is aggressive, expansive and therefore more likely to attack (if not conquer) our territories.

The community owns you a big thanks for the time you've put into this.

I hope this data will give CA some insight into the game mechanics over time, and will help them in their plans for the future of the TW series, if not RTW itself.

[Puzz3D]

In the test I did with a Julii campaign, the war declaration didn't change between playing continuously and reloading.

Julii imperial campaign, M/M difficulty, FOW false:

Start, 270 BC. Position the map so that you can see both Syracuse and Athens. Hit end turn.

Turn 1, 270 BC. Scipii sieges Syracuse which can hold 7 turns. This initial siege of Syracuse might be scripted. Hit end turn.

Turn 2, 270 BC. Decline the trade offer from Gaul. Syracuse can hold for 6 turns. Hit end turn.

Turn 3, 269 BC. Accept the marriage offer. Syracuse can hold for 5 turns. Hit end turn.

Turn 4, 269 BC. Syracuse can hold for 7 turns. (Perhaps an unsuccessful assault was made and the siege has been reinstated.) Macedonia sieges rebel owned Athens which can hold for 6 turns. Macedonia is at war with the rebels and no one else. Save the game. Hit end turn.

Turn 5, 268 BC. Syracuse can hold for 6 turns. Athens can hold for 5 turns. Hit end turn.

Turn 6, 268 BC. Syracuse can hold for 5 turns. Athens falls to Macedonia. Exit the game.

Load the savegame made on turn 4. The random number generator could introduce a difference at this point because after a reload the AI may not perform the exact same sequence of decisions requiring a random number. It's possible that a random number is used to weight the decision to break a siege, but I find this test to be very repeatable. Hit end turn.

Turn 5, 268 BC. Scipii lifts the siege on Syracuse and moves away. Macedonia lifts the siege on Athens and moves away. Macedonia is still at war with the rebles, and the Macedonian army which was sieging Athens moved north to an area where there are no rebel armies. There is in fact no observable threat to Macedonia anywhere on the map, and they are not at war with any other faction. Hit end turn.

Turn 6, 268 BC. Scipii sieges Syracuse which can hold for 7 turns. Macedonia sieges Athens which can hold for 6 turns with the same army as before. This is a futher indication that there are no other threats to Macedonia because the AI returned to the same strategy is was using before the save on turn 4.

End of test.

Conclusion:

Clearly, loading the game caused both sieges to be lifted because neither siege was lifted when play was not interrupted by the save/load cycle. The expansion by Scipii and Macedonian is set back by the lifting of these sieges. It's apparent that the AI reformulated it's siege strategy, and it took more than one turn to get the sieges back in place. It took the first turn (turn 5) to reassess the situation just as CA claims, and the second turn (turn 6) to reinstated the sieges. This reassessment was necessary only because the AI was not fully aware of the situation after the reload. The course of the campaign is altered by saving and loading because Macdonia should already be in possession of Athens on turn 5, and Scipii should be further along in it's conquest of Syracuse. This has a far reaching effect because a faction is set back economically for the rest of the campaign, it affects all AI factions and happens repeatedly as often as you reload.

It's true than not all sieges are necessarily lifted after a reload, but from observation with FOW off I would estimate that 90% are lifted. The AI may be moving these formerly sieging armies away from cities because it's programmed not to position them that close to cities that it isn't sieging.


Bromely's data is very nicely done, and it doesn't surprise me at all that it shows 4 continuous turns to be the level which allows expansion since the max siege length is typically 7 turns. I'll bet the average siege length is about 4 turns. I would say to get on a level footing with the AI you have to not only play at least 4 turns per sitting but also break off your own sieges and perform no diplomacy on the turn after a reload.

[roguebolo]

OK, after confirming that there was a breaking point at Turn 12 in my game, I began to wonder a few things. Why was it that if I did save/loads and "End Turn" consecutively after turn 12 that other factions would not take rebel provinces, when they seemed to have no qualms about doing it when I started from Turn 14 or Turn 19? The only thing I did other than building my internal infrastructure between turns 12 and 19 was establish alliances. But how could that have such a drastic effect on the AI?

So I went back to the original test scenario, starting at Turn 1. This time, I looked at the diplomacy panel each turn, both with save/loads and without them, to determine which factions were allies and which had declared war and -- more importantly, because it's not explicitely listed in that panel -- which ones had established a ceasefire from the previous turn.

As anyone who has played MTW is familiar, the course of the game was largely decided by the alliances that you established. The same is true of RTW.

It appears that in the reassessment process, the AI does not merely reassess its moves -- it reassesses who would make the best allies and the best enemies before making any other decisions. When "load game" is NOT used, it sticks with its original choices.

When a "load game" causes it to reassess the political landscape, and it sees multiple choices that are equally viable, it will often establish a ceasefire with hostile countries and select a new target -- I suppose all on the grounds of remaining "unpredictable". (It also has the byproduct that the human player can exploit load games to change the political landscape, allowing him to establish alliances that were previously not possible.)

The end result is that repeated "load games" on consecutive turns throws it into a floundering, confused state, with so many allies and enemies to select from, of switching sides over and again. You can confirm this by watching as ceasefires go into effect when sieges are released, and as they begin to ally with factions that they were besieging a few turns ago.

To further complicate things, the AI apparently percieves the Rebels as "just another faction" and sees the same benefits of nonhostile relations with them as with other factions. So the rebel factions make out like bandits (no pun intended) in this situation.

However, as I've pointed out, there seems to be a stabilizing factor that occurs as you move further and further away from this point of extrema, which is partially provided by the actions of the human player. As the human player begins to interact with the other factions, establishing declarations of war and ceasefires and alliances, and as other factions do so as well, each faction starts getting "locked in" to a situation in which it is less and less desirable to cease hostilities or establish or break alliances. This is the stabilizing factor that made it so much more difficult to recreate the phenomenon in midgame.

It does have some kind of an effect on the beginning of the game; primarily that if you load and save a lot, it will take longer for rebel territories to be occupied. The fact that the Scipii have two extra territories and the Greek Cities and Carthage each have one less territory before this stabilization process begins, assuming that you did not interfere with that course of events by using load games or by establishing alliances which prevent it, has very little effect on the progress of the game as a whole; in fact, as the Julii I'm more likely to be at war with Carthage or the Greeks before the Scipii, so it makes the game a little LESS challenging without the save/loads. Note that load games are not the only thing that will prevent the Scipii from conquering Sicily -- you can intentionally prevent it by manipulative use of alliances.

But the fact that save/loads causes rebel territories to be occupied later is more of a problem, because it sets back the development of those territories by several turns.

In my real game, the situation was becoming stabilized by turn 14; but as you can see, I was making a considerable number of alliances. It may take longer for players who wait for the AI to establish the political landscape.

Now that I understand the situation a little better, I actually do have a real gripe about the fact that rebels are being treated the same as other factions. Although it may not ultimately have a major impact on midgame or endgame, which is really the part that I'm looking forward to, it does have a minor impact. And furthermore, it just plain doesn't make sense.

I'm not exactly sure why you guys are studying this issue if CA has stated they do not intend to release another patch because of their two-patch limit. I think in the long run bugs and glitches and other phenomenon like this one will make themselves evident; what really needs to be addressed is the two-patch limit. Speaking as a programmer, I can honestly state that this is a ridiculous, arbitrary policy and needs to be changed. So go for it. I'm going back to my game. If there's any way I can help with your efforts, let me know.

And, btw, congratulations on whoever first discovered this phenomenon. Good work!

[Kraxis]

Wow... I'm impressed. You guys have done a gerat job.
It seems the player will have to play a good number of turns before doing a load, so that the political scene can work towards active AI factions. But even in that case many people are forced to reload pretty much every turn for a wide range of reasons. For them it is still impossible to get the AI to pose a threat, at least a fun one.

We know they read this forum, so this is more of a plea.

[HarunTaiwan]

So, roguebolo, should I save load every turn, for even BETTER AI?

Notice Bromley confirms that's it's all rebel territory many without walls that you are seeing being taken, and not AI territory.

I doubt alliance does much at all, since the AI breaks those at will.

I suspect the reasssessment if it exists leans toward peace and friendship, which could explain the protectorate issue a bit. Maybe it's designed so the human player who is coming back has some breathing space. LOL

Also, Roguebolo, the last thing you want in a wargame is stability. You need to feel threatened as well as threatening. In MTW, you definitely felt that. In RTW, only if I play a small nation am I ever worried.

[Kraxis]

Nah, I rather think that the AI goes "Hey what is going on here... Ok I better get in a position where I can formulate a good strategy (hence sieges broken off) and I need to secure myself (hence acceptance of Protectorates)."
Basically the AI goes into 'start'-mode. It starts over again as if the game starts at that date, which seems to be a fitting explaination as the info to make the AI realize it was actually in a running game could possibly be rather substantial, and an obvious place to cut if size was demanded smaller.

[roguebolo]

I labeled each of my save games before "End Turn" as "1st Turn", "2nd Turn", etc...all the way to the 20th turn. I was quite surprised when I looked back and saw this because I had no idea how useful it would become a few weeks later for these tests.

I had just reinstalled RTW and installed Patch 1.2 for the first time. (I had been playing a different game for a couple of months.) I played as the plain-vanilla Julii (again) since all the factions were locked and I decided to unlock them with gameplay rather than by modifying the text file, as before. Assuming that I did not accidently skip a turn, I started the tests just before hitting "End Year" of the 19th turn. At any rate, it was the Winter of 261 B.C.

At the end of the turn in Winter 261 B.C., despite frequent saves and loads, here was the political climate:

1.) The Seleucids and Egypt were at war, Egypt gaining a territory on turn 14 and losing a territory on turn 18; the Seleucids gained a territory on turn 18.
2.) Parthia and I (Julii) both gained a territory on turn 2.
3.) The Brutii gained a territory on turn 4 and another on turn 14.
4.) Pontus gained a territory on turn 14 and another on turn 16.
5.) The Greek Cities were at war with Macedon and Pontus.
6.) Egypt and the Seleucids were at war.
7.) Brittainia and Germania were at war.
8.) Scythia and Parthia were at war.

Alliances:

1.) Scythia was allied with Armenia and the Seleucids.
2.) Carthage and Spain were allies.
3.) Parthia and Thrace were allies.
4.) As the Julii, I was allied with Carthage, Macedonia, Gaul, Germania, Dacia, the Greek Cities, Numidia, Scythia, Spain and Thrace. I believe I initiated all of those alliances with the possible exception of Gaul.

On the following turn, I began an invasion of Gaul and we declared war, which is why I chose this turn to start testing stagnation; up to this point my only military conflict had been with Segesta and random rebel stacks.

I think I will go back a little further toward the point of greatest extrema -- say, turn 12 since most territorial acquisitions began around turn 14 -- and see if the byproduct of a stagnant AI is more pronounced. At that point only Parthia, the Brutii, and myself had gained a territory.

[HarunTaiwan]

Did Parthia gain the rebel territory?

I think Julii and Bruti do that as well.

Could it be AI versus rebels as opposed to other AI is different?

[HarunTaiwan]

Note Pontus and Egypt could also be taking Rebel only territory.

[HarunTaiwan]

4.) As the Julii, I was allied with Carthage, Macedonia, Gaul, Germania, Dacia, the Greek Cities, Numidia, Scythia, Spain and Thrace. I believe I initiated all of those alliances with the possible exception of Gaul.

LOL. That's pretty amazing for 20 turns. If you had asked them all to become protectorates, you'd have sewn the game up. When I play without save/load, I don't think I can achieve so many alliances.

[roguebolo]

Well, I guess that's just proof that I do save/loads every turn, in support of the other data I've provided.

I always send out my diplomat by boat from the first turn of the game. He moves from faction to faction trying to sell trade rights, usually unsuccessfully after which he gives them away. Then he tries to sell an alliance, which he also gives away if unsuccessful. As the Romans, it's easier to get alliances because it inhibits other Roman factions from warring with the proposed ally and often causes a ceasefire if hostilities are already underway. The AI is aware of this and likes that insurance policy. The wealthy Greeks are particularly suckers for this technique. If trade rights have already been established, an alliance seems easier to sell or give away. The last thing he tries to sell is map information.

I also build a diplomat in my territories so that I can approach foriegn diplomats traveling nearby. Sometimes I reject their diplomatic offers and come back to them with the same offer on the following turn, so that my diplomat will get the influence bonus.

I will check which territories were taken over, and I'm also going to try the same test from Turn 12.

[roguebolo]

To be quite honest with you, I had never succeeded in getting a faction to submit as protectorate until this most recent campaign. I had beaten Gaul back to just it's capital in Celtiberia, and laid siege to it, and they patently refused Protectorate status. I read about the save/load exploit on a forum, and I was absolutely determined to have them become a protectorate even if I had to use an exploit. So I tried it, and it had absolutely no effect. They only finally caved in after the siege had been held for several more years -- for a price, of course.

I ran into the same situation with Spain a few turns later at their capital in Lusitania. They had no other territories and their capital was under siege. Again, the save/load had no effect. However, after a few turns of being besieged $14000 convinced them.

I should have save games of both of these situations as well.