Investigation of AI reassessment upon reload

[Bromley]

EDIT: Apologies, but I have missed out Rhodes (Greeks) in all of my tests. It's immaterial to the conclusions though.

I did a short extension, just to see what happens when alliances/wars etc are firmly in place.

Loaded the no-loads 260 file. Then (a) played to 250 with no loads and (b) played to 250 with 4 turn loads (i.e. long enough for the AI to recover from the previous load but still expected to impact AI behaviour).

Here are the results with a few notes. Basically, loading cut the Rebels some slack but also prevented the destruction of Gaul and held Egypt back. Also, the Brutii didn't try to establish a foothold in North Africa.



Interestingly enough, if you look at the absolute differences for the 250 no-save and 250 interval 4 compared to the 260 load used to start the test, you see some startling results.

The bottom line shows that the no-load has increased in total by 7 and the Int-4 by 3. A big difference, but not huge.

But then look at the territories by country though and compare them to the 260 start.
* Looking at the absolute changes (i.e. ignoring + and -, just the number)
* Removing the number of rebel provinces in that amount (which we know by looking at the total provinces owned before and after)
* Dividing the resulting number by 2 (as all provinces were taken from another faction, so are double counted)

That gives some stunning results. No provinces were taken from other factions in the interval 4 run, whereas 8 were taken in the no-load run.

That's huge. I know there might be a few logic errors in what I just did (as I haven't done maths for 15 years ), but not enough to explain away a variance that large.

Where it says "20", that's meant to read "start" or "260". Can't be bothered to fix the image .

[roguebolo]

[edit] Sorry, I misunderstood something. OK, the AI factions have all made alliances but the human player has not made any alliances nor expanded yet after 20 turns, right? That introduces an entirely different extrema that is a little hard to concieve of, since the AI definitely attempts to respond to the human player.

I'm finding that alliances are actually a key factor. If I load my game at turn 14, then play forward to turn 19 with or without save/loads, I get results that are identical to when I was playing the actual game. The same factions have acquired the same territories, except that when I did not use save/loads Egypt apparently beat the Seleucids in their war because on turn 18 the Seleucids did not gain a territory and the Egyptians did not lose one; so the game with save/loads was actually less static, but that's merely a matter of an autocalculated battle turning out differently.

HOWEVER, there IS a breaking point. If I move back to turn 12 and play the game forward to turn 19 with save/loads, none of the factions take their rebel territories. Whether I use save/loads or not, the alliances and declarations of war are considerably different than when I had established alliances on my own, as well as considerably different than each other. The save/loads seem to be affecting who they select as allies and who they select as enemies, much the same way that random events are affected by a load game.

I know exactly why different declarations of war are occuring. At the beginning of turn 12, my only alliances were with Gaul, Germania, the Greek Cities, Numidia and Spain. At the beginning of turn 14, most of my alliances had been established and included Carthage, Germania, and Thrace as well, thereby preventing many of the other factions from declarations of war in order to keep their alliance with my Roman faction. Without these alliances, which is the only thing I did while actually playing the game except for building my faction's infrastructure, the other factions were free to make war or love as they pleased, without any regard to my faction's alliances. This strongly reminds me of MTW.

When starting from turn 12, without my alliances to guide their decisions, the alliances and declarations of war that they chose were considerably different depending on whether I used save/loads each turn or not. Without save loads, several of the factions were less active (but not inactive) in taking rebel territories, and several other factions were more active. However, WITH save/loads, no faction took any rebel territories. If this had something to do with the different nature of the alliances and war declarations, because they thought they needed the forces elsewhere, or because of other reasons, I cannot say. In any case, it does seem to support your idea that the rebels get a break, and it also seems to be growing in prevalence as I move closer and closer to the point of greatest extrema. But let me point out that it also seems to grow less in prevelance as I move away from it, so that by midgame it seems to have no effect on gameplay at all.

I don't know if this is coincidence or if it has something to do with the anomalies you've pointed out, but turn 12 also happens to be an even multiple of 4 and 6.

HarunTaiwan:

The point of reassessment after a load game is that it makes the AI less predictable. You can't say, "Oh, I didn't know it was going to do that. I'll just load the last turn and play a little differently."

[ShellShock]

Thank you Bromley.

Your second analysis answers the question that occurred to me on reading your initial results - the number of territories that changes hands is a much better measure of the effectiveness of the AI compared to the number of territories each faction owns. I assume as players we generally prefer an AI that is aggressive, expansive and therefore more likely to attack (if not conquer) our territories.

The community owns you a big thanks for the time you've put into this.

I hope this data will give CA some insight into the game mechanics over time, and will help them in their plans for the future of the TW series, if not RTW itself.

[Puzz3D]

In the test I did with a Julii campaign, the war declaration didn't change between playing continuously and reloading.

Julii imperial campaign, M/M difficulty, FOW false:

Start, 270 BC. Position the map so that you can see both Syracuse and Athens. Hit end turn.

Turn 1, 270 BC. Scipii sieges Syracuse which can hold 7 turns. This initial siege of Syracuse might be scripted. Hit end turn.

Turn 2, 270 BC. Decline the trade offer from Gaul. Syracuse can hold for 6 turns. Hit end turn.

Turn 3, 269 BC. Accept the marriage offer. Syracuse can hold for 5 turns. Hit end turn.

Turn 4, 269 BC. Syracuse can hold for 7 turns. (Perhaps an unsuccessful assault was made and the siege has been reinstated.) Macedonia sieges rebel owned Athens which can hold for 6 turns. Macedonia is at war with the rebels and no one else. Save the game. Hit end turn.

Turn 5, 268 BC. Syracuse can hold for 6 turns. Athens can hold for 5 turns. Hit end turn.

Turn 6, 268 BC. Syracuse can hold for 5 turns. Athens falls to Macedonia. Exit the game.

Load the savegame made on turn 4. The random number generator could introduce a difference at this point because after a reload the AI may not perform the exact same sequence of decisions requiring a random number. It's possible that a random number is used to weight the decision to break a siege, but I find this test to be very repeatable. Hit end turn.

Turn 5, 268 BC. Scipii lifts the siege on Syracuse and moves away. Macedonia lifts the siege on Athens and moves away. Macedonia is still at war with the rebles, and the Macedonian army which was sieging Athens moved north to an area where there are no rebel armies. There is in fact no observable threat to Macedonia anywhere on the map, and they are not at war with any other faction. Hit end turn.

Turn 6, 268 BC. Scipii sieges Syracuse which can hold for 7 turns. Macedonia sieges Athens which can hold for 6 turns with the same army as before. This is a futher indication that there are no other threats to Macedonia because the AI returned to the same strategy is was using before the save on turn 4.

End of test.

Conclusion:

Clearly, loading the game caused both sieges to be lifted because neither siege was lifted when play was not interrupted by the save/load cycle. The expansion by Scipii and Macedonian is set back by the lifting of these sieges. It's apparent that the AI reformulated it's siege strategy, and it took more than one turn to get the sieges back in place. It took the first turn (turn 5) to reassess the situation just as CA claims, and the second turn (turn 6) to reinstated the sieges. This reassessment was necessary only because the AI was not fully aware of the situation after the reload. The course of the campaign is altered by saving and loading because Macdonia should already be in possession of Athens on turn 5, and Scipii should be further along in it's conquest of Syracuse. This has a far reaching effect because a faction is set back economically for the rest of the campaign, it affects all AI factions and happens repeatedly as often as you reload.

It's true than not all sieges are necessarily lifted after a reload, but from observation with FOW off I would estimate that 90% are lifted. The AI may be moving these formerly sieging armies away from cities because it's programmed not to position them that close to cities that it isn't sieging.


Bromely's data is very nicely done, and it doesn't surprise me at all that it shows 4 continuous turns to be the level which allows expansion since the max siege length is typically 7 turns. I'll bet the average siege length is about 4 turns. I would say to get on a level footing with the AI you have to not only play at least 4 turns per sitting but also break off your own sieges and perform no diplomacy on the turn after a reload.

[roguebolo]

OK, after confirming that there was a breaking point at Turn 12 in my game, I began to wonder a few things. Why was it that if I did save/loads and "End Turn" consecutively after turn 12 that other factions would not take rebel provinces, when they seemed to have no qualms about doing it when I started from Turn 14 or Turn 19? The only thing I did other than building my internal infrastructure between turns 12 and 19 was establish alliances. But how could that have such a drastic effect on the AI?

So I went back to the original test scenario, starting at Turn 1. This time, I looked at the diplomacy panel each turn, both with save/loads and without them, to determine which factions were allies and which had declared war and -- more importantly, because it's not explicitely listed in that panel -- which ones had established a ceasefire from the previous turn.

As anyone who has played MTW is familiar, the course of the game was largely decided by the alliances that you established. The same is true of RTW.

It appears that in the reassessment process, the AI does not merely reassess its moves -- it reassesses who would make the best allies and the best enemies before making any other decisions. When "load game" is NOT used, it sticks with its original choices.

When a "load game" causes it to reassess the political landscape, and it sees multiple choices that are equally viable, it will often establish a ceasefire with hostile countries and select a new target -- I suppose all on the grounds of remaining "unpredictable". (It also has the byproduct that the human player can exploit load games to change the political landscape, allowing him to establish alliances that were previously not possible.)

The end result is that repeated "load games" on consecutive turns throws it into a floundering, confused state, with so many allies and enemies to select from, of switching sides over and again. You can confirm this by watching as ceasefires go into effect when sieges are released, and as they begin to ally with factions that they were besieging a few turns ago.

To further complicate things, the AI apparently percieves the Rebels as "just another faction" and sees the same benefits of nonhostile relations with them as with other factions. So the rebel factions make out like bandits (no pun intended) in this situation.

However, as I've pointed out, there seems to be a stabilizing factor that occurs as you move further and further away from this point of extrema, which is partially provided by the actions of the human player. As the human player begins to interact with the other factions, establishing declarations of war and ceasefires and alliances, and as other factions do so as well, each faction starts getting "locked in" to a situation in which it is less and less desirable to cease hostilities or establish or break alliances. This is the stabilizing factor that made it so much more difficult to recreate the phenomenon in midgame.

It does have some kind of an effect on the beginning of the game; primarily that if you load and save a lot, it will take longer for rebel territories to be occupied. The fact that the Scipii have two extra territories and the Greek Cities and Carthage each have one less territory before this stabilization process begins, assuming that you did not interfere with that course of events by using load games or by establishing alliances which prevent it, has very little effect on the progress of the game as a whole; in fact, as the Julii I'm more likely to be at war with Carthage or the Greeks before the Scipii, so it makes the game a little LESS challenging without the save/loads. Note that load games are not the only thing that will prevent the Scipii from conquering Sicily -- you can intentionally prevent it by manipulative use of alliances.

But the fact that save/loads causes rebel territories to be occupied later is more of a problem, because it sets back the development of those territories by several turns.

In my real game, the situation was becoming stabilized by turn 14; but as you can see, I was making a considerable number of alliances. It may take longer for players who wait for the AI to establish the political landscape.

Now that I understand the situation a little better, I actually do have a real gripe about the fact that rebels are being treated the same as other factions. Although it may not ultimately have a major impact on midgame or endgame, which is really the part that I'm looking forward to, it does have a minor impact. And furthermore, it just plain doesn't make sense.

I'm not exactly sure why you guys are studying this issue if CA has stated they do not intend to release another patch because of their two-patch limit. I think in the long run bugs and glitches and other phenomenon like this one will make themselves evident; what really needs to be addressed is the two-patch limit. Speaking as a programmer, I can honestly state that this is a ridiculous, arbitrary policy and needs to be changed. So go for it. I'm going back to my game. If there's any way I can help with your efforts, let me know.

And, btw, congratulations on whoever first discovered this phenomenon. Good work!

[Kraxis]

Wow... I'm impressed. You guys have done a gerat job.
It seems the player will have to play a good number of turns before doing a load, so that the political scene can work towards active AI factions. But even in that case many people are forced to reload pretty much every turn for a wide range of reasons. For them it is still impossible to get the AI to pose a threat, at least a fun one.

We know they read this forum, so this is more of a plea.

[Bromley]

Thanks for the hug ShellShock .

Another extension of my test. I loaded up the 260 save where I'd not save/loaded during play. That gave me some nicely expanded AI factions, so they could be expected to turn most of their efforts towards other factions rather than Rebels.

I then played 20 turns with a 2 turn save/load interval. But this time, I was active and really played them. If nothing else, this proved to me that the Medium portion of the strategic game is ridiculously easy . Greece and Macedon both felt my wrath.

Check out the results. Although there's obviously a slight problem with my method of determining how many provinces were taken from other faction (as I know I took 6 yet the function spews out 5), the variance is pretty immaterial.

What isn't immaterial is that I was responsible for ALL of the faction on faction province swaps. So I think that it's fair to assume that, for the purposes of AI expansion, player activity is not a determining factor.

I'm not going to perform an interval-2 test with no activity by myself because I think it's pretty obvious from earlier tests that the AI won't expand. I've included the 250 no-load data where I was passive for comparison - you can see that the AI might be expected to have nabbed 8 provinces off other AIs in those 20 turns without the save/load feature (on top of an extra 5 Rebel provinces)..

[roguebolo]

But even in that case many people are forced to reload pretty much every turn for a wide range of reasons. For them it is still impossible to get the AI to pose a threat, at least a fun one.

Not really. It just takes longer for the game to reach a more stable situation. The political climate will be DIFFERENT, not less challenging; the factions you are at war with may be different and they may be stronger or weaker, but eventually you need to face the majority of them and their wishy-washy alliance changes become less likely. However, the same logic that allows it to change alliances if they look equally viable could cause a long-time ally to suddenly break their alliance with you and form one with your primary rival. Or vice versa. Especially right after a load game, or any other event that triggers a reassessment.

The territories that started out as rebels, however, will always be underdeveloped if you were using a lot of load games in the earlier moves of the game. That's why my main gripe is regarding the fact that the AI is treating rebel factions as "just another faction." Eventually they will conquer the nearby rebel territories...just a little too late, and too slowly.

In my game I was saving and loading every turn, and the political climate *started* stabilizing by turn 14 -- but as someone mentioned, I formed an awful lot of alliances. It would take quite a bit longer if the human player is not as proactive about forming alliances. I like forming alliances early because it gives me more control over the political climate.

[drone]

Something Puzz3D said got me thinking:

Turn 1, 270 BC. Scipii sieges Syracuse which can hold 7 turns. This initial siege of Syracuse might be scripted. Hit end turn.

Does anyone know if the first AI moves are scripted? If they are, do the scripted decisions get saved and how do they fit into the reassessment? If they are scripted but not saved, this could be causing problems at the start of campaign games.

Nice work, Bromley!

[HarunTaiwan]

So, roguebolo, should I save load every turn, for even BETTER AI?

Notice Bromley confirms that's it's all rebel territory many without walls that you are seeing being taken, and not AI territory.

I doubt alliance does much at all, since the AI breaks those at will.

I suspect the reasssessment if it exists leans toward peace and friendship, which could explain the protectorate issue a bit. Maybe it's designed so the human player who is coming back has some breathing space. LOL

Also, Roguebolo, the last thing you want in a wargame is stability. You need to feel threatened as well as threatening. In MTW, you definitely felt that. In RTW, only if I play a small nation am I ever worried.

[Kraxis]

Nah, I rather think that the AI goes "Hey what is going on here... Ok I better get in a position where I can formulate a good strategy (hence sieges broken off) and I need to secure myself (hence acceptance of Protectorates)."
Basically the AI goes into 'start'-mode. It starts over again as if the game starts at that date, which seems to be a fitting explaination as the info to make the AI realize it was actually in a running game could possibly be rather substantial, and an obvious place to cut if size was demanded smaller.

[HarunTaiwan]

I agree with you. I think the results show if you play 6-7 turns it's not so bad...so I will play less now since sometimes I only have 20-30 minutes...shouldn't have had that kid, should I?

[roguebolo]

Notice Bromley confirms that's it's all rebel territory many without walls that you are seeing being taken, and not AI territory.

Right. I've done the same tests. I've watched what happens in Sicily without load games. I've also watched what happens in midgame with them. Even then the AI will relieve sieges, but the net effect is different. It continues to threaten the same territories. Like I said, don't take my word for it. I'll send you a number of different save game positions that show it to you, from the same game even.

The truth is that the AI has never caught me with my pants down so that waiting out a siege would ever work anyway...so it boils down to a matter of, "Should I sally or meet them in the field?"

The fact of the matter is that in the first 20 moves, without load games, it's primarily rebel territories that fall.

I doubt alliance does much at all, since the AI breaks those at will.

Here, I agree with The Shogun in that there is (in his words) "a misunderstanding of the AI." In midgame, the AI definitely does not easily break or form alliances. There's a very clear logic on what it considers acceptable. However, in the early phases of the game it will change them willy-nilly whenever there's a reload. It doesn't seem to merely reassess the situation and roll some dice for 50% odds -- it seems to intentionally reverse its previous decisions.

Play through the test scenario one more time with load games, but this time watch what happens in the diplomacy Pane with Egypt/Seleucids/Armenia or the factions in Sicily. Make note of ceasefires as well. You'll soon see a pattern of ... they removed the siege ... a ceasefire is in effect ... now they've formed an alliance ... etc. I should have recorded that information and posted it because it gives some good insight as to what is going on.

At any rate, if you doubt that alliances have an effect on the AIs willingness to follow through with sieges, I can illustrate with specific save games in which that is the one and only determining factor that modifies the behavior of the AI, despite load games. Isn't there an upload area for sharing data?

the last thing you want in a wargame is stability. You need to feel threatened as well as threatening.

Now, that's taking everything I've said out of context. Stability is a very generic word which can mean a lot of different things. I think you know perfectly well that I was not referring to a "comfortable, easy-going game where everything is under my control." I was referring to a stable political landscape which causes the AI to behave in a more stable, reasonable manner. You certainly don't want the kind of instability that it exhibits early in the game with numerous save/loads to continue throughout the game.

Sometimes I wonder how many people here are actually playing games beyond the first twenty moves. I can definitely see that there has not been much testing beyond that period of extrema. But to truly understand the phenomenon, you need to come up with instances in which it is NOT exhibited, and determine why. You need both a positive and a negative.

While the statistical data supporting the load game behavior is very good, I have seen very few suggestions that explain this behavior -- nothing that illustrates a truly good understanding of it.

It's a hard proposition to make projections about the overall difficulty of the AI factions in midgame on the grounds of its early behavior resulting from load games. Generalized statements like "it's less aggressive" don't carry much weight in the light of actual savegames which illustrate the AI's midgame behavior to be substantially different. They tend to be semantical statements arguing a foregone conclusion of an empirically observed and repeatable fact, and that's simply not good enough. You not only need to be able to make projections about the future behavior of the game, but you need to back them up by observing and measuring the resultant behavior as well.

A similar semantical argument (and similarly illogical) would be, "It sounds to me like you want one of the AI factions to beat all of the others for you, so that you can take it on after it's overextended itself." When the Scipii, your allies at that stage of the game, grow stronger, Carthage and the Greek Cities grow weaker. But the sum total of the AI factions is the same, and must all eventually be defeated.

I really feel that some serious thought should be given to proven instances in which this phenomenon does not seem to exist or does not seem to exhibit the projected results. I don't think it will be fully understood unless those are understood as well. I have a number of such save games, which is why I've asked several times about sharing them with people. I'd like someone else to take a look at them and see what conclusions they come to. If anyone is interested, please PM me and I can send them by email.

[Bromley]

How does that fit with my last test? That seemed to show, to me, that the AI broke off all sieges (note that I did not visually confirm this, it's just that they ceased to expand into each other and only took a few Rebels). That's even with a human playing and after the diplomatic scene had been given 20 turns to mature (although the human was passive in those first 20 turns).

Also, although without save/loads the Rebel territories are a larger part of the total territories taken in the first 20 turns, there were more provinces swapped in my 270-260, no save/load test than taken from Rebels. The proportion of Rebel territories to all territories taken approaches 100% as save/load interval is reduced. I personally believe that this is because the Rebel territories are easier to take (fewer defences, smaller defending armies, no supporting armies), but may be wrong there.

[Puzz3D]

Basically the AI goes into 'start'-mode. It starts over again as if the game starts at that date, which seems to be a fitting explaination as the info to make the AI realize it was actually in a running game could possibly be rather substantial, and an obvious place to cut if size was demanded smaller.

I think that start-mode is a good word to describe what's happening. The AI has to rebuild some memory resident data structures.