Gendarmes vs. Templars: a statistical analysis

[Kobal2fr]

Puzzling fact : France vs Spain control group should have come out as 50/50 instead of 60/40. What does that tell us ?

[Rebellious Waffle]

It's awfully close to 50/50; positing that the procedure will even out to 50/50 as the sample size increases, the chances of getting a 60/40 split in 20 trials is

(20 choose 12)*(0.5 to the 20th power) = 0.12013

that is, it should happen about 12% of the time. Unusual, but not outrageously so. We should be more suspicious if the 60/40 split continued as the number of trials increased, but with 20 trials we can afford some leeway.

[John_Longarrow]

Kobal,

My guess is that, since France is supposed to have better cavalry then CA has hard coded a bonus to any cav unit that is French.

A way to test that would be to do the same kind of experiment with English Mailed Knights VS French Mailed Knights. I'm guessing we'd see the same 60/40 split.

[Rebellious Waffle]

That, or the AI does something goofy. There are typically three ways the AI will respond:

1) Rushing its cavalry at you all at once,
2) Rushing a bit of the unit followed by the larger remainder,
3) Reversing the soldiers' position instead of charging (right to left)

When (1) or (2) happens, success is up for grabs. When (3) happens the AI almost always loses because it doesn't get a charge bonus. What we might be seeing in the 60/40 split is the AI's tendency to flip its horsemen instead of charging -- I won't know that for sure until I revise my experiment to control for that variable, but that's what strikes me off the top of my head.

[FactionHeir]

You really ought to write my thesis with all that stat knowledge

But the findings are interesting. I don't think the 60/40 is significant, considering it numerically means 12-8, but Gendarmes are 10(11)/15 and templars 13(14)/16, meaning the templars should theoretically win.
Gendarmes do have higher armor rating though, so if they get hit at the backside, they tend to live longer.

A major skew in your findings will be the presence of a captain though.

[Doug-Thompson]

Impressive.

FactionHeir's point is not a quibble, though. If one side's captain dies first, that will skew results. I'd argue that the effect of that should have been countered by the sheer number of the tests. Still, it would be good to know whose captain died first in the contests.

[Kobal2fr]

Kobal,

My guess is that, since France is supposed to have better cavalry then CA has hard coded a bonus to any cav unit that is French.

A way to test that would be to do the same kind of experiment with English Mailed Knights VS French Mailed Knights. I'm guessing we'd see the same 60/40 split.

Kinda doubt it. France has better cavalry because they get Lancers, Noble Knights that are probably a bit better than the generic Chivalric ones (though I don't know how - the only difference is their higher upkeep and mount types, but mailed horses and barded horses have the same stats and mass so...) and Gendarmes, plus the Tourney line of buildings.

Oh and @OP yes, I had forgotten about it, but Factionheir is right : the "1 unit vs. 1 unit on grassy plain" test setting is kind of screwy because the occasional early general death can mess with the results. Admittedly, it should be statistically insignificant since it should happen to both sides equally but with 20 tests only it's still noticeable.

[FactionHeir]

Not only the death, but also the positioning of the captain and how many he gets to kill. Captains and generals tend to do one hit kills and are very tough to kill.

The good way to do that test would be to give each side a very slow unit as the general so only the knights get to duke it out properly. Something like halberd militia or so. What you can do is mod your unit file and give each side say a ballista as the general's unit and in the projectile file mod ballistae to have 0 accuracy, so they will always miss.

[Rebellious Waffle]

So, new refinements to the experiment include:

1) Recording whose general died first;
2) Increased sample size;
3) What the AI did -- part charge first, all charge together, flip.


The last might be hard to do since we'd effectively be drawing with replacement, and we'd need maybe 10 of each reaction to gauge how much they matter to the outcome of the whole engagement. We might be able to get around that by bringing up the sample size to, say, 50 or 60; since the AI likely chooses which course it takes at random, we can follow the dictum to "control what you can, randomize the rest" after that fashion.

[FactionHeir]

You'd get more reliable results doing what I said because then it also takes out the captain attack factor.

[Rebellious Waffle]

Aye, I hit the button to make that post before the thread was updated with what you said, so I waited to say anything about it to avoid double-posting.

The zero-accuracy artillery piece sounds like a good idea, but I don't trust myself enough around the config files not to do something terrible and muck up the program beyond fixing. A wise man once said, "no product is foolproof; fools are ingenious" -- a maxim whose truth I prefer not to test until I'm certifiably unfoolish and know what I'm about. Barring that, it may be just as well not to introduce other units onto the field because the game shifts from pure tactics to part strategy, and unexpected confounding variables could well creep in as a result.

[Rhedd]

Where is the file that determines accuracy for siege engines and missile troops, anyway?