Mechanics Research Process

When therother and I started the Ludus Magna back in the RTW days, one thing I wish we had done is have a discussion on the best way to research battle mechanics. So, let's do it for S2!

Researching battle mechanics seems straightforward enough: start a custom battle, pick the same units (or slightly variant units if that's what you're testing), line up units and hit go, record data as the battle progresses, and then graph it and analyse.

The problem with that process is that the game throws strange little extras into the mix that are not, and may never be, entirely known -- general bonuses, flanking bonuses, speed bonuses, morale bonuses, etc etc.

One way to deal with that is to do the tests via MP, and have both players record all the data they can. This limits the number of unknown variables. However, it's also a pain to coordinate unless the two players 1) happen to live next door, and 2) are pretty patient people. (and yes, I was trying for the alliteration)

Are there things you all have found to counteract these problems? Anyone found a really fool-proof way to run mechanics tests?

Battle mechanics always were harder to get right in the modern TW games. I think MP or a very controlled AI battle, one unit vs one unit on a flat map probably give the most reliable/repeatable results. Don't know how tricky it is to arrange an MP battle though. Fancy giving it a go?

An effort to come up with some form of standardization or best practices for reporting the results of mechanics tests would also be worthwhile. It can be challenging to read through threads on this forum, because it's often unclear what might be someone's uninformed opinion vs. someone's well-tested findings.

Yepp, totally agreed gauch0. Enforcing that is pretty challenging though, from past experience.

However, I think if we at least put a simple framework together, sticky it, and then use it in all our own posts, we should be a long way towards that goal.

therother, are there times that work generally for you during a typical week? I have a fairly flexible job, so most of the time I can get time during your evenings if that works out best (-7 GMT here... what better way to spend a lunch hour? :) )

One thing to note is that test results vary between trials, even though conditions are identical. So you have to do repeated trials and take an average. A single test can yield an exceptional result that can lead to the wrong conclusions. This random element to combat is a good thing I think, but it means when I do testing I do 10 trials with each set up.

Question for all the researchers out there...

How do you (personal you, you yourself) record data when testing infantry? Archers are very straightforward because there are discrete kill points at which you can record data (i.e. the volley). Infantry are just in there hacking at each other. All I could think to do was record time delta from first contact to rout, and population at routing.

It would be ideal to record population every x seconds, but that will take more time than I've got.

Thanks for any ideas.