Settlements: File Formats; Take 2

[alpaca]

Yeah, you just have to keep in mind to not post more than 50 lines or so.

Ok so the anim file apparently contains the damage and open door anims, what's in animinstances?

[KnightErrant]

Aah, so the 50 lines may have been the problem.

Haven't done anything more, BUT life looks managable at work
now. Will try to get to the end of towns and then comparing villages
to towns may reveal something. Will look at animinstances next with
maybe a check on large_towns to see what else may come into play.

At this point why am I doing this?

(1) Hundreds of modellers are waiting to mod settlements?
Naw, hardly think so.

(2) Tens of modellers are waiting to mod settlements?
Naw, hardly think so.

(3) Bwian is waiting to make Hobbit villages?
Probably.

(4) I've bit off more than I can chew and am just too embarrassed
to admit it.
Yeah, that one! That's exactly it!!!

Sheer bloody-mindedness is a strong motivator.

KE

[alpaca]

Yeah I know the feeling. If it makes you happy though I'm also looking forward to edit settlements (I liked playing Lego and what Caliban told me about the process it's a lot like that, it only looks much cooler )

[Redmeth]

I can't speak for everyone, but I for one and I think most of the EB team is also tremendously interested in the possibility of editing settlements.

[SigniferOne]

KE, how can you say that!



Imagine that in M2 engine.

[KnightErrant]

Ok, point taken. I am encouraged to keep going.
Here's a link to the beta reader.

http://www.twcenter.net/forums/downl...o=file&id=1578

The two files in the zip are animationlibrary.py and settlementutilities.py.
animationlibrary.py is almost the same as the one distributed for the
animation utilites except I put a trap in the function getstring to jump
back out if it gets a requested number of characters more than 1000.
This usually means I'm reading the wrong int and it puts Python into a loop
where it beeps every time it reads the bell code, very annoying to have
happen at work!

settlementutilities.py is the one to double-click on. Just put
north_european_village_a.world and north_european_town_a.world
in the same directory with the two Python files. Currently the .txt
file doesn't have the actual mesh data output to it. If you want this
just search for WRITEDATAFLAG and set it to True. The output file
will be very large!!! You can cut and paste the vertices and tris
to make an obj file. The u and v coordinates are also converted from
RGB encoding into floats if you want them too.

Now for numerology, I'll start. The village has 31 "complexes" containing
a total of 98 "structures" or self contained meshes. The town has 30
complexes with a total of 123 structures.

Villages have a block of data at the top (after the 40 entries block of
which 31 have a positive sequence number that should correspond to the
complexes) that has 867 entries. This number shows up again near the
bottom for another block of data.

For towns the number is 1228.

Question 1: What do these entries correspond to?

For no good reason I kept a running total of the number of triangle groups
defined in all the structures. For villages this number is 1274. Guess what,
there are two blocks of data right at the end (after a block with 867 entries)
that have 1274 entries. For towns this number is 2017 with the same
correspondence. So, of course,

Question 2: What do these entries do or is this a coincidence?

There is a block of data preceded by this comment by me:
# ------------------------------------------
# Beginning of hard coded 17 floats but
# variable ints with 5 byte signatures
# ------------------------------------------
It has 156 entries for villages and 173 entries for towns, BUT, nowhere
can I find that number to read in order to tell me how to read this data.
Right now it is just hard-coded in. This is very unusual for granny strings;
they always tell you how to read themselves so I'm a bit perplexed on this
one. The entries are 17 floats followed by what I read as 5 bytes being
a signature, then followed by a variable number of ints which, by trial and
error, I have made up rules for until I was able to read both cases, village
and town. One entry looks like this:

+1.00000 +0.00000 +0.00000 -0.11595 +0.00000 +1.00000 +0.00000 -4.10875 +0.00000 +0.00000 +1.00000 -4.22434 +0.00000 +0.00000 +0.00000 +1.00000 +1.00000 sig:   1 0 0 2 0 , ints:   0 0 0 0 68 3 0 0 0 0 155 0 0 0 13 4 0 0 220 0 0 0 2 0 0 0 0 0 0 0 3 0 0 0 54 0 0 0

If I rearrange the first 16 floats into a 4x4 matrix I get

+1.00000 +0.00000 +0.00000 -0.11595 
+0.00000 +1.00000 +0.00000 -4.10875 
+0.00000 +0.00000 +1.00000 -4.22434 
+0.00000 +0.00000 +0.00000 +1.00000 

+1.00000 sig:   1 0 0 2 0 , ints:   0 0 0 0 68 3 0 0 0 0 155 0 0 0 13 4 0 0 220 0 0 0 2 0 0 0 0 0 0 0 3 0 0 0 54 0 0 0

In homogeneous coordinates this is a translation with no rotation.
All of the entries seem to be translation only.

Question 3: Why 156 and 173 and how can I figure out when I'm done reading
this data block?

This is probably enough for now. If anyone gets a chance to run the program
and look at the grannies and data blocks please post back ideas.

Edit: Aargh! Found a write out bug for villages. After all the complexes are read
there are three strings for .worldTerrain etc. After that my code wrote out

57 0 1 2 [4307] 0
57 0 1 2 [4307] 0 0
57 0 1 2 [4307] 0 0 0
57 0 1 2 [4307] 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
56 0 1 0 [4308] 57 0 [4307] 0 0 255 255 255 255 0 0

That's not what is in the file just this

57 0 1 2 [4307] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
56 0 1 0 [4308] 57 0 [4307] 0 0 255 255 255 255 0 0

I think my editor auto-indented the write statement to be inside the loop
instead of after it, the Perils of Python.

[alpaca]

Short notice before I leave (and after I return I'll have to do some learning so I probably won't get around to research anything more today):

The 1274 4-byte entries you mention are in my opinion a kind of coordinate-transformation between IDs. They consist of a starting ID and a second int that specifies how many of this structure have that ID (usually 1 but can be 2 or more) so the transformation is between a block ID and a unique ID for each instance.
In fact they're not 1274 entries but 1274 different unique IDs. They're only 867 entries (counting offsets and dividing by 4) and therefore coincide perfectly with your 867.

[Aymar de Bois Mauri]

I can't speak for everyone, but I for one and I think most of the EB team is also tremendously interested in the possibility of editing settlements.

I concur. It would be very sad to see development on this area stopping. Keep up the great work.

[Redmeth]

I have no expertise in this area but this seems like a very complex problem that could be easily solved by CA if they were willing.

Does anyone fancy the idea of a petition by modders addressed to CA with the objective of allowing the building in M2TW and Kingdoms to become available for modding?

I am not volunteering to lead this idea but a lot of people that are the best modders of the community seem to be interested in this and allowing the buildings to be moddable would show CA's goodwill towards the community and encourage modders to do even more extraordinary things.

[alpaca]

I have no expertise in this area but this seems like a very complex problem that could be easily solved by CA if they were willing.

Does anyone fancy the idea of a petition by modders addressed to CA with the objective of allowing the building in M2TW and Kingdoms to become available for modding?

I am not volunteering to lead this idea but a lot of people that are the best modders of the community seem to be interested in this and allowing the buildings to be moddable would show CA's goodwill towards the community and encourage modders to do even more extraordinary things.

Believe me, a petition would have the same effect as swatting a locust in a swarm of two billion.

CA management isn't prone to show any goodwill to modders or even consider the idea. What help we got was because certain individuals (mainly Caliban and Palamedes, with the help of a few programmers) really tried to push things in our favor - but even with this, very little came out of it.

It simply looks like the bigwigs at CA and SEGA aren't interested in modders at all. They tolerate us, but they don't try to help us. If anything, they at least didn't try to stop their subordinates helping us, either.