A start on the .MESH file format

[Suraknar]

Hrm...

I kind of understand now why the Data\Text\export_units.txt is not a .txt file this time around...methinks we were not expected to be making new models.

But, I have to say many of you seem to be doing very great progress here :)

[Suraknar]

Alrighty,

After a bit of reasearch, I came up with this:

http://www.radgametools.com/granny/download.html

You can freely Download the Granny Viewer, which comes with 2 sample .gr2 files, which may help you ellucidate some things if you can analyse them.

It seems like Radgametools however holds the licences (and they seem a greedy bunch too), many other games have this issue, and I got this from moding forums of AOEIII.

Correct me if I am wrong but from what I understand what your trying to do here is break the .mesh format not the gr2 format, and then if you can do that it could be reversible providing us with a way to do some modeling.

I will be looking for alternatives such as using .xml templates to probably be able to mix and match models (Body of A with head of B) from existing stock models (should not be against any licensing doing that).

I wish success to you :)

[KnightErrant]

@Suraknar
Yes, what GrumpyOldMan, Casuir, and myself have been calling
header/footer lines are probably granny strings. So we're essentially
trying to reverse engineer the granny strings and understand their
syntax to make a back and forth converter that makes no use of
any proprietary software. This would then be legal and freely
distributable.

[Lusted]

I kind of understand now why the Data\Text\export_units.txt is not a .txt file this time around...methinks we were not expected to be making new models.

You could say the same for all the fiels which are now .string.bins, and which alpaca has made a converter for.

[alpaca]

You could say the same for all the fiels which are now .string.bins, and which alpaca has made a converter for.

Nah they were just converted into a format that could be read more quickly by the machine.
The game was probably developed without thinking of modding to any large extent (except for some individuals, such as Caliban and Palamedes) and therefore the designers and programmers just used what yielded the best results in the shortest time possible.

Now it seems that we won't get any more official tools any time soon (maybe Caliban can explain why?) I'm glad to see that you guys are making quite a good deal of progress on this, and even more, provide us with a possible free way of editing models
I wish I had more time on my hands so I could work a bit on this, too. Anyways, after you cracked this customer, you should also have a look at the settlement packages (it seems there's some cool stuff in there and it's a lot more flexible than RTW)

[GrumpyOldMan]

Hi Guys

With regards to the standard '256' and non-standard '0' meshes I thought I'd go and look at some of the model files (export_descr_unit.txt and battle_models.modeldb ) to see if I could get some clues . The non-standard files are NOT used. They are WIPs and anyway the vertex assignments are all screwed up because of the funny skeletons. After I realised this the plaintive cries of "Dohhh" echoed off the hills for a long time, this shows you shouldn't get too focussed on just the one issue. With this in mind I'll 'lock out' the non-standard meshes since everything they have is available on standard meshes anyway.

I'd just like to make the point that I am not interested in reverse engineering anything, Granny and .gr2 are a mystery to me and will forever remain so. I am just looking at the .MESH format that CA is using and looking at ways we can modify these files. There are lots of bits of data that I don't understand and I don't want to understand but I don't need to as I can just use them. Just a clarification in case anybody from Radgames is interested.

Getting closer now.

@Casuir

You said you did some tests on taking out the boolean streams and using the figure in game. What exact blocks did you take out? Just the num_vertex and stream figures, with the header/footer stuff intact or the header/footer stuff as well?

Interesting stuff you pulled out on weapon bones, we'll have to look closer at that stuff to see how we actually get a figure in game.


I'm PMing Caliban with some questions on weapon protocol. It's just about sorted out in my head (and backs of envelopes), it's a matter of getting it into code.

Cheers

GrumpyOldMan

[Casuir]

They're definitly the exported caracters that are processed by the xmls, think theres a bit more to it than simply adding weapons, theres wip/base meshs in the mounts folder as well, geometry seems to be the same but the ingame file is considerably smaller than the base one, havent looked at them other than that though. Dont think we'll be able to replicate the method CA used to make the models anyways.

As for the booleans, I changed the 03 byte to 00 then tested that, then re-wrote most of the values with 00's then tested that, finally I deleted from the 2 byte num_vertex to the last byte of the third data stream (inclusive) and tested that. I also edited the texture on the model in question so I'd be sure the game was using the edited files.

[Casuir]

Question here on weapon bones, a weapon skeleton(s) is specified in the unit xml which is used to compile the finished .mesh so I'm guessing this bone data is pulled from there?

	<Skeletons>
		<Skeleton>
			<Mount>Horse</Mount>
			<Primary>MTW2_HR_Spear</Primary>
			<Secondary>MTW2_HR_Non_Shield</Secondary>
			<PrimaryAttachment>MTW2_HR_spear_Primary</PrimaryAttachment>
			<SecondaryAttachment>MTW2_Sword_Primary</SecondaryAttachment>
		</Skeleton>

There looks to be a number of different primary/secondary skeletons in the idx, would these affect things or are they just neccessary due to the way ca compiled the models? Also could animated weapons be possible?

Re bounding spheres, theres only 4 values here so its xyz and a radius aye? From what I've seen the radius for dismounted is usually around 1.0-3 and mounted lancers 2.0-1, mounted archers and such are usually the same as infantry so I'm guessing the mounts are handled separately and weapons affect the size.

I will be looking for alternatives such as using .xml templates to probably be able to mix and match models (Body of A with head of B) from existing stock models (should not be against any licensing doing that).

This wont be possible with an xml file afaik, the data is mixed up to much for it to be possible without extracting the parts from the .mesh and writing a completely new file. Changing primary/secondarys is possible with a bit of hex editing but unless you had some way of automatically re-writing the triangle values it could take you some time.

[Caliban]

I think maybe the knifeman has an animated weapon but I'm told animating the weapon bone can cause some freaky results.

The skeletons below simply tell the game what animation set to use for this unit. The example below means that his primary attack is a Spear animation set and his secondary is non_shield.
You can look up these animation sets by searching for MTW2_HR_Spear in data\descr_skeleton.txt All unit skeleton types are listed in here.
For example if you wanted him to play sword animations for his secondary, you would replace MTW2_HR_Non_Shield with -> MTW2_HR_Sword
If you wanted his primary to be mace animations you would replace MTW2_HR_Spear with -> MTW2_HR_Mace

Hope this helps :)

<Skeletons>
<Skeleton>
<Mount>Horse</Mount>
<Primary>MTW2_HR_Spear</Primary>
<Secondary>MTW2_HR_Non_Shield</Secondary>
<PrimaryAttachment>MTW2_HR_spear_Primary</PrimaryAttachment>
<SecondaryAttachment>MTW2_Sword_Primary</SecondaryAttachment>
</Skeleton>

[Suraknar]

This wont be possible with an xml file afaik, the data is mixed up to much for it to be possible without extracting the parts from the .mesh and writing a completely new file.

Indeed you are right Casuir,

Working from an Export of an Export is unfortunently not possible the XML way :( as I have come to realise in some tests and recently got it confirmed aswell.

So your efforts, KnightErrant and GrumpyOldman are the best way to go about it :)

Go! Go! Go! hehe :)

[Casuir]

It was more the PrimaryAttachment and SecondaryAttachment skeletons I was wondering about. Looked at the desc_skeletons and theres a few weapons which have animations other than stand_idle or default but they seem to be mainly ready, hold and release animations for missiles. Doesnt look to have any effect ingame though. What I was wondering was would it be possible to animate the actual weapon itself, ie have the arms of the bow draw back when drawn and return when the arrow is released.

Btw theres more than one weapons bone in the mesh, whats the story there?

@Suraknar, GOM and KE are the guys you should hang your hopes on, the major work heres being done by them and my contribution is very limited compared to theirs.

[Caliban]

It was more the PrimaryAttachment and SecondaryAttachment skeletons I was wondering about. Looked at the desc_skeletons and theres a few weapons which have animations other than stand_idle or default but they seem to be mainly ready, hold and release animations for missiles. Doesnt look to have any effect ingame though. What I was wondering was would it be possible to animate the actual weapon itself, ie have the arms of the bow draw back when drawn and return when the arrow is released.

Btw theres more than one weapons bone in the mesh, whats the story there?

Ok, the attachment skeletons are only used with weapons that contain more than one bone. Which is the case for say swords but not for shields.
Sword has an additional sword bone that is attached to a hand, and we use that skeleton to animate it. But the shield is attached directly to the hand bone so there is no need for additional animation.

At one stage there were going to be proper animated weapons such as bow strings etc but there were issues with this and so it never made it in. I don't think it will be possible to animate a bow string straight up, it would properly require a bit of hackery. I'm told animating weapon bones can cause some wierd effects. There is also a lot of junk files in the folders, tests and obsolete units etc, there was simply no way to check what resources the game used and so you might find a few things that were left in.

[KnightErrant]

Regrets on the reverse engineering remark. I just meant
we want to understand header/footers enough to know
how to cut out things like the mystery blocks and still have
the units play in-game. The thing that worried me was looking
at a lod2 mesh file that only had one mystery block and not
three but still had this header:

 18  0 48  0  0  0  1  0  0  0          1924  (vert. wts: float pairs)    0  0  0  0 17  0 49  0  0  0 18  0 48  0  0  0  0  0  1  0  0  0  0  0

 23  0  1  0
       50  0  0  0  0  0  0  0  0  0    1924  (vert. vecs: float triple)  0  0  0  0 22  0  1  0
                                                                                           51  0  0  0 23  0 50  0  0  0  0  0  2  0  0  0  0  0

 28  0  1  0
       52  0  0  0  2  0  0  0  0  0    1924  (vert. bones: byte quads)   0  0  0  0 27  0  1  0
                                                                                           53  0  0  0 28  0 52  0  0  0

 28  0 54  0  0  0  3  0  0  0          1924  (mystery strm: byte quads)  0  0  0  0 27  0 55  0  0  0 28  0 54  0  0  0
  0  0  0  0  0  0  0  0  0  0
Bounding sphere data: 4 floats
0.487427771091 0.393853664398 0.109047822654 1.71645057201

This was the armoured_sergeants_lod2.mesh. I thought that the
field, the one in red, was supposed to mean three blocks followed
and that we could zero it out and remove the data.

So now I'm confused as well, how do we repackage the mesh data
around the excised data? I echo GOM's question for Casuir, what is the
sequence of steps needed to remove mystery data? This might
help thrashing out the header/footers; I'm not convinced I split them
right, I just thought it was plausible.

Side note: I just took the unit count of 701 from the top of the modeldb file
and multiplied by 4 for the number of meshes to get something like
3000. Not every unit has 4 meshes, a lot have 3, and there are some
leftovers in the modeldb like the baltest units and some en_dummy units
near the end that aren't referenced in the EDU so they're not real units.

[Casuir]

Well the few bits that were left in have helped us figure stuff out so cheers for that :) Shame about the animated weapons, would have been a nice touch. I'm guessing we'd have seen flails too if it had panned out. What was the main idea behind having attachments sets btw, less resource usage?

@GOM Doubt the other non-standard meshs will work ingame, they get skeletons assigned to them by the xmls used to add the weapons so I'm guessing thats why the vertex datas screwed up. The xmls Caliban provided can be found here incase you havent seen them https://forums.totalwar.org/vb/showthread.php?t=74083
Interesting all the .gr2's referenced here and in the other leftover attsetdb files are found in the unit_models folder but as .mesh's not .gr2

I wouldnt worry about too much functionality, as long as it can export functional .mesh's most people will be happy enough.

@KE

As for the booleans, I changed the 03 byte to 00 then tested that, then re-wrote most of the values with 00's then tested that, finally I deleted from the 2 byte num_vertex to the last byte of the third data stream (inclusive) and tested that. I also edited the texture on the model in question so I'd be sure the game was using the edited files.

Cant test at the mo but I'm guessing if the boolean values arent used then it doesnt matter how many streams it says will follow.

Btw theres 3213 meshs including mounts and generals

[zxiang1983]

Hi all! I've gone through this whole thread for no less than 10 times and finally I think I understand most of its content, such as the structure of mesh file etc... But I still don't know how to import the mesh file (or part of it) into milkshape 3D. Would someone please tell me so I can join this historic work, too:)

[taylorj2]

Hi Guys, I tryed out the Rhino Program 3.0 and downloaded the plugin for MESH files, but it doesn't work still. It seems as though it should because even the Rhino program as options for joining triangles and stuff, can anyone help me why I can't import them?

[Andromachus Theodoulos]

@taylorj2 and zxiang1983

I think the guys (GOM, KE, Cas) are still working hard on trying to establish what exactly is necessary within the mesh files. I think they are also trying to figure out how to get the components repackaged so that they will play in game. They are getting very close. Hopefully it won't be too much longer and we can start making changes to meshes.

Keep up the good work!!!

AT