A start on the .MESH file format

[KnightErrant]

@GOM

Of course, you have my permission to do anything
that helps figure things out.

This Saturday is one of those errands and yardwork days
(spring has come to Huntsville) but I'll start back again
tonight looking at things. Maybe I should make my cobbled
together script more general so it searches for the start of
the mystery block for any unit rather than a hard-coded
byte value. Any thoughts on whether that would be helpful?

@Casuir
The first picture you posted looks exactly like what I'm
seeing, shadows and stuff on different guys in the unit
depending on lighting.

KE

[GrumpyOldMan]

Hi Guys

I finally made my in-game stuff look like yours. Looks like the mystery blocks are not just harmless boolean streams as we were led to believe. I've sent a PM to Caliban and hopefully we can get some more information.

What this means -

Can we still go ahead? - Yes, with the restriction that we can only use M2TW groups until we figure out how to replicate the shading data. However, creative texturing and use of alpha maps will give some leeway for variations.

Is Milkshape still an option? Yes, I can store the mystery blocks within the file and parse it out on conversion to Mesh.

All is not lost and the way ahead is clearer now

Off to mow the lawn, then rewrite the converter and read up on shader streams .

Cheers

GrumpyOldMan

[alpaca]

Heh you guys got me interested. Is any of you up to brief me a bit on this via MSN, ICQ or IRC later next week?
I could try to help you figure out the mystery values afterwards (well I have no idea about 3d programming but I'm good at puzzles) - and I prefer chat to forums

[KnightErrant]

@alpaca
Well GrumpyOldMan is the most knowledgable person on the
team about 3D graphics but if he's too busy I can give you
my less in-depth understanding. You'd have to tell me how to do
the protocols you mentioned; I've haven't used chat.

@anybody
This is a noobie request but I never learned how to post screenies
and this will be necessary for dealing with the mystery data.
Could someone point me to a tutorial or give a brief description.
Please include something about hosting services.
Thanks.

[zxiang1983]

@KE, Let me do this easy job:)
1) I suggest this site to host your image:
https://imageshack.us/

2) The webpage is very user-friendly. Click "browse" button on the right and choose the image you want to upload and then click "host it!" at the bottom

3) After all is done you will get several address. Just copy the "Hotlink for forums" line and paste it in you post. Then others can see your image.

[KnightErrant]

@zxiang1983
Many thanks, my friend. I'll
try a test this afternoon.

edit: that was fast trying it now.



Wow! That was easy! Thanks again!

This isn't the best picture of the mystery data effect but some of the guys are shadowed
because of the zeroed out mystery data.

[Foz]

Could it have something to do with reflections? It occurs to me that there's been no discussion yet of how the engine determines the reflective properties of a given mesh, yet that is almost certainly something you'd want to include, as the various other mesh aspects you guys have discussed can't really tell you the reflective properties of the materials involved.

[Re Berengario I]

Which parts of the mesh are reflective is handled by the alpha channel of the normal map texture, but those "shadows" seems related at what color is reflected (in 3ds max would be the specular color). Usually an environment map is used for reflections, the testing is easy though, just change the normal texture and see what it happens to the new reflective parts.

[Caliban]

Those screenshots you posted Casuir look like the envmap (cubemap) at full strength. If you turn the unit's normal map alpha to white, you will get the same effect. When we create meshes in 3dsmax, we also specify the material shader per lod. Each lod has it's own shader so we can control the amount of pixel/vertex shaders on each lod for performance reasons. When these are set, shaders are turned down or off depending on the distance lod.

I have sent a bit of info off to our programmers to find out more about the mystery data.

[KnightErrant]

This baby ain't cracked yet....But it's a crackin'

Maybe doing yardwork was a good idea this weekend, I finally found the
misplaced 2 byte in the granny strings. Most frustrating side-by-side
hexedit session ever. Anyway I got the conversion from .mesh to .ms3d
done last Wednesday and pulled in the feudal_kinghts_lod0.mesh file and
changed the lance as so:




Today I finally got the reverse process to work. Here's the modded
file in a custom battle. Look at the lances in the foreground.




Note that most of the bone info is hard coded into the
script. The .mesh to .ms3d relies on the hierarchy tree that GOM
sent me for the .smd format. The other way is also hard coded and
very brittle. You can't make the slightest error with the header/footers
or granny strings.

Please regard this as a proof of concept only. There's a lot more work
ahead but like GrumpyOldMan said, things are looking clearer. If Caliban
cracks the whip to get info on the mystery data I'm very hopeful.

Permit me a little happy modder's dance here:

KE

Edit: Hi Caliban, nice to see you. Yes, please more info on the mystery data!

[Caliban]

Well I should have checked my email first thing this morning, I had already been sent a bit of info about this problem:

Character models use this vertex format:
Position : float[3]
Normal : byte[4] // only first 3 bytes are used for xyz, last one is 0
TexCoord : float[2]
Tangent : byte[4] // same format as normat
Binormal : byte[4] // same format as normat
BoneIndiciesAndWeight : byte[4]

so there are streams that each mesh should have. They need to find out which they know of and the mystery blocks will likely be the rest of them.

These are the IDs that should be used to mark them in the file:

Position: 0
BoneIndiciesAndWeight : 2
Normal : 3
TexCoord : 4
Tangent : 10
Binormal : 11

Without analyzing their data I've noticed they've been highlighting number 3 somewhere, which could mean it's normals.
I think only lod0 has all that info, other lods probably drop tangent and binormal, the rest should be there. In theory they could mark even lod0 to use lod1 material and get away with not having these two components, but that would remove per pixel normal mapping and replace it by just vertex lighting.

To get more info about what tangent and binormal is, they should look up tangent space normal mapping as that's what we're using. Mesh needs to contain valid tangent space (normal, tangent & binormal).

Nice lances KE, are they for charging around corners :)

[KnightErrant]

Hi Caliban,
Don't know, I was too excited to let the battle play out.
Maybe they just play bumper cars with them.

Thanks for the info. The 3 precedes the first mystery
block and the 10 and 11 precede the following two from what I've seen.
So that's normal, tangent, and binormal. Well I've been computing
vertex normals to go into the .ms3d format so that leaves tangent
and binormals. I thing I'm relieved this is geometry stuff rather than
something else because we should be able to compute that.

Edit: Wait, I don't know what the bleep I'm talking about. Those are
byte streams so are they just low resolution vectors, meaning we
compute a real live vertex normal in float and then coarse map it
to 0-255 for each x,y,z component?

[Re Berengario I]

No, normal vertices using normal maps have native values ranging from 0 to 255 because they used the RGB channells of a bitmap (the "normal map" exactly) to store the data.

If you open a normal map of MTW2 you'll see by yourself what I'm talking about...

This is an example taken from this forums

[KnightErrant]

Yes, I've opened one of them before but didn't understand
what I was seeing (still don't). But anyway, I think you answered
my question, I don't know what Caliban just told me. We have
normal, tangent, and binormal data in the mystery blocks and these
are indices into the "normal map" but I don't know what that is.
Someone tried to start a tutorial on the normal map over at twcenter
but I think I scared him off by asking too many questions.
I think I'm the most "fish out of water" player in this whole problem; I'm working
on 3D graphics with absolutely no knowledge of 3D graphics.

Edit: Accidentally posted early trying to preview. Meant to follow up with:
Can you recommend a good reference on this subject so I can educate
myself?

[Re Berengario I]

Ok a normal map is simply a way to fake geometries which don't exist.

Pratically a normal map "displaces" the normal of a vertex using the values contained in a RGB image so to achieve a 3D bumping effect.

The RGB colors (each 8 bits/1 byte in a 24 bit image) are in reality the XYZ of the normal that is used to create the effect.

From Wikipedia:
the red channel should be the relief of the material when lit from the right, the green channel should be the relief of the material when lit from below, and the blue channel should be the relief of the material when lit from the front(practically, full except on the "slopes"); or, to put it another way, the XYZ coordinates of the face normals are placed in the RGB values of the normal map. If a material is classified as being reflective, the albedo is usually encoded in the alpha channel if one exists.

This is the easiest way I know to explain what a normal map is.

[KnightErrant]

Ok, becoming clearer now. Most of the mystery data
has three byte values followed by a zero byte but Casuir reported
about a few instances where the fourth byte was non-zero,
should I assume that might be a case where the albedo
is being encoded in the alpha channel, assumed to be
the fourth byte?

[Suraknar]

hrm...

This looks like Vertex Shading, Shininess and refraction in relation to weather lighting and environment, so it is information for the engine's rendering instructions.

Just a thought, are these mystery values the same accross the board for all units?

If they are then maybe the Converter could just add(insert) this information to the final .mesh file upon export?

EDIT: NVM, I posted this after reading page 5 of the thread, did not notice there was Page 6, which I am reading now :P

[KnightErrant]

Yes, the format seems to be the same for all units.
The aztec variants that Casuir found had float triple
streams for the mystery data but those units aren't
used in-game (don't appear in the EDU) so were they
mistakes, early prototypes with a different format, etc. ?

Did a quick google on normal map tutorials and there
are a few so maybe I should just read up for a few evenings
until I can catch up.

[Re Berengario I]

For what you are interested in a normal map it's just another texture because most of the 3d programs know how to handle it. It's not geometry for them.

From the Caliban post I understood that the mystery block values are the tangent and biNormal (bitangent) vectors of the model to support normal mappin in per pixel lighting through shaders.

[Re Berengario I]

Ok, because I need to be forgiven for the last cryptic post this is a good and simple tutorial about the argument:

http://www.3dkingdoms.com/tutorial.htm

and this for tangent space normal mapping

http://www.3dkingdoms.com/weekly/weekly.php?a=37

[KnightErrant]

Well when GrumpyOldMan comes online he'll understand this
so I guess I shouldn't be too worried about my ignorance.
What I would like is an assurance of is this data computable
from the vertex vectors, weights, and uv coords so that when
we pull all of that out of a mesh and put it into the .ms3d
format (which wants the vertex normals) can we go back the
other way and put it all back into the mesh format and compute
all the required mystery data. (I realize we may be "talking past each other"
on this question, I see this problem from data in a file format and you
guys who actually make models may be seeing this as something
you apply in a given application.)

Edit: You're too fast, I didn't even get a chance to see that.
I'm going off to study this stuff and then I'll understand it better.
Many thanks for being patient with me.

2nd Edit: Yes, that's the tutorial I found. Reading it now.

3rd Edit: THANK YOU! This has math in it. Now I'll understand what we've
been talking about.

[Re Berengario I]

I don't possess deep knowledge on normal mapping as I usually let the 3d application handle it but in this case they use directly a shader so those values (if I'm not completely wrong) are a 3x3 matrix for each mesh vertex.

(From the second link above:)
To use tangent space normal mapping, we need to send the tangent and bitangent vectors to the vertex shader. Then we can create a 3x3 rotation matrix from them that can be used to rotate vectors such as the light vector and half-angle vector into tangent space.
Now we can dot these vectors with a normal from our tangent space normal map, using the same pixel shader as with object space normal mapping.

From Caliban's post:

To get more info about what tangent and binormal is, they should look up tangent space normal mapping as that's what we're using. Mesh needs to contain valid tangent space (normal, tangent & binormal).

[KnightErrant]

Exactly! from the same article

void ComputeTangentBasis(
      const Vec3& P1, const Vec3& P2, const Vec3& P3, 
      const Vec2& UV1, const Vec2& UV2, const Vec2& UV3,
      Vec3 &tangent, Vec3 &bitangent )
{
   Vec3 Edge1 = P2 - P1;
   Vec3 Edge2 = P3 - P1;
   Vec2 Edge1uv = UV2 - UV1;
   Vec2 Edge2uv = UV3 - UV1;

   float cp = Edge1uv.y * Edge2uv.x - Edge1uv.x * Edge2uv.y;

   if ( cp != 0.0f ) {
      float mul = 1.0f / cp;
      tangent   = (Edge1 * -Edge2uv.y + Edge2 * Edge1uv.y) * mul;
      bitangent = (Edge1 * -Edge2uv.x + Edge2 * Edge1uv.x) * mul;

      tangent.Normalize();
      bitangent.Normalize();
   }
}

This is exactly the education I was looking for. Thank you very much
Re Berengario I. I'm am deeply in your debt.

[Suraknar]

KE,

Just as a small test here,

when you compare feudal_knightslod0 and feudal_knights_lod1, is the mystery bytes information the same?

[KnightErrant]

No, in all the "normal units" I've looked (a very small subset unfortunately)
the lod1, lod2, and lod3 only have one mystery block, set off with a
granny int of 3 0 0 0 which, according to Caliban's info, means normal
info only, no tangent or binormal. Is that significant?

Edit: Sorry, if you're asking if the first mystery block is the same for
each lod I don't know. I can find out but I have to run some examples.

[Re Berengario I]

There's probably no normal mapping on low resolution meshes (it wouldn't make any sense anyway), so tangent space vectors are not needed.

[Casuir]

Casuir reported
about a few instances where the fourth byte was non-zero,
should I assume that might be a case where the albedo
is being encoded in the alpha channel, assumed to be
the fourth byte?

Just having a look back on the files I looked at before some of them are indeed the base figures, others are ingame and while they do have blanks in one of the first three values I havent seen anything in the fourth yet, its possible I was looking at a bone stream as I dont think I'd noticed that they came last in some files at that time.

So we have all the info now to make a mesh? Where do we stand on formats, is milkshape able to handle the normal/tangent/binormals?

The first block shouldnt be the same if the geometry is different btw

[KnightErrant]

@Casuir
Sorry, I might be misattributing from my faulty memory.
I got the transformation to go both ways but its really
highly tied to feudal_knights_lod0. In my slightly dazed
condition from all this texture mapping information people
have been giving me I can't make a determination but
I THINK if we sort it out we, I mean GrumpyOldMan, can
make a converter that goes both ways. The mystery
data seems computable, I do vertex normals, and a few
posts back was some C++ math about the tangents and
binormals that seems doable so maybe if I can compute this
and stuff it back into the mesh we have a demo of the
process?

Could you try a different unit and see if it crashes or works.
The bones stuff is way out of my league, I keep seeing funny
blue lines for bones so I don't know if this is my basic ignorance
of Milkshape or if I've reversed the vertex normals in my conversions.

[Suraknar]

@ KE

Edit: Sorry, if you're asking if the first mystery block is the same for
each lod I don't know. I can find out but I have to run some examples.

Yes, reading above from what Caliban has posted in combination with the excelent information Re Berengario linked, thank you RB :), I would assume that the block of information in question here would indicate a somewhat different information for eahc LOD.

Now, if this is indeed the case, then a second peace of information here would be to confirm if that information is the same between all lod0's, lod1's etc...

In other words, if the info from one unit's lod0 is the same with the info of another units lod0, and the info of one unit's lod1 is the same with anotehr unit's lod1, while being different from another lod level.

It would mean that the engine uses a uniform scheme for each lod across the board, which can simplify the task of the converter to reproducing (copy-paste) these values for each lod instead of trying to calculate them on a per modded model.

As I said before I am no programmer, so I cant verify the validity of all this but from what RB linked, I understand the process conceptually. So before continuing on what I am trying to say I'll post the relevent info.

----

So this..

What are the tangent and bitangent vectors used in tangent space normal mapping? A plane's normal is perpendicular to the plane, while the tangent and bitangent are parallel to the plane. Their direction is determined by the UV coordinates, one points in the direction of U-axis in 3d space, the other in the direction of the V-axis.

The tangent space normal map stores in each component the distance along each of these vectors. For example, one component of the normal map (usually stored in the blue part) points directly away from the mesh triangle, ie. along the triangle normal. This allows the same normal map to be reused on different parts of a mesh.

with this...

Blending the Tangent/Bitangent Vectors

Let's say you are creating smoothed vertex normals, by adding the face normals of adjacent triangles for each vertex, then normalizing the vector. You can do the same thing with tangent and bitangent vectors.

Note that the 3 vectors (normal, tangent, bitangent) may not form an orthogonal basis since the tangent & bitangent directions are based on texture uv coordinates and possibly averaged from multiple vectors. What you can do is use Gram-Schmidt orthogonalization. This is performed simply by subtracting the part of a vector in the direction of another vector then renormalizing. For example the code to make the tangent orthogonal to the normal is:

tangent -= normal * tangent.dot( normal );
tangent.normalize();

Once you have two orthogonal vectors you can use the cross product to compute the 3rd. The orthogonalization step seems to be mainly necessary on complex curved models or places with texture seems. Many texture seems however will cause weird lighting no matter what, so if the tangent/bitangent vectors are too far apart you'll probably be better off leaving a split at the seem and not smoothing them together.

And finally this...

Using Tangent Space with Shaders

To use tangent space normal mapping, we need to send the tangent and bitangent vectors to the vertex shader. Then we can create a 3x3 rotation matrix from them that can be used to rotate vectors such as the light vector and half-angle vector into tangent space.

float3x3 objToTangentSpace = float3x3( IN.Tangent, IN.Bitangent, IN.Normal );
OUT.halfVec = mul(objToTangentSpace, halfVec );
OUT.lightVec = mul(objToTangentSpace, lightVec);

Now we can dot these vectors with a normal from our tangent space normal map, using the same pixel shader as with object space normal mapping.

Are important steps to come to this information in the .mesh format that you see, as I understand it.

Added to all this is a process that the documentation calls normalize in order to better use Tangent Space with vertex and pixel shading.

----

Now returning to the idea here, that after reading page 6 of this thread I think GOM also had.

If, it is proven that the game uses a uniform scheme for handling the units and their various lod's.

Then in the converter we need to not worry about calculating and producing these values ourselves.

The converter could simply insert that information in the appropriate lod.

So the modder would start by loading an lod0.mesh file, make their desired changes, and save as .ms3d.

Then when converting to .mesh, the converter could produce 4 files instead of one.

lod0, lod1 and lod2 and lod3 respectivelly with the appropriate tangent space mapping information to each level.

maybe this is a simpler approach for all?

[Casuir]

@KE I wont be able to do anything with the script you sent me till later but I'll try it out on a few other models with different makeup later and see if anything looks odd.

@Suraknar, the normal/tangent/binormals are calculated per vertex so two different models are going to have two different sets of values, i.e pasting the data from a file with 4000 vertices into a file with 5000 vertices will crash the game.