A start on the .MESH file format

[Tsar Stefan Dusan IV]

First of all, I want to thank You guys for all the hard work!!!
It is nice to know that someone is making any kind of progress on this matter... Keep up the good work as all comunity will love you as I do! :)

Thanks goes to Caliban, too, for helping on those itchy matters.

I can't really help You about the programing and tech part, but am eagerly waiting for Your first release.

My opinion is vertex weighting isn't really hard, at least for experienced artist in 3d field, so my advise would be to concentrate on the vital info, meshes, then you can build on it at later stages. Convert in and outs for .mesh.

Fbx could prove valuable as a wide range of programs use it. Obj format also.
So user can import/export file info in those programs they are familiar with.

Once again, I sencerely bow, as You made my day with lot's of sunshine....:)

I am willing to help in any way I can, maybe test Your plugins in other 3d programs as I am bound to them by my profession and have acces to a wide range...

Sincerely Yours, Tsar Stefan Dusan IV

[KnightErrant]

Well, I don't know if this constitutes progress, but this is
where I'm at right now. I've been studying the fbx format this
past weekend. I downloaded a Milkshape tutorial which had a
form with bones, bone assignments, etc. but before keyframing
(which doesn't seem to be a .mesh attribute). It did have assignment
of vertices to bones but no vertex weighting because Milkshape
doesn't have this right now. I exported this model to 8 file formats
but the only ones that supported bone information seem to be the
native Milkshape ASCII and fbx. fbx is owned by Autodesk now,
(used to be Alias) and doesn't seem to have any format specification.

However, there is a free download from Autodesk of an SDK, in C++,
not one of my languages, which I downloaded and installed. No information
there about its actual format. In the Autodesk forums I found a
main() for converting the fbx binaries to fbx ASCII. Here's the main()
for those interested

int main(int argc, char** argv)
{
	KFbxSdkManager* lSdkManager = NULL;
	MyMemoryAllocator lMyMemoryAllocator;
	KFbxScene* lScene = NULL;
	bool lResult;

	// Specify a custom memory allocator to be used by the FBX SDK
	KFbxSdkManager::SetMemoryAllocator(&lMyMemoryAllocator);

	// Prepare the FBX SDK.
	InitializeSdkObjects(lSdkManager, lScene);

	// Create the scene.

	lResult = CreateScene(lSdkManager, lScene);

	if(lResult == false)
	{
		printf("\n\nAn error occured while creating the scene...\n");
		DestroySdkObjects(lSdkManager);
		return 0;	
	}

	// Save the scene.

	// The example can take an output file name as an argument.
	if(argc > 1)
	{
		lResult = SaveScene(lSdkManager, lScene, argv[1]);
	}
	// A default output file name is given otherwise.
	else
	{
		lResult = SaveScene(lSdkManager, lScene, SAMPLE_FILENAME);
	}

	if(lResult == false)
	{
		printf("\n\nAn error occured while saving the scene...\n");
		DestroySdkObjects(lSdkManager);
		return 0;
	}

	// Destroy all objects created by the FBX SDK.
	DestroySdkObjects(lSdkManager);

	return 0;
}

Anyway here's the bones entries from the Milkshape 3D ASCII format

Bones: 10
"Main Bone"
""
8 0.133333 -68.428429 0.000000 1.570796 -0.000000 -3.141593
0
0
"Pelvis"
"Main Bone"
8 0.000000 0.000000 61.999996 0.000000 0.000000 0.000000
0
0
"Hip left"
"Pelvis"
8 -6.333340 -2.250002 -0.583334 -3.061560 0.070235 -3.141593
0
0
"Knee left"
"Hip left"
8 0.000000 0.000000 25.016495 -0.079812 0.035746 0.000000
0
0
"Calcaneum left"
"Knee left"
8 0.000000 0.000000 32.519444 -1.232623 0.610880 0.000000
0
0
"Toes left"
"Calcaneum left"
8 0.000000 0.000000 11.230869 0.000000 0.000000 0.000000
0
0
"Hip right"
"Pelvis"
8 6.654752 -2.249992 -0.464289 3.061907 0.059931 0.000000
0
0
"Knee right"
"Hip right"
8 0.000000 0.000000 25.124689 0.079672 0.018985 0.000000
0
0
"Calcaneum right"
"Knee right"
8 0.000000 0.000000 32.527370 1.237598 0.569478 0.000000
0
0
"Toes right"
"Calcaneum right"
13 0.000000 0.000000 11.213799 0.000000 0.000000 0.000000
0
0
GroupComments: 0
MaterialComments: 0
BoneComments: 0
ModelComment: 0

And here's the bones from the fbx ASCII file converted by the
above C++ program (Note: I'm not a C++ programmer, the main
shown above is taken from the Autodesk forums. I just built
it with the Autodesk SDK.)

; Object connections
;------------------------------------------------------------------

Connections:  {
    Connect: "OO", "Model::Main Bone", "Model::Scene"
    Connect: "OO", "Model::DefaultMaterial", "Model::Scene"
    Connect: "OO", "Model::jacke_seite_rechts", "Model::Scene"
    Connect: "OO", "Model::Regroup01", "Model::Scene"
    Connect: "OO", "Model::Duplicate04", "Model::Scene"
    Connect: "OO", "Model::Pelvis", "Model::Main Bone"
    Connect: "OO", "Model::Hip left", "Model::Pelvis"
    Connect: "OO", "Model::Hip right", "Model::Pelvis"
    Connect: "OO", "Model::Knee left", "Model::Hip left"
    Connect: "OO", "Model::Calcaneum left", "Model::Knee left"
    Connect: "OO", "Model::Toes left", "Model::Calcaneum left"
    Connect: "OO", "Model::Knee right", "Model::Hip right"
    Connect: "OO", "Model::Calcaneum right", "Model::Knee right"
    Connect: "OO", "Model::Toes right", "Model::Calcaneum right"
    Connect: "OO", "Material::DefaultMaterial", "Model::DefaultMaterial"
    Connect: "OO", "Deformer::Skin DefaultMaterial_0", "Model::DefaultMaterial"
    Connect: "OO", "SubDeformer::Cluster Pelvis_0", "Deformer::Skin DefaultMaterial_0"
    Connect: "OO", "SubDeformer::Cluster Hip left_1", "Deformer::Skin DefaultMaterial_0"
    Connect: "OO", "SubDeformer::Cluster Knee left_2", "Deformer::Skin DefaultMaterial_0"
    Connect: "OO", "SubDeformer::Cluster Calcaneum left_3", "Deformer::Skin DefaultMaterial_0"
    Connect: "OO", "SubDeformer::Cluster Toes left_4", "Deformer::Skin DefaultMaterial_0"
    Connect: "OO", "Model::Pelvis", "SubDeformer::Cluster Pelvis_0"
    Connect: "OO", "Model::Hip left", "SubDeformer::Cluster Hip left_1"
    Connect: "OO", "Model::Knee left", "SubDeformer::Cluster Knee left_2"
    Connect: "OO", "Model::Calcaneum left", "SubDeformer::Cluster Calcaneum left_3"
    Connect: "OO", "Model::Toes left", "SubDeformer::Cluster Toes left_4"
    Connect: "OO", "Material::Material01", "Model::jacke_seite_rechts"
    Connect: "OO", "Texture::.\Test2.jpg", "Model::jacke_seite_rechts"
    Connect: "OO", "Video::.\Test2.jpg", "Texture::.\Test2.jpg"
    Connect: "OO", "Material::Material01", "Model::Regroup01"
    Connect: "OO", "Texture::.\Test2.jpg", "Model::Regroup01"
    Connect: "OO", "Deformer::Skin Regroup01_1", "Model::Regroup01"
    Connect: "OO", "SubDeformer::Cluster Pelvis 1_5", "Deformer::Skin Regroup01_1"
    Connect: "OO", "Model::Pelvis", "SubDeformer::Cluster Pelvis 1_5"
    Connect: "OO", "Video::.\Test2.jpg", "Texture::.\Test2.jpg"
    Connect: "OO", "Material::Default", "Model::Duplicate04"
    Connect: "OO", "Deformer::Skin Duplicate04_2", "Model::Duplicate04"
    Connect: "OO", "SubDeformer::Cluster Pelvis 2_6", "Deformer::Skin Duplicate04_2"
    Connect: "OO", "SubDeformer::Cluster Hip left 1_7", "Deformer::Skin Duplicate04_2"
    Connect: "OO", "SubDeformer::Cluster Hip right_8", "Deformer::Skin Duplicate04_2"
    Connect: "OO", "SubDeformer::Cluster Knee right_9", "Deformer::Skin Duplicate04_2"
    Connect: "OO", "SubDeformer::Cluster Calcaneum right_10", "Deformer::Skin Duplicate04_2"
    Connect: "OO", "SubDeformer::Cluster Toes right_11", "Deformer::Skin Duplicate04_2"
    Connect: "OO", "Model::Pelvis", "SubDeformer::Cluster Pelvis 2_6"
    Connect: "OO", "Model::Hip left", "SubDeformer::Cluster Hip left 1_7"
    Connect: "OO", "Model::Hip right", "SubDeformer::Cluster Hip right_8"
    Connect: "OO", "Model::Knee right", "SubDeformer::Cluster Knee right_9"
    Connect: "OO", "Model::Calcaneum right", "SubDeformer::Cluster Calcaneum right_10"
    Connect: "OO", "Model::Toes right", "SubDeformer::Cluster Toes right_11"
}
;Takes and animation section
;----------------------------------------------------

This looks promising doesn't it? The information is all there.

Could Tsar Stefan Dusan IV or another 3dsmax professional answer
this question? Can 3dsmax import fbx in its ASCII form or must it
be binary. Secondly, can it export both or again, must it be binary?

It's late where I am so I have to go to bed. The vertex and face data
is there along with vertex weighting. I don't understand bone assignments
so I can't judge that. I'll look at it more tomorrow night. Sorry this is
so slow but all this stuff is really new to me.

[Tsar Stefan Dusan IV]

@Knight Errant
I don't really know, but here is some info (if it is, at all)
LINK, but I guess You already visited this one.

What I know for certain is that it can read binary, I'm not sure about ASCII...

Maybe pdf's I attached can help a bit , I don't know. There is a lot of irrelevant information (for this cause), but maybe you can dig something up... I'm sorry if this doesn't helps, but I'm a noob at programing , I'm an artist... I know how to import/export from different programs, but have no idea how it is actually composed, in program language .

http://rapidshare.com/files/18540794/fbx.rar.html

[KnightErrant]

@Tsar Stefan Dusan IV
Thanks very much, I'll look through the stuff in the .rar.
I have looked through the Autodesk forums and
what I've noticed is that every post asking about
the fbx format is never answered by an Autodesk
employee. They'll answer anything else but never
this.

I'll try and figure out how to put mesh data into the
fbx ascii format and then see if I can reverse the converter
code to go backwards to fbx binary and see if Milkshape
can read that.

[Caliban]

Could Tsar Stefan Dusan IV or another 3dsmax professional answer
this question? Can 3dsmax import fbx in its ASCII form or must it
be binary. Secondly, can it export both or again, must it be binary?

The animators here tell me you can import and export .fbx files to and from 3dsmax - Motion builder and most other 3d packages. It seems pretty widely supported. .obj is a good mesh format as it's also widely supported by the big 3d packages.

..but before keyframing (which doesn't seem to be a .mesh attribute).

The .mesh format should be purely for the mesh data and weighting. The actual animation data and keyframes are stored in the old .cas animation format specified by \data\descr_skeleton.txt which references the .cas animations stored in \data\animations

[Foz]

While I don't know much about computer graphics, especially the stuff you guys are getting into here, I do know some about C++. That C++ main() that you posted, KE, reads almost like plain text (to me at least, lol). That could be a very good thing, because it means they have basically everything already done for you and packed into nice functions in the SDK. CreateScene() and SaveScene() look to be the important bits that process all the data and get your conversion work actually done. Since those functions are so easy to use, I'd say it's likely your SDK also can quite easily turn the conversion around for you, probably with only 1 or 2 lines different from the main() you posted.

I would endeavor to comment further, but at this point I've become hazy on how exactly you guys envision the flow of data in this project, and what software exists or must be made to facilitate each step. If you assume .fbx will be the working format for the project, can someone outline where the data starts and what happens to it along the way, as well as which parts have software to execute them and which do not as yet? I'm envisioning a flowchart with data state boxes and process action arrows... I just want to make sure the one in my head is what's really going on lol

[KnightErrant]

@Caliban
Thank you for that. I think Foz is correct that I can easily
make an fbx converter that goes the other way using the
Alias SDK so my original concern over ASCII versus binary has
gone away.

The Mystery Blocks

A reply to Grumpy's question about the mystery blocks:

I read his file decription again and it certainly seems it is the unused bool stream. Just before them there should be number 1 somewhere (number of these streams) followed by number of vertices (this number is the same for all streams) followed by bool values themselves. He should be able to just change that 1 to 0 and then remove count + bool values completely. But I'd have to see the file to be sure, I don't really remember.

I'll have a definate answer on this soon after a bit of investigation.

Do you have any more information concerning the mystery block(s)?
Can we really ignore it or is it in fact some other type of information?

@Foz
Repeating myself but I think you're right, I just need to change 1 to 0
and 0 to 1 and little bit else and I've got an ASCII to binary fbx converter.
Meant to try this last night but didn't even get on the computer.

I'd have to think longer about it to give you a decent description but let
me just do a short one. From the .mesh file we get:
face data (triangles) grouped together with group names
uv co-ordinates (Nvert by 2 array of floats, x y coords)
vertex weights (Nvert by 2 array of floats, primary weight, secondary weight)
vertices (Nvert by 3 array of floats, x, y, z)
bone assignments (Nvert by 4 bytes)
Mytery block?
bone data

Unfortunately this isn't the way Milkshape ASCII or fbx ASCII groups it.
They organize the vertices, vertex normals (which aren't in the .mesh),
uv coords, triangle data, and UV indices? each in their own named group.
The programming task is then to pull out vertices, normals, and uv data
from the mesh's big list, put them in little lists, reindex them, recompute the
reference indices in the face data for each group, and then write
all this junk out with a bunch of surrounding boiler plate ASCII which
the fbx format requires. I'm just starting to code what I
just described but it'll probably be a slow process.

[Andromachus Theodoulos]

@KE,

First I want to say... take your time!!! If its a slow process, I am sure it will be worth it. I can certainly wait.

Anyway, to get a feel of what is developing, are we still looking at something that could be used with MilkShape?

I just want to say, you all have had me on the edge of my seat for the last several days, I have been following this thread closely. This would constitute one of the biggest breakthroughs for me personally if I am able to have access to an "affordable" modification tool for the meshes... I might even have something of value to post for others to download... which has not been true for my modifications in the past.

Keep up the good work!!! I am cheering you guys on!

@Caliban...
Thanks for your help on this and your gentle guidance...

AT

[KnightErrant]

Hi AT

GrumpyOldMan said he was doing the Milkshape binary format .ms3d,
so yes and this is probably the best path to success. I'm just looking
at fbx as an alternative just for a backup and because it's just interesting
in and of itself. 3dsmax doesn't support .ms3d but since Milkshape can
export fbx there's only one extra step to get meshes to the 3dsmax
modellers. GOM said something like "it's easy to take apart a clock, it's
the reassembly that's the killer" which refers to going the other way,
turning stuff back into a mesh file with all those funny header/trailer
byte blocks.

I've got my fingers crossed that he got info from Caliban that
makes that doable.

[Caliban]

Do you have any more information concerning the mystery block(s)?
Can we really ignore it or is it in fact some other type of information?

Engine guy is still on holidays, I'll prod him again next time I see him. My guess is that they are unused and can probably be ignored. A test should be able to tell but I'll find out for sure.

@Caliban...
Thanks for your help on this and your gentle guidance...

Aww shucks

[KnightErrant]

Thanks Caliban! Casuir has reported a test a few
posts back, where he removed the mystery blocks and changed
the 3 to a 0 in the header/footer line above where they appeared,
and suffered no ill effects with the game. I'll assume they
can be safely ignored.

Casuir also reported that there is at least one variant mesh format
where the vertex bone assignment block follows the mystery block
rather than proceeds it. I've only partially found all the changes but
the very first header/footer line is different between "regular" units
and these variants so I think it can be programmed around. Is this a
granny thing or just different ways that your guys assemble the
mesh files? Just wondering if we should be on the lookout for other
variant type meshes. (It's the az_body_suit types where he says
most of these variants are.)