Need help with descr_skeleton.txt

[SigniferOne]

This will be a time consuming process, so any help or individual attempts to do the same thing are welcome. That way results can be verified with more than one example.

[Bwian]

Grrr...this is turning into catch 22.....

I have tried removing lots of skeleton references, but all Iget is CTD's. There is no error message given to hint at what it is missing. This means I can't tell if I have done something silly when I chopped great lumps out of the file, or that there is something more fundamental here.

I believe the game is crashing purely because the skeleton DAT and IDX files are not there. It 'should' then use the text file and extracted animations....but it doesn't seem to.

If we created a 'dummy' set of files, we could test it... but I can't get V's tool to actually delete files!!!

Presumably, if it HAS to have the stock files, we should be able to put a dummy file with EVERYTHING V's tool extracts...removed. Then...though, there is no guarantee the text file will be doing anyhthing.....

[Bwian]

double post....

[SigniferOne]

I have the same problem. Let's wait for Jerome on Monday. :) There's probably one last little thing we gotta do, that we don't know about yet.

[Vercingetorix]

There is a command option "-animdb". I'm sure it does something related to what we are doing. From looking at the error messages it generates you can get some idea of what it does but the actual usage is unknown.

[Myrddraal]

Jerome?

[Bwian]

I tried taking out animations from the pack and replacing them with extracted copies of the same files. I moved out about a dozen 'Testudo' animations and deleted them from the animation archive.

Game then CTD's

Now..If you create new animations and add them to the file, it works OK. Or, at least, I presume so. I haven't tried the new Hoplite animations....but I assume they worked properly

Anyway....this leads me to some assumptions:

1) If you can create working animations which did not exist in the packs then there is nothing extra that is required to make a working animation

2) If you REMOVE things from the packs, the game does not work EVEN if you have provided the content you removed outside of the packs

3) Remove the packs and the game CTDs

Conclusion:

The game ALWAYS reads the pack files, and checks their contents somehow. This may be looking for a check in the file, or it may be comparing the contents against the descr_skeleton file ...or SOMETHING.

As soon as you tamper with the pack files, the game won't start up.

[SigniferOne]

But how do you know you've extracted everything from the packs? If not for Jerome, we wouldn't know there were EVT files in them.

[Tux]

Oh and also one thing if you change the file with a hex editor and add more information the game will ctd also. And this would be usefull also for creating new sekeletons.

[Bwian]

But how do you know you've extracted everything from the packs? If not for Jerome, we wouldn't know there were EVT files in them.

My assumption is based on the contents of the descr_skeleton file. If you look at the things it calls for, they are only .cas or .evt files, as far as I can see.

If this file 'takes over' when the animation and skeleton archives are removed, then the only files it would look for are the ones it lists.

Now...if there was something 'else' in the files that we need, how come the game crashes when I remove a couple of animation files from the Pack.dat / idx files. Surely, the game would run OK until some unit or other tried to use those animations UNLESS there was a check function at startup that verifies the presence of the animations. If this process ONLY looks at the compressed archives and NEVER at the file structure outside of the archives, then that would explain what we see.

I assume ( though I could never make it work ... ) that it is OK to add NEW animations to the archive without the game CTDing. This implies that the extra items are not on the game's internal 'checklist' but since it found what it was expecting to find, it was OK about it.

the BIG question now, though, is WHERE is the check made and how do we stop it!

I think this can only be answered from the inside

Also...this drags up one or two other realted issues that have been troubling the old grey matter....

If...for example.... I wanted to make a '4 armed' creature...

1) I would need a new mesh ( easy .... know how to do it )
2) I would need a new skeleton with some extra bones. These would need to be linked and setup in Max ( not so easy..but know how to do it )
3) I would need new animations. 20 frame limit, move the parts in max frame by frame and save the result as a CAS file. ( never done it..but it should be do-able!)

WHAT NEXT...
1) Stick model and texture in usual place. Easy
2) Now...there is no way I know of to load this skeleton into the DAT and IDX file. It can't be anywhere else.
3) Should be possible to add the new anims to the animation folder, or add them to the archive without creating a CTD.

Linking it all together.... the unit can be directed to use the new skeleton...assuming I had got it into the archive..
The animations SHOULD be readable...if there was any way of making the game read the descr_skeleton file...RATHER THAN LOOK FOR THIS DATA ELSEWHERE, or of editing that data which MUST live in the dat or idx file archives

Otherwise, you never actually over-ride the skeleton data held in the skeleton.dat / idx file and your skeleton will not be found. It also forces ALL skeletons in the game to follow the same basic patterns and proportions.

Arrgh....this is so close...and yet so far away from making it possible to REALLY change around this game and bring in some truly exotic untis.

[Bwian]

Ohh...er....another thought occurs... ( happens sometimes with me )

I was deleting BOTH pack files and BOTH skeleton files. Now...As far as I can gather, V's tool is extracting the animations from PACK.DAT and PACK.IDX but I don't recall that it extracts anything from SKELETONS.dat or idx.

The tool allows you to EDIT those archives, but does not take anything out of them. At least....that is how it looks to me.

What form are the SKELETONS actually stored in?

Teh animation CAS files are just giving instructions to the skeleton structure in the model file, but something else is clearly inside the SKELETON files. It has to be holding more than just a copy of descr_skeleton and what it does hold has to be linked to both the CTD issue and the fact that no matter what changes you make to the skeleton in the model CAS or the animation CAS files, only the changes made to scale in the DAT actually have an effect.

[Vercingetorix]

Skeleton.idx and Skeleton.dat is simply descr_skeleton.txt in a binary format. Pack.dat contains only the animation .cas files not evt files. I don't think that there is a check going on because if you delete descr_skeleton.txt (and you have the .dat/idx of course) the game runs fine. It's funny why this would not work exactly like the events.dat and the descr_sound* files. In the case of sounds you delete events.dat/idx and it parses the sound txt files, I'm not sure why it doesn't work the same with animations.

The error messages of '-animdb' from the exe:

AnimDB: Could not build skeleton animation for anim '%s' in '%s' skeleton
AnimDB: Could not read parameters of anim '%s' in '%s' skeleton
AnimDB: Could not load - '%s'
AnimDB: Unknown animation - '%s'
AnimDB: %s - %i skeletons %i anims

For those not familar with sprintf, %s means string and %i a number...

[Bwian]

Aww.....

So, from what V is saying.....we have everything we need ..IN THEORY, but in practice, the game just wants the files and isn't doing what it ought to do.

We have the animation CAS files
We have the EVT files
We have the skeleton dat / idx in the form of the descr_skeleton.txt

But the game CTD's because it is still looking for the archives, rather than just what they contain.

If that is true...there really isn't a way round this short of a patch from CA.

Mind you...all this doesn't really explain my '4 armed man' scenario. When I was trying to make a new skeleton for large creatures, I tried every permutation of change to animation cas files etc. but never managed to get a REALLY new skeleton. I tried to move the pivot points on both the game model AND the animation files, but never got a change that took effect.

[Vercingetorix]

Yea the actual skeleton bone positions are in skeletons.dat

[Bwian]

Ahh...thanks Vercingetorix :)

THATS what I need to change! I know the data for this is not part of the text file, since that just seems to provide a list of animations to use.

Any chance of being able to load up a completely new skeleton with new bone positions through your excellent editor? In truth, this is the only thing we CAN'T do right now

[SigniferOne]

In truth, it'd be more important if we could change things other than mere skeletons. For example, as I said before, there's a lot of fascinating capacity written in descr_skeleton that we simply cannot do right now through mere skeleton replacement. If we could get access to that, it'd be a miracle that lots of people could take advantage of.

[Bwian]

Ahh...'mere' skeleton replacement is rather understating what I want to do. At the moment, we have a set and finite number of skeletons. They are:

1) Human
2) Horse
3) Dog
4) Elephant

Thats all. Each has it's proportions rigidly fixed and immutable. If you wanted to make a barbarian with a broader chest...well...you can't. I have tried, but the fixed nature of the shoulder pivots means you are stuck. All ofthe 'warrior' models are teh same stature as the peasants. Thats Ok for ordinary troops...but for Heroes...well...I want some beefier General models, and I can't make tehm work right. All you an do is add a scaling factor. Take a look at the Middle Earth mod....it's Dwarves are a classic example of why I want to be able to do this. They don't look like short warriors...they look like children.

Then, of course, there are the non-humans. My Centaurs work fine until you kill them. Then you get bits of horse and body. Hit them hard and you get bodies popping around like popcorn! I can't make a centaur skeleton.

I also can't make dragons, hydra, cerberus ( with more than 1 moving head ) giant scorpions, trolls that look like I think they should, cyclops, Minotaurs etc. They end up as just 'bigger or smaller human shaped creatures' and I am unhappy with the results. I have put the 'Mythology Mod' on the shelf until this is cured.... so to me new skeletons are THE most important thing we could add to the game. We can already add new animations...we are just tied to the stock skeletons.

[SigniferOne]

I don't know what you mean. You can go throuigh a few animations and change how broad the guy's chest looks, and it will look broader. You don't even have to change that many, just a few idle animations ones when the guy is stationary enough for people are likely to pay attention. During things like dodging or blocking it won't matter much anyway because the body is in motion and proportions are temporarily distorted, as it happens in real life. That's the solution I've adopted.

[Bwian]

hmmm....have you actually got that to work ? Every time I tried to change that on a skeleton, it ignored my change and my unit was unchanged.

Have you got something that actually has that sort of change in it that you could send me ?

I tried moving pivot points a LOT before V's animation editor allowed scaling of skeletons to work. Try moving the shoulder pivots out to double the shoulder width, and see if the change shows up in game. As I understood it, the animation controlled the rotational changes made to the joints, but not the joint locations. This was in the SKELETON.DAT file, which is unaffected by animation changes. Any translational or scale changes made in teh animation CAS files, or the model CAS file were ignored. SOMETHING over-rides them.

This is doubly puzzling, though, if th DAT files is just a binary version of the descr_skeleton.txt file.

Perhaps someone in the know could confirm for me:

1) WHAT defines the position of each joint .... model file skeleton, skeleton dat, skeleton idx, animation CAS ????

2) What is in SKELETON.DAT
3) What is in SKELETON.IDX
4) What is in PACK.DAT
5) What is in PACK.IDX

I think the confusion currently inducing my brain to melt and dribble out of my ears is largely down to the basic lack of understanding of those 5 questions. I thought I understood it..but clearly I don't!

Anyway...important as this clarification is to my mental wellbeing, it doesn't bring us any closer to running a game without the files in Q2-Q4.....

[Tux]

So this is what i know but i'm nor 100% sure:
In the idx file are the list of animations and skeletons.
In the PACK.DAT are the animations.
In the SKELETON.DAT are defined the skeletons.

[SigniferOne]

The animation controls everything - the joint locations, rotations, the whole bit. Besides, changing the joint rotation itself necessarily affects the location of joints that depend on it.

Yes I have gotten it to work. You don't make the changes in the Model, you make the changes in the particular animations, where you move the shoulder joints to where you want them to be for that frame, and presto. How do you think animations happen at all? Shoulder joints, among others, move! It doesn't matter what the original starting point was, and you can make it whatever you want.

You just have to make sure 3dmax is in animation mode when you affect the joint locations.

[Bwian]

I will test this.

I know how animations work .... done plenty elsewhere. Just things don't seem to fit the logic I am used to. I will experiment with this later on. As far as I was aware, the skeleton is defined in the model CAS, where you set the vertex weightings etc.

The animation file was providing the various skeletal rotations, and instructing each joint how to rotate. You never need to 'translate' any bones, as such, since the bones automatically affect those lower done the 'skeleton'. I tried doing this sort of thing before, but got no luck. Hopefully.... I just messed up a bit, and failed to get the animation to work properly.

[SigniferOne]

Just import a model, then import an animation that can work with that model (ie. no horse animations for soldiers). Then affect the keyframes for the shoulder bones in such a way as to have them visibly different from before, and make sure this difference stays for the entire animation, i.e. not just for that frame. Often that means changing the first keyframe the way you want it to be, copying it into the last frame, and deleting everything in-between. Then after the new improvement is visibly evident, and is persistent throughout the entire animation, export the animation file, update it within the packs using V's program, and you should see it in the game (provided you know which skeleton, and thus which soldier, uses that animation).

A word of caution though, V's program still has a few glitches to iron out in terms of exporting, so in a few rare exceptions the exported animation will not show up properly. But other than that, I guarantee you this will usually work, as I have done many times. That's how I've been creating my hoplite animations, archer animations, etc.

Jerome... are you still with us :) We are almost there, but we still need your help!

[Vercingetorix]

pack.dat/idx : .cas animation files.
skeleton.dat/idx: skeleton definitions which include everything in descr_skeleton as well as the bone positions and hierarchy.

.cas animation files: quaternion rotations and the position of the root bone through time, otherwise no other bone positions.

[Bwian]

Have to say...my experiments are tending to show no change....

Dsyrow1....can you take a standard animation, reposition the bone pivots so that the shoulders are wider and the arms are roughly twice normal length and send it over ( once you have checked it has worked in game )

I cannot make this happen. If you can, having a model to work from would be REALLY helpful.

My tests have all resulted in absolutely no change to the dimensions of the human normal. This agrees with V's definition of the DAT file, where he is stating that the Skeleton.DAT file holds the bone positions and hierarchy.

The animations DO NOT define the positions of the bones base location in the 'crucified' posture, and their hierarchy. Sing along....the leg bones connected to the knee bone.....

Now....if that is true ( and I have every reason to believe V knows whats inside the DAT ) it would follow that we could NOT run the game without the DAT file. The descr_skeleton text file doesn't contain the skeleton hierarchy OR the base bone positions. It just controls the way the animations are assigned to game actions.

In order to make new skeletons, we would need to be able to load up new skeleton definitions and base positions / hierarchy. At present we can only copy the existing ones and assign new animations to them.

Sorry to say...but every experiment I have done on this backs this up. I would LOVE to be wrong....PLEASE prove me wrong....

[Duke John]

Jerome
Are the EVT files you provided from before the 1.2 patch? The 1.2 patch included new skeleton and pack files so the CTD when using the evt files might be caused by that.

The 1.2 patch introduced units shielding themselves during marching from missile fire. Is there a new entry (EVT or animation) that deals with this? If so and the EVT files are pre-1.2 patch then the engine tries to find those files, fails to, and CTD.

Edit: I tried it with a 1.1 installation but it didn't work, although it was modded, I don't think there was any confilicts in animation.

All
Has anyone been able to import new animations into the game? It works with fs_slinger_new, but I tried fs_spearman and it CTDs when loading the game. My method:

1. extract animations with xidx
2. use AnimEditor to make new skeleton based on fs_spearman, answer n so that animations are copied.
- When copying the files to R:TW it works, but I haven't added new animations yet.
3. When looking into data/animations/new_skeleton there are no animations, so
4. extract the animations again with xidx
5. instead of replacing animations I just update the pack with the bat file.
6. copy the files to R:TW and C:TD.

Thanks for any hints or help,
Rob

[JeromeGrasdyke]

Are the EVT files you provided from before the 1.2 patch? The 1.2 patch included new skeleton and pack files so the CTD when using the evt files might be caused by that.

Yes, the .evt files are from the 1.2 patch, and should contain everything you need. The descr_skeleton.txt parser should flag anything that's missing. On a side note, the skeleton base pose (Vercinge's bone positions) is normally taken from the first animation which is added to a skeleton, so it's not necessary to supply this information seperately. It is important to make sure that the transforms in an animation are all relative to the same basepose - we did this by making sure that in Max the animations all contain a single frame at the front of the animation which contains the model basepose.

As to why people are still seeing so many problems: the 1.2 patch build has been archived, so it's a little hard for me to get at right now, but I'm looking into it.

[Bwian]

I think most of our problems stem from us groping blindly in the dark

Where we are all ( and I think it is all of us ) getting stuck is in simply getting the game to run with the archived animations and skeletons all removed. There is something happening here that we don't understand.

Your outline of the process was quite simple and straightforward....but does assume that the tools we are using work as well as the tools you built the game with. It is possible that Vercingetorix's animation unpacker misses something, but we have no real way of knowing.

It would ( I assume ) be a bit beyond the pale to ask CA to release the tools they used to archive / unarchive these files since there are bound to be copyright issues with such things, but I wonder if there is any way that someone who worked with it could run up a list of what should come OUT of the archives when we empty them out. I couldn't see anything listed in the descr_skeleton.txt file that wasn't there....but then...there are a lot of files, and I could easily have missed something.

The other animation infor that has come out of this thread has proved useful to me despite the basic problems with the process.

From Jerome's last post.....

The first pose in Max was always the base pose ( crucified posture ). Presumably, this was at 'frame 0'...with the first actual frame of animation being generated by moving the bones FROM that posture into the first position you want the actual animation to start from. This would store teh translations and rotations that make up the first frame as movements from 'base'. Logical....for predicatable response.

I am still curious as to what actually fixes the relative positions of the bones within the skeleton framework. When I have tried to make new animations, I have been able to rotate bone joints and see the effects..but I have been unable to TRANSLATE a joint. This implies a rigid bone structure that was defined somewhere other than the animation or base model skeletons. From what Vercingetorix has said, this data is stored in the skeletons.dat file. When we extract animations and delete the dat, the game should be reading the descr_skeleton text file..

.. but the game CTD's.

This implies 'something' missing when the game parses the files. It's not the EVT files. We can be sure of that. I don't think it's the animation files....since we don't get an error. Every time I messed up an animation file before, I would get an error that said it was 'missing' such and such a file. No error message here at all.

So..either it crashes because it HAS to have these archived files and isn't looking for an alternative....

or it crashes because something else is missing. Something other than the descr_skeleton.txt, evt files and animation CAS files.

Jerome...I think you could be or only hope of getting to the bottom of this

[SigniferOne]

As to why people are still seeing so many problems: the 1.2 patch build has been archived, so it's a little hard for me to get at right now, but I'm looking into it.

Thanks a lot Jerome!! We'll all be hopeful and waiting :)

EDIT: Here's a question: what determines how often a unit enters one of the Idle animations? Is there some hardcoded timer, some soft (text) coded timer, or is it simply determined by the number of idles, wherein a unit with 20 idles will enter one of those animations more frequently than a unit with 6 idles?

[Tux]

Maybe the descr_skeleton.txt wasn't updated in the patch and since there were done some changes to the animations we need descr_skeleton.txt for 1.2 patch.