Updated formatted modeldb file and associated syntax checker

[MikeV]

First off, thanks KnightErrant for all your hard work! Without your explanation, I doubt the rest of us could have figured out this file.

I don't know the significance of the deadly double space 0's so I deliberately
chose to break AFTER the double space and otherwise leave them strictly alone

As it turns out, these are very important. They represent a NULL value, which tells the code reading the file "nothing interesting here, move along." That observation, plus the following regular structure:

In outline form a unit definition entry would be:

# unitname
1 mesh count
mesh file section
faction count
body textures section
faction count
AttachmentSets section
rest of entry section
termination delimiter

... helped me decode the structure of the entire file!
The bad news: whoever wrote the parser for this file made it very naive and rigid (that is, as we've observed, very human un-friendly).
The good news: the regular structure should make it possible to write more tools to validate it -- less frustration for modders!

Please don't ask what the significance of these two different termination strings are, I haven't the foggiest notion. I thought at first it had something to do with the non-presence of AttachmentSets for mounts but elephant_crew and the captains, generals, and king units all have AttachmentSets textures so that's not it.

My guess is the last line contains timing information for the animations (all of this file's content addresses meshes, textures, and animations ...). If the line starts with -1, the other values are ignored. If the line starts with 16 (binary: 1111), the other values are floating-point numbers. Someone more familiar with the animation software can explain them.

I'm going to go over what I've learned, hopefully clearing up any remaining mystery about this file.

[MikeV]

The file data/unit_models/battle_models.modeldb (the "Model DB" file) is key to driving the animations of the little sim soldiers on the tactical battle display. For many (most?) of us Total War addicts, this is one of the cooler aspects of the game system -- and the focus of many modders' attention. So, understanding how to update this file correctly is critical to getting a mod to work.

I think I've finally cracked the code! In these posts, I'm going to present what I've learned, in the hopes that it will help reduce the frustrations of aspiring modders everywhere. That way, they can publish their results sooner, and we can all have more fun!

There are three key concepts for understanding this file:

1. Counted-length Lists (the number of items in the list, followed by the items themselves. A String is a List of characters).
2. Record structures (different Record types can have different structure).
3. Placeholders (NULL values, also known as 'nil' in some places).

With these, we can explain the whole file.

As KnightErrant explained, the file is structured. In fact, it is one long counted-length List of Records. Each of these top-level Records has the same structure:

1. Name
2. Number
3. List of mesh Records
4. Faction List of texture/sprite Records
5. (Parallel) faction List of texture/sprite Records for AttachmentSets
6. List of animation Records

Where things get a little confusing is that the last two sometimes omit values, and leave a "0 " (representing NULL) placeholder -- because the Record structure expects something there (even if it's "nothing").

Here's a short example:

16 albanian_cavalry 
1
4 
60 unit_models/_Units/LB_Plain_Fancy/albanian_cavalry_lod0.mesh 121 
60 unit_models/_Units/LB_Plain_Fancy/albanian_cavalry_lod1.mesh 900 
60 unit_models/_Units/LB_Plain_Fancy/albanian_cavalry_lod2.mesh 2500 
60 unit_models/_Units/LB_Plain_Fancy/albanian_cavalry_lod3.mesh 6400 
1 
4 merc 
71 unit_models/_Units/LB_Plain_Fancy/textures/LB_plain_fancy_mercs.texture 
72 unit_models/_Units/LB_Plain_Fancy/textures/LB_plain_fancy_normal.texture 
45 unit_sprites/merc_Albanian_Cavalry_sprite.spr 
1 
4 merc 
65 unit_models/AttachmentSets/Final European Light_merc_diff.texture 
65 unit_models/AttachmentSets/Final European Light_merc_norm.texture 
0  
1 
5 Horse 13 MTW2_HR_Spear 12 MTW2_HR_Mace 
1 21 MTW2_HR_spear_Primary 
1 17 MTW2_Mace_Primary 
16 -0.090000004 0 0 -0.34999999 0.80000001 0.60000002

What this means:

• Name: "albanian_cavalry"
• The 1 means ... something (height? scaling? mass?). For mounts, this is always 1.12.
• The mesh Record follows. It is a List of 4 Records, each always has 2 items:
  
  1. a mesh file path String (the path is relative to the data directory) and
  2. an Integer (the distance, in terrain units, at which to use the mesh)
• 1 faction texture/sprite Record follows, they always have 4 items:
  
  1. a String (the faction's name). In this case, it's Mercenary-only unit (faction merc).
  2. the faction-specific texture file path String (again, relative file paths)
  3. the "normal" texture file path String (also relative)
  4. 1 sprite file path String (also relative)
• 1 (parallel) faction texture/sprite Record follows, this time for the AttachmentSet(s).
  NOTE: the 3rd (sprite) path String is always NULL for the AttachmentSets.
• 1 animation Record follows, they always have 4 items:
  
  1. a Record containing 3 Strings naming the animations to use:
     
     1. Mount (or "none"). In this case, a Horse.
     2. Primary Weapon. In this case, a Horse Rider (HR_, Camel Riders are CR_, and Elephant Riders are ER_) Spear.
     3. Secondary Weapon (or NULL, if the unit has only a Primary). In this case, a Right-Handed Mace.
  2. a List of Primary Weapon animations (usually 2: offensive and shield). In this case, only the Spear -- no shield.
  3. a List of Secondary Weapon animations (if present, same structure as Primary). In this case, only the Mace -- no shield.
  4. animation timing settings (unless the line starts with -1, in which case the values will all be zero)

Once you realize that the non-zero numbers are just the length of the things to follow, you almost stop noticing them. When you understand the structure, and format the text accordingly, you can easily spot errors.

Then, you can focus on the important details: does the name of the (primary, secondary) animation match the corresponding name of the weapon? Do these agree with the values in the export_descr_unit.txt ("EDU") file? Do the named files exist at the right locations? etc.

I've uploaded a copy of the re-formatted file I'm using to: http://www.twcenter.net/forums/downl...o=file&id=1979

[MikeV]

Here's another example:

28 mercenary_pavise_crossbowmen 
1 4 
77 unit_models/_Units/LN_Brigandine_heavy/mercenary_pavise_crossbowmen_lod0.mesh 121 
77 unit_models/_Units/LN_Brigandine_heavy/mercenary_pavise_crossbowmen_lod1.mesh 900 
77 unit_models/_Units/LN_Brigandine_heavy/mercenary_pavise_crossbowmen_lod2.mesh 2500 
77 unit_models/_Units/LN_Brigandine_heavy/mercenary_pavise_crossbowmen_lod3.mesh 6400 
1 
4 merc 
76 unit_models/_Units/LN_Brigandine_heavy/textures/LN_ Brigandine_mercs.texture 
77 unit_models/_Units/LN_Brigandine_heavy/textures/LN_ Brigandine_normal.texture 
57 unit_sprites/merc_Mercenary_Pavise_Crossbowmen_sprite.spr 
1 
4 merc 
66 unit_models/AttachmentSets/Final European CB Gun_merc_diff.texture 
66 unit_models/AttachmentSets/Final European CB Gun_merc_norm.texture 
0  
1 
4 None 20 MTW2_Pavise_Crossbow 14 MTW2_Swordsman 
2 21 MTW2_Crossbow_Primary 14 fs_test_shield 
2 18 MTW2_Sword_Primary 14 fs_test_shield 
16 -0.090000004 0 0 -0.34999999 0.80000001 0.60000002

The mercenary_pavise_crossbowmen is a foot soldier, and so lists "None" for the Mount. It has 2 weapons (Crossbow, then Sword), and a shield (seems like all units with shield animations use the fs_test_shield. Has anyone made any others?).

Another example:

11 highlanders 
1 4 
54 unit_models/_Units/EN_Highlander/highlanders_lod0.mesh 121 
54 unit_models/_Units/EN_Highlander/highlanders_lod1.mesh 900 
54 unit_models/_Units/EN_Highlander/highlanders_lod2.mesh 2500 
54 unit_models/_Units/EN_Highlander/highlanders_lod3.mesh 6400 
2 
8 scotland 
72 unit_models/_Units/EN_Highlander/textures/en_highlander_scotland.texture 
70 unit_models/_Units/EN_Highlander/textures/en_highlander_normal.texture 
44 unit_sprites/scotland_Highlanders_sprite.spr 
5 slave 
70 unit_models/_Units/EN_Highlander/textures/en_highlander_rebels.texture 
70 unit_models/_Units/EN_Highlander/textures/en_highlander_normal.texture 
41 unit_sprites/slave_Highlanders_sprite.spr 
2 
8 scotland 
69 unit_models/AttachmentSets/Final European Light_scotland_diff.texture 
69 unit_models/AttachmentSets/Final European Light_scotland_norm.texture 
0  
5 slave 
66 unit_models/AttachmentSets/Final European Light_slave_diff.texture 
66 unit_models/AttachmentSets/Final European Light_slave_norm.texture 
0  
1 
4 None 14 MTW2_Fast_Mace 0  
2 17 MTW2_Mace_Primary 14 fs_test_shield 
0 
16 -0.090000004 0 0 -0.34999999 0.80000001 0.60000002

The highlanders is another foot unit, available to the faction scotland and as rebels (the slave faction). They only have a Primary weapon (a Mace), and a shield.

[MikeV]

A more interesting example:

27 mount_teutonic_barded_horse 
1.12 3 
60 unit_models/Mounts/Barded_Horse/mount_barded_horse_lod0.mesh 121 
60 unit_models/Mounts/Barded_Horse/mount_barded_horse_lod1.mesh 1225 
60 unit_models/Mounts/Barded_Horse/mount_barded_horse_lod2.mesh 10000 
1 
3 hre 
77 unit_models/Mounts/Barded_Horse/textures/horse_barding_teutonic_order.texture 
69 unit_models/Mounts/Barded_Horse/textures/horse_barding_normal.texture 
57 unit_sprites/teutonic_order_Mount_Barded_Horse_sprite.spr  
0 
1 
5 Horse 14 fs_heavy_horse 0  
0 
0 
-1 0 0 0 0 0 0

The mount_teutonic_barded_horse entry (available only the hre faction) is meant to be used from another entry in this file. Although the animation Record lists the mount name as "Horse" and Primary weapon as "fs_heavy_horse", is has neither a Primary nor a Secondary weapon animation. Note that the animation timing line starts with a -1 and has all zeros. Presumably, the code "knows" what to do with this situation.

Interestingly, even the upgraded Teutonic Knights unit only mentions a generic Horse as a mount:

20 teutonic_knights_ug2 
1 3 
65 unit_models/_Units/EN_Pplate_Plate/teutonic_knights_ug2_lod0.mesh 121 
65 unit_models/_Units/EN_Pplate_Plate/teutonic_knights_ug2_lod1.mesh 1225 
65 unit_models/_Units/EN_Pplate_Plate/teutonic_knights_ug2_lod2.mesh 6400 
1 
3 hre 
70 unit_models/_Units/EN_Pplate_Plate/textures/mtw2_EN_Pplate_hre.texture 
73 unit_models/_Units/EN_Pplate_Plate/textures/mtw2_EN_Pplate_normal.texture 
45 unit_sprites/hre_NE_Late_Bodyguard_sprite.spr 
1 
3 hre 
64 unit_models/AttachmentSets/Final Heater Special_hre_diff.texture 
64 unit_models/AttachmentSets/Final Heater Special_hre_norm.texture 
0  
1 
5 Horse 13 MTW2_HR_Lance 12 MTW2_HR_Mace 
2 21 MTW2_HR_Lance_Primary 14 fs_test_shield 
2 17 MTW2_Mace_Primary 14 fs_test_shield 
16 -0.090000004 0 0 -0.34999999 0.80000001 0.60000002

[MikeV]

How about siege units and their crews? Here's a Greek Trebuchet crew's definition:

20 greek_trebuchet_crew 
1 4 
70 unit_models/_Units/ES_Greek_Greek_Heavy/greek_trebuchet_crew_lod0.mesh 121 
70 unit_models/_Units/ES_Greek_Greek_Heavy/greek_trebuchet_crew_lod1.mesh 900 
70 unit_models/_Units/ES_Greek_Greek_Heavy/greek_trebuchet_crew_lod2.mesh 2500 
70 unit_models/_Units/ES_Greek_Greek_Heavy/greek_trebuchet_crew_lod3.mesh 6400 
1 
9 byzantium 
80 unit_models/_Units/ES_Greek_Greek_Heavy/textures/mtw2_es_greek_byzantium.texture 
77 unit_models/_Units/ES_Greek_Greek_Heavy/textures/mtw2_es_greek_normal.texture 
53 unit_sprites/byzantium_Greek_Ballista_Crew_sprite.spr 
1 
9 byzantium 
70 unit_models/AttachmentSets/Final European Light_byzantium_diff.texture 
70 unit_models/AttachmentSets/Final European Light_byzantium_norm.texture 
0  
1 
4 None 14 MTW2_Trebuchet 0  
1 18 MTW2_Knife_Primary 
0 
16 -0.090000004 0 0 -0.34999999 0.80000001 0.60000002

They have only a Knife for a Primary weapon animation, and no Secondary -- clearly, no match for a serious melee unit!

But note that their Primary weapon is listed as a Trebuchet. Hmm...

Over in the export_descr_units.txt ("EDU") file, this entry references the Greek_Trebuchet_Crew:

type             GR Trebuchet
dictionary       GR_Trebuchet      ; Trebuchet
category         siege
class            missile
voice_type       Heavy
banner faction   main_missile
banner holy      crusade
soldier          Greek_Trebuchet_Crew, 16, 2, 1
engine           trebuchet
attributes       sea_faring, can_withdraw, hardy, artillery
formation        1.5, 1.5, 3, 3, 3, square
stat_health      1, 0
stat_pri         7, 3, no, 0, 0, melee, artillery_mechanical, piercing, knife, 25, 1.8
;stat_pri_ex      0, 0, 0
stat_pri_attr    no
stat_sec         55, 3, trebuchet, 285, 30, siege_missile, artillery_mechanical, blunt, none, 25, 1
;stat_sec_ex      0, 0, 0
stat_sec_attr    ap, bp, area, launching
stat_ter         12, 3, cow_carcass, 215, 3, siege_missile, artillery_mechanical, blunt, none, 25, 1
;stat_ter_ex      0, 0, 0
stat_ter_attr    ap, bp, area, launching,
stat_pri_armour  2, 5, 0, leather
;stat_armour_ex   2, 0, 0, 0, 5, 0, 0, leather
stat_sec_armour  0, 0, flesh
stat_heat        1
stat_ground      0, 0, 0, 0
stat_mental      9, disciplined, highly_trained
stat_charge_dist 30
stat_fire_delay  0
stat_food        60, 300
stat_cost        1, 430, 250, 60, 140, 430, 1, 120
armour_ug_levels 1
armour_ug_models Greek_Trebuchet_Crew

The Trebuchet is one of the few EDU entries to have a 3rd (tertiary) weapon (the cow_carcass -- so much fun! ).

And we have this comment in that file about the stat_pri entry:

Details of unit's primary weapon. If the unit has a missile weapon it must be the primary

So, apparently, the "_crew" unit type is a special case.

[MikeV]

Here's another interesting example, the basis for General units:

16 northern_general 
1 4 
78 unit_models/_Generals_and_Captains/Northern_General/northern_general_lod0.mesh 121 
78 unit_models/_Generals_and_Captains/Northern_General/northern_general_lod1.mesh 900 
78 unit_models/_Generals_and_Captains/Northern_General/northern_general_lod2.mesh 2500 
78 unit_models/_Generals_and_Captains/Northern_General/northern_general_lod3.mesh 6400 
18 
7 england 
89 unit_models/_Generals_and_Captains/Northern_General/textures/late_general_england.texture 
88 unit_models/_Generals_and_Captains/Northern_General/textures/late_general_normal.texture 
0  
 ... 
18 
7 england 
61 unit_models/AttachmentSets/Final General_england_diff.texture 
61 unit_models/AttachmentSets/Final General_england_norm.texture 
0  
 ...
4 
4 None 15 MTW2_Non_Shield 0  1 18 MTW2_Sword_Primary 0 
5 horse 18 MTW2_HR_Non_Shield 0  1 18 MTW2_Sword_Primary 0 
8 elephant 18 MTW2_Elephant_Crew 0  1 18 MTW2_Sword_Primary 0 
5 camel 18 MTW2_HR_Non_Shield 0  1 18 MTW2_Sword_Primary 0 
-1 0 0 0 0 0 0

Note that there are no sprite files. And the animation section has a List of 4 entries -- one for dismounted, and one for each known type of mount. But no animation timing. Looks like another special case, or a stub of earlier work no longer used?

For comparison, here's the entry for a Northern European Bodyguard unit:

12 ne_bodyguard 
1 4 
56 unit_models/_Units/EN_Lmail_Hmail/ne_bodyguard_lod0.mesh 121 
56 unit_models/_Units/EN_Lmail_Hmail/ne_bodyguard_lod1.mesh 900 
56 unit_models/_Units/EN_Lmail_Hmail/ne_bodyguard_lod2.mesh 2500 
56 unit_models/_Units/EN_Lmail_Hmail/ne_bodyguard_lod3.mesh 6400 
10 
7 england 
73 unit_models/_Units/EN_Lmail_Hmail/textures/EN_Lmail_Hmail_england.texture 
72 unit_models/_Units/EN_Lmail_Hmail/textures/EN_Lmail_Hmail_normal.texture 
44 unit_sprites/england_NE_Bodyguard_sprite.spr 
 ...
7 england 
60 unit_models/AttachmentSets/Final Heater_england_diff.texture 
60 unit_models/AttachmentSets/Final Heater_england_norm.texture 
0  
 ...
1 
5 Horse 13 MTW2_HR_Lance 13 MTW2_HR_Sword 
2 21 MTW2_HR_Lance_Primary 14 fs_test_shield 
2 18 MTW2_Sword_Primary 14 fs_test_shield 
16 -0.090000004 0 0 -0.34999999 0.80000001 0.60000002

This time there's an entry for a sprite file, and sensible values for Primary and Secondary (Horse Rider) weapon animations.

[MikeV]

Finally, you should know that there are various "test" entries in the file. Here's one example, for a 2-handed Swordsman unit:

19 baltest_2hswordsman 
1 4 
63 unit_models/_Units/EN_Lmail_Hmail/baltest_2hswordsman_lod0.mesh 121 
63 unit_models/_Units/EN_Lmail_Hmail/baltest_2hswordsman_lod1.mesh 900 
63 unit_models/_Units/EN_Lmail_Hmail/baltest_2hswordsman_lod2.mesh 2500 
63 unit_models/_Units/EN_Lmail_Hmail/baltest_2hswordsman_lod3.mesh 6400 
2 
7 england 
73 unit_models/_Units/EN_Lmail_Hmail/textures/EN_Lmail_Hmail_england.texture 
72 unit_models/_Units/EN_Lmail_Hmail/textures/EN_Lmail_Hmail_normal.texture 
0  
6 france 
72 unit_models/_Units/EN_Lmail_Hmail/textures/EN_Lmail_Hmail_france.texture 
72 unit_models/_Units/EN_Lmail_Hmail/textures/EN_Lmail_Hmail_normal.texture 
0  
2 
7 england 
60 unit_models/AttachmentSets/Final Heater_england_diff.texture 
60 unit_models/AttachmentSets/Final Heater_england_norm.texture 
0  
6 france 
59 unit_models/AttachmentSets/Final Heater_france_diff.texture 
59 unit_models/AttachmentSets/Final Heater_france_norm.texture 
0  
1 
4 None 16 MTW2_2HSwordsman 0  
1 24 MTW2_2HSwordsman_Primary 
0 
16 -0.090000004 0 0 -0.34999999 0.80000001 0.60000002

It's useful as a starting "blank," adding your own factions, textures, etc. You'll need to add the right sprite file, though.

--
That's it, for now. If anyone deciphers other details of this file, please feel free to add your comments.

Happy modding ...