Settlements: File Format Investigations
I started this topic is the tutorials and tools section but thought since it looks like a long-term thing it might be better to make it a Wiki thread. I''ll quote Alpaca's comments here so this is self-contained and then add my latest results. No tools yet, this is still just research. First quote:
Quote:
Well from what I remember: The first step is to create so-called blocks. These are parts of buildings and directly modeled (and I guess in mesh format). These blocks are then merged to create buildings (for the tech-tree or ambient buildings) or whole cities (for the walls and fixed buildings) in 3dsmax and exported to the world files. However, I don't know what the worldpackage exactly consists of, or how the animations are done. The different files in the directories give us some hints: 1. The terrain is probably independent from the actual buildings 2. So is vegetation 3. Animations are somehow store in the animinstances file. The small size of this suggests it's only transition info 4. Pathfinding is also exported to a distinct file 5. Collision information is probably a representation of the actual world file with lower details. How Caliban described it to me, I believe it could be that the relative positions are stored in these files and not absolute ones. This means you have a lot of blocks (like the towers which are only the highest part of the actual building), give the engine the alignment info and granny puts the pieces together. This would fit with the idea that terrain 3d info is in another file, and buildings are automatically snapped to the ground. It could be that this is determined on "compile-time" though It's not much, I know Oh, it also seems these files aren't described in any text file, so for the more generic ones like tech tree buildings or ambient buildings there seems to be some slot-in mechanism (the game seems to scan the whole folders). Some of this info seems to be in the worldpkgdesc, but I'm not 100% sure about that. __________________
Second quote:
Quote:
That's already some good success I also remembered Caliban told me that in the dev version, the blocks that are used to piece together the worlds are in the blockset directory, but we only have some .debugdata files there. They may be of some use to you anyways. I asked Caliban for example files, maybe we're lucky and he can give us one or two
Ok, now this is some stuff pulled from the northern european village .world file: Structure 1: tris and verts only as .obj file then pulled into Milkshape. https://img65.imageshack.us/img65/96...mplewf9.th.jpg ____________ Structure 2: this is called litter so I think this is just barrels and other stuff scattered around the village. https://img452.imageshack.us/img452/...uct2pf2.th.jpg Structure 3: another structure that has 3D form, i.e. it works up a hill side. Can we make Hobbiton with this? round doors and windows and maybe structure going into hillsides? https://img513.imageshack.us/img513/...uct3pv1.th.jpg Ok, end of data show, here's the file structure for the geeks among us (I of course mean GrumpyOldMan and Casuir for this kind of endeavor) Header:
Code:
serialization::archive 3 4 4 4 264 0 267 0 0 0 2 0 184.550750732 0 40 0 -7.09150409698 1.21208238602 -2.87348890305
The 40 is the count for the next weird stuff: 17 ints and 9 floats; what this is for I haven't a clue.
Code:
2 2 2 1 6 2 8 24 2 2 10 18 2 14 4 0 -1 -155.622634888 -51.4938278198 -151.834869385 148.574661255 46.8978500366 146.345230103 -53.2291908264 -44.9256057739 -49.011177063 2 2 2 -1 -1 2 2 36 2 2 -1 -1 2 3 37 1 -1 -155.622634888 -44.0137252808 -151.834869385 26.4544830322 38.9309844971 -2.73648524284 -76.298034668 -21.8567619324 -72.0800170898 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 0 -155.622634888 -44.0137252808 -139.663024902 -47.2217559814 38.9309844971 -39.1934700012 -30.1603469849 -21.8567619324 -72.0800170898 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 1 -52.0706977844 -37.63048172 -151.834869385 26.4544830322 33.2117424011 -34.6802749634 39.0461845398 47.3497695923 43.2641983032 2 2 2 28 20 2 -1 -1 2 2 27 5 2 -1 -1 1 -1 -0.44660949707 -18.9591369629 -8.98810577393 144.976104736 34.0708389282 135.085174561 62.1150283813 24.2809257507 66.3330383301 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 2 38.6354370117 -18.9591369629 17.2576904297 144.976104736 34.0708389282 135.085174561 -53.2291908264 -44.9256057739 43.2641983032 2 2 2 -1 -1 2 16 17 2 2 -1 -1 2 -1 7 1 -1 -136.959442139 -51.4938278198 -20.4864692688 -8.63206863403 44.6523666382 146.345230103 -30.1603469849 -21.8567619324 66.3330383301 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 3 -48.3864402771 -15.2977743149 55.7432060242 -8.63206863403 15.793211937 135.366821289 -53.2291908264 47.3497695923 -49.011177063 2 2 2 9 23 2 -1 -1 2 2 33 21 2 -1 -1 1 -1 -124.963951111 -14.9869804382 -126.477371216 1.60579538345 34.0556869507 -0.383612632751 -76.298034668 24.2809257507 -72.0800170898 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 4 -124.963951111 -14.9869804382 -84.6128387451 -80.0982208252 20.4758796692 -33.7329750061 39.0461845398 -44.9256057739 -49.011177063 2 2 2 -1 -1 2 12 35 2 2 -1 -1 2 11 13 1 -1 -7.26889801025 -46.9330062866 -144.168701172 148.574661255 46.8978500366 25.8397808075 62.1150283813 -21.8567619324 -72.0800170898 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 5 54.6254119873 -46.9330062866 -144.168701172 148.574661255 46.8978500366 -18.4172821045 15.9773406982 -21.8567619324 -72.0800170898 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 6 -4.75338363647 -37.7475395203 -118.283607483 72.5040283203 39.5098762512 -39.6816711426 62.1150283813 -21.8567619324 -25.9423332214 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 7 7.73614883423 -35.0194892883 -46.0703086853 126.949501038 35.0441703796 25.8397808075 39.0461845398 47.3497695923 -49.011177063 2 2 2 32 22 2 -1 -1 2 2 34 15 2 -1 -1 1 -1 -13.1791849136 -14.7797765732 -128.338973999 129.626464844 21.0392723083 8.39222812653 62.1150283813 24.2809257507 -25.9423332214 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 8 40.4518089294 -14.7797765732 -47.0732574463 129.626464844 21.0392723083 8.39222812653 -76.298034668 -21.8567619324 20.1953544617 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 9 -136.959442139 -32.4922447205 -20.4864692688 -69.3377151489 34.6142845154 46.620059967 -76.298034668 -21.8567619324 66.3330383301 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 10 -134.329345703 -51.4938278198 42.5441513062 -37.7052383423 44.6523666382 146.345230103 39.0461845398 -44.9256057739 43.2641983032 2 2 2 -1 -1 2 31 19 2 2 -1 -1 2 38 39 1 -1 -3.77192687988 -9.82592868805 -3.47510313988 58.5111312866 9.88546657562 122.70514679 15.9773406982 -21.8567619324 66.3330383301 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 11 -3.77192687988 -9.82592868805 54.2780914307 28.7844619751 9.88546657562 122.70514679 15.9773406982 24.2809257507 66.3330383301 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 12 -0.44660949707 -9.4465637207 33.6804428101 43.0900650024 32.8922309875 129.833618164 -30.1603469849 24.2809257507 -25.9423332214 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 13 -56.8189239502 -3.14870548248 -44.7135314941 1.60579538345 34.0556869507 -8.97021102905 15.9773406982 24.2809257507 -25.9423332214 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 14 -13.1791849136 -3.24935531616 -49.312210083 39.8192138672 18.9774055481 2.66998124123 -76.298034668 24.2809257507 -25.9423332214 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 15 -73.837600708 -3.30341291428 -38.8326568604 -47.7539901733 15.6267280579 -0.383612632751 -53.2291908264 47.3497695923 43.2641983032 2 2 2 25 30 2 -1 -1 2 2 26 29 2 -1 -1 1 -1 -121.161117554 -8.20901298523 -11.1202039719 -10.781373024 29.0430030823 129.798721313 -76.298034668 24.2809257507 20.1953544617 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 16 -121.161117554 -5.63294696808 -11.1202039719 -49.7004699707 29.0430030823 35.2571754456 -30.1603469849 24.2809257507 20.1953544617 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 17 -56.627620697 -0.55002450943 5.9689745903 -47.7831497192 9.99457645416 29.6839351654 62.1150283813 24.2809257507 20.1953544617 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 18 30.5515899658 -2.67515802383 -4.59002447128 123.303771973 24.4779167175 49.9851074219 15.9773406982 24.2809257507 20.1953544617 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 19 13.3730754852 -2.30725717545 -8.98810577393 41.1701126099 18.4326286316 46.3331718445 -30.1603469849 24.2809257507 66.3330383301 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 20 -54.6934204102 -6.82613229752 44.6657829285 -10.781373024 15.5162200928 129.798721313 -76.298034668 24.2809257507 66.3330383301 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 21 -110.793014526 -8.20901298523 43.826965332 -49.2373542786 15.5162191391 114.937850952 15.9773406982 -21.8567619324 20.1953544617 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 22 17.7234573364 -2.39971446991 -3.47510313988 37.4422836304 3.23146367073 12.2805767059 15.9773406982 24.2809257507 -72.0800170898 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 23 -9.27330303192 -4.94784832001 -119.308181763 16.5616779327 11.567240715 -84.5141601563 -30.1603469849 24.2809257507 -72.0800170898 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 24 -19.2961025238 -3.71061968803 -126.477371216 -6.31276941299 9.27271270752 -113.494033813 62.1150283813 24.2809257507 -72.0800170898 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 25 35.2527427673 -6.66765499115 -128.338973999 128.974838257 13.0476951599 -54.9774284363 15.9773406982 -21.8567619324 -25.9423332214 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 26 -7.26889801025 -0.946434378624 -47.5049209595 36.5158996582 2.35940933228 -8.68527603149 -76.298034668 -21.8567619324 -25.9423332214 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 27 -71.3251037598 -0.869940280914 -38.0507507324 -52.6898918152 2.75568437576 -2.73648524284 -30.1603469849 -21.8567619324 -25.9423332214 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 28 -50.4078445435 -0.917699098587 -38.0734367371 -46.6416664124 2.13272404671 -35.2981987 62.1150283813 -21.8567619324 20.1953544617 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 29 52.5570068359 -1.09718203545 39.8137130737 58.5111312866 2.27236127853 44.7806396484 62.1150283813 -21.8567619324 66.3330383301 2 2 2 -1 -1 2 -1 -1 2 2 -1 -1 2 -1 -1 2 30 44.1865997314 -1.00981736183 45.5382385254 48.242980957 1.33804678917 48.5266189575
Now comes 857 20-byte entries. I'm sure you'll be glad I don't actually include them all:
Code:
nstuff = 867 0 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 0 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 1 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 2 0 8 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 3 0 8 -4 15 0 -4 -97 4 -90 54 -55 1 -1 -1 -1 -1 -1 -1 4 0 8 -4 15 0 -4 -81 4 -90 54 -55 1 -1 -1 -1 -1 -1 -1 5 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 6 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 7 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 8 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 9 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 10 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 11 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 12 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 13 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 14 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 15 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 16 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 17 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 18 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1 19 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
The data proper now begins, meaning data that I can understand: Triangles, verts, and byte quads. No data is shown here, just the structure:
Code:
nend = 866 footer: 0 0 31 0 0 0 0 4 1 0 0 0 0 0 16 0 1 0 1 0 0 0 0 0 2 0 0 0 0 0 20 0 1 3 2 0 0 0 0 0 23 0 1 0 3 0 0 0 0 0 Triangle data blocks: num_groups = 15 header: 27 0 1 3 4 0 0 0 0 0 0 0 0 0 0 0 0 0 ntris = 128 footer: 0 0 0 0 0 0 0 0 0 26 0 1 0 5 0 0 0 27 0 4 0 0 0 header: 27 0 6 0 0 0 0 0 0 0 0 0 0 0 ntris = 90 footer: 0 0 0 0 0 26 0 7 0 0 0 27 0 6 0 0 0 header: 27 0 8 0 0 0 0 0 0 0 0 0 0 0 ntris = 402 footer: 0 0 0 0 0 26 0 9 0 0 0 27 0 8 0 0 0 header: 27 0 10 0 0 0 0 0 0 0 0 0 0 0 ntris = 215 footer: 0 0 0 0 0 26 0 11 0 0 0 27 0 10 0 0 0 header: 27 0 12 0 0 0 0 0 0 0 0 0 0 0 ntris = 360 footer: 0 0 0 0 0 26 0 13 0 0 0 27 0 12 0 0 0 header: 27 0 14 0 0 0 0 0 0 0 0 0 0 0 ntris = 133 footer: 0 0 0 0 0 26 0 15 0 0 0 27 0 14 0 0 0 header: 27 0 16 0 0 0 0 0 0 0 0 0 0 0 ntris = 256 footer: 0 0 0 0 0 26 0 17 0 0 0 27 0 16 0 0 0 header: 27 0 18 0 0 0 0 0 0 0 0 0 0 0 ntris = 145 footer: 0 0 0 0 0 26 0 19 0 0 0 27 0 18 0 0 0 header: 27 0 20 0 0 0 0 0 0 0 0 0 0 0 ntris = 309 footer: 0 0 0 0 0 26 0 21 0 0 0 27 0 20 0 0 0 header: 27 0 22 0 0 0 0 0 0 0 0 0 0 0 ntris = 12 footer: 0 0 0 0 0 26 0 23 0 0 0 27 0 22 0 0 0 header: 27 0 24 0 0 0 0 0 0 0 0 0 0 0 ntris = 12 footer: 0 0 0 0 0 26 0 25 0 0 0 27 0 24 0 0 0 header: 27 0 26 0 0 0 0 0 0 0 0 0 0 0 ntris = 12 footer: 0 0 0 0 0 26 0 27 0 0 0 27 0 26 0 0 0 header: 27 0 28 0 0 0 0 0 0 0 0 0 0 0 ntris = 12 footer: 0 0 0 0 0 26 0 29 0 0 0 27 0 28 0 0 0 header: 27 0 30 0 0 0 0 0 0 0 0 0 0 0 ntris = 17 footer: 0 0 0 0 0 26 0 31 0 0 0 27 0 30 0 0 0 header: 27 0 32 0 0 0 0 0 0 0 0 0 0 0 ntris = 244 footer: 0 0 0 0 0 26 0 33 0 0 0 27 0 32 0 0 0 Vertex data: header: 22 0 1 0 34 0 0 0 23 0 3 0 0 0 109 14 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 35 0 1 0 35 0 0 0 0 0 0 0 0 0 nverts = 3693 footer: 0 0 0 0 34 0 1 0 36 0 0 0 35 0 35 0 0 0 0 0 5 0 0 0 0 0 quad byte block 1: header: 40 0 1 0 37 0 0 0 8 0 0 0 0 0 footer: 0 0 0 0 39 0 1 0 38 0 0 0 40 0 37 0 0 0 quad byte block 2: header: 40 0 39 0 0 0 9 0 0 0 footer: 0 0 0 0 39 0 40 0 0 0 40 0 39 0 0 0 quad byte block 3: header: 40 0 41 0 0 0 3 0 0 0 footer: 0 0 0 0 39 0 42 0 0 0 40 0 41 0 0 0 quad byte block 4: header: 40 0 43 0 0 0 10 0 0 0 footer: 0 0 0 0 39 0 44 0 0 0 40 0 43 0 0 0 quad byte block 5: header: 40 0 45 0 0 0 4 0 0 0 footer: 0 0 0 0 39 0 46 0 0 0 40 0 45 0 0 0 Bounding sphere?, 8 zero bytes, 4 floats, then 8 zero bytes 0 0 0 0 0 0 0 0 -80.7888946533 -0.4720692635 -93.5925827026 +60.7462997437 0 0 0 0 0 0 0 0 footer: 19 0 1 0 47 0 0 0 20 0 2 0 0 0 20 0 48 0 0 0 Structure Number 2 header: 23 0 49 0 0 0 num_groups for second structure = 9 header: 27 0 50 0 0 0 0 0 0 0 0 0 0 0 ntris = 78 footer: 0 0 0 0 0 26 0 51 0 0 0 27 0 50 0 0 0 header: 27 0 52 0 0 0 0 0 0 0 0 0 0 0 ntris = 12 footer: 0 0 0 0 0 26 0 53 0 0 0 27 0 52 0 0 0 header: 27 0 54 0 0 0 0 0 0 0 0 0 0 0 ntris = 60 footer: 0 0 0 0 0 26 0 55 0 0 0 27 0 54 0 0 0 header: 27 0 56 0 0 0 0 0 0 0 0 0 0 0 ntris = 44 footer: 0 0 0 0 0 26 0 57 0 0 0 27 0 56 0 0 0 header: 27 0 58 0 0 0 0 0 0 0 0 0 0 0 ntris = 102 footer: 0 0 0 0 0 26 0 59 0 0 0 27 0 58 0 0 0 header: 27 0 60 0 0 0 0 0 0 0 0 0 0 0 ntris = 102 footer: 0 0 0 0 0 26 0 61 0 0 0 27 0 60 0 0 0 header: 27 0 62 0 0 0 0 0 0 0 0 0 0 0 ntris = 102 footer: 0 0 0 0 0 26 0 63 0 0 0 27 0 62 0 0 0 header: 27 0 64 0 0 0 0 0 0 0 0 0 0 0 ntris = 48 footer: 0 0 0 0 0 26 0 65 0 0 0 27 0 64 0 0 0 header: 27 0 66 0 0 0 0 0 0 0 0 0 0 0 ntris = 60 footer: 0 0 0 0 0 26 0 67 0 0 0 27 0 66 0 0 0 Vertex data: header: 22 0 68 0 0 0 23 0 49 0 0 0 139 2 0 0 0 0 0 0 1 0 0 0 35 0 69 0 0 0 0 0 0 0 footer: 0 0 0 0 34 0 70 0 0 0 35 0 69 0 0 0 5 0 0 0 quad byte block 1: header: 40 0 71 0 0 0 8 0 0 0 footer: 0 0 0 0 39 0 72 0 0 0 40 0 71 0 0 0 quad byte block 2: header: 40 0 73 0 0 0 9 0 0 0 footer: 0 0 0 0 39 0 74 0 0 0 40 0 73 0 0 0 quad byte block 3: header: 40 0 75 0 0 0 3 0 0 0 footer: 0 0 0 0 39 0 76 0 0 0 40 0 75 0 0 0 quad byte block 4: header: 40 0 77 0 0 0 10 0 0 0 footer: 0 0 0 0 39 0 78 0 0 0 40 0 77 0 0 0 quad byte block 5: header: 40 0 79 0 0 0 4 0 0 0 footer for quad block 5 0 0 0 0 39 0 80 0 0 0 40 0 79 0 0 0 0 0 0 0 -63.8147735596 -4.3073754311 -102.2621383667 +19.5136985779 0 0 0 0 footer: 19 0 81 0 0 0 20 0 48 0 0 0 0 0 2 0 0 0 0 0 49 0 1 4 82 0 0 0 12 buildinglod0 17 46 BlockSet/Textures/ambient_buildings_NE.texture 53 BlockSet/Textures/ambient_buildings_NE_normal.texture 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 1 0 0 0 3 0 0 0 18 192 63 0 4 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 2 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 48 0 1 0 83 0 0 0 49 0 82 0 0 0 49 0 84 0 0 0 18 buildinglitterlod0 17 40 BlockSet/Textures/NE_litter_pass.texture 47 BlockSet/Textures/NE_litter_pass_normal.texture 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 1 0 0 0 3 0 0 0 18 192 63 0 4 0 0 0 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 2 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 63 0 0 0 0 0 0 0 0 48 0 85 0 0 0 49 0 84 0 0 0 -80.7888946533 -0.4720692635 -93.5925827026 +60.7462997437 15 0 1 0 86 0 0 0 16 0 1 0 0 0 0 0 0 0 21 0 0 0 1 0 0 0 0 0 2 0 0 0 num_ints = 1854 96 2 0 0 2 0 0 0 15 0 0 0 0 0 0 0 1 0 0 0 2 0 0 0 3 0 0 0 9 0 0 0 10 0 0 0 49 0 0 0 50 0 0 0 51 0 0 0 52 0 0 0 53 0 0 0 54 0 0 0 55 0 0 0 91 0 0 0 93 0 0 0 1 0 0 0 16 0 87 0 0 0 3 0 0 0 20 0 88 0 0 0
Sorry, what a mess. This will take a while to get a handle on. Good news, hey its just a really awful .mesh file. Bad news, .mesh files are a bear. C'est la vie. Regards, KE PS: That came out awful, I reworked the HTML but is kept removing my end-of-lines in the code sections. I'll try to clean this up tomorrow for a proper post. Abject apologies for now:embarassed:
Re: Settlements: File Format Investigations
Holy Windmills and Brickwalls, Batman (err, I mean Knight Errant!!!):laugh4:
You got buildings in Milkshape, that is awesome,
I have been in the Outremer a while since the kiddos got out of school for the summer, but I will be watching this one closely...
Great Work!!!!
AT
Re: Settlements: File Format Investigations
Hi AT! Long time no see, glad to see you back.
Well, everytime I preview this post my linefeeds get removed just
like last night so I'll do this in one go and cross my fingers.
Edit: Sigh! It came out sans linefeeds again. I edited it but it
looks ok in the editor. Saved it and it looked ok again. Then I
logged out and came back and the linefeeds are gone again.
Might have to do this offline and just post a pdf of it.
Header:
Code:
serialization::archive
3 4 4 4
264 0
267 0 0 0
2 0 184.550750732 0
40 0 -7.09150409698 1.21208238602 -2.87348890305
Red 40 is the count for the blocks of 17 ints and 9 floats as follows:
Then 867 20-byte blocks like these (I removed most of them for brevity):
Code:
nstuff = 867
0 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
0 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
1 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
2 0 8 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
3 0 8 -4 15 0 -4 -97 4 -90 54 -55 1 -1 -1 -1 -1 -1 -1
4 0 8 -4 15 0 -4 -81 4 -90 54 -55 1 -1 -1 -1 -1 -1 -1
5 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
6 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
7 0 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
.
.
.
857 27 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
858 28 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
859 28 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
860 28 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
861 29 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
862 29 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
863 30 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
864 30 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
865 30 4 -4 15 0 -4 -1 -1 -90 54 -55 1 -1 -1 -1 -1 -1 -1
So far I don't know what any of that stuff is for so please feel free
to speculate. Now we get to recognizable data. A granny footer
for the above and then triangle data. I've removed the actual
data just to show the structure:
Next vertex data:
Code:
header: 22 0 1 0 34 0 0 0 23 0 3 0 0 0 109 14 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 35 0 1 0 35 0 0 0 0 0 0 0 0 0
nverts = 3693
footer: 0 0 0 0 34 0 1 0 36 0 0 0 35 0 35 0 0 0 0 0 5 0 0 0 0 0
Next comes 5 byte quad blocks:
Code:
quad byte block 1:
header: 40 0 1 0 37 0 0 0 8 0 0 0 0 0
nquads = 3693
footer: 0 0 0 0 39 0 1 0 38 0 0 0 40 0 37 0 0 0
quad byte block 2:
header: 40 0 39 0 0 0 9 0 0 0
nquads = 3693
footer: 0 0 0 0 39 0 40 0 0 0 40 0 39 0 0 0
quad byte block 3:
header: 40 0 41 0 0 0 3 0 0 0
nquads = 3693
footer: 0 0 0 0 39 0 42 0 0 0 40 0 41 0 0 0
quad byte block 4:
header: 40 0 43 0 0 0 10 0 0 0
nquads = 3693
footer: 0 0 0 0 39 0 44 0 0 0 40 0 43 0 0 0
quad byte block 5:
header: 40 0 45 0 0 0 4 0 0 0
nquads = 3693
footer: 0 0 0 0 39 0 46 0 0 0 40 0 45 0 0 0
Those of you who remember the .mesh format will recognize the
3 0 0 0 int and the 10 0 0 0 int as being codes for cube normalized
RGB encoded vertex normals and tangents, respectively. Now we're
seeing 8 0 0 0, 9 0 0 0, and 4 0 0 0 codes. Caliban has kindly asked
his software guy and these are for:
8 - light indices in rgb, ambient occlusion in .a
9 - light attenuation, ambient occlusion in .a
4 - texture coordinates encoded as hi and lo bytes.
I'll add a little more to this in another post. Let's just finish the format
for now.
Code:
Bounding sphere?, 8 zero bytes, 4 floats, then 8 zero bytes, then footer
0 0 0 0 0 0 0 0
-80.7888946533 -0.4720692635 -93.5925827026 +60.7462997437
0 0 0 0 0 0 0 0
footer: 19 0 1 0 47 0 0 0 20 0 2 0 0 0 20 0 48 0 0 0
Then another complete structure follows with much the same layout.
Some of the granny strings are different, like for the first triangle
group and the vertex data. Let's skip listing this and go to the textures
and really big granny strings.
At this point we are about 200,000 bytes into a 1.5 Mbyte file so we are
only about 15% done. That's the first two structures: a village called
buildinglod0 and then "litter" called buildinglitterlod0, like
barrels and the like.
Re: Settlements: File Format Investigations
Alright, I can't post code until the HTML stuff is fixed but I would
like to return to lighting as promised:
Caliban's info had more information that I'd like to share, especially
on the lighting issue which I'm a little worried about and would
appreciate input from GOM and Casuir (or modellers in general)
to see if this is viable.
Quad block 8: light indices in rgb, ambient occlusion in .a
As given to me, is .a a reference to a structure element?
From the village.world file (not correct ne_village etc file)
these are mostly 0 0 0 byte number.entires BUT occasionally
they have 1 0 0 byte number entries.
Quad block 9: light attenuation, ambient occlusion in .a
These seem to be mostly byte number 0 0 0 entries.
What am I worried about? Where do you put these things
in Milkshape so they can be modified and put back in game?
Here's what Caliban's software guy sent along, I think this is telling
about how they use these constructs: I'm afraid to quote, given
what happened to my previous posts so I'll leave it in blank verse:
11 - we don't store binormals for settlements, in shader we get them by normal cross tangent - we can do it as we don't do texture mirroring on buildings (but we do on characters)
used by our static lights which are preevaluated during conversion. Each mesh can be effected by I think 69 static lights. Per vertex we store indicies to 3 of them and attenuation (intensity) of these 3. During real time we have a table of 69 colors, look up color based on index and multiply it by attenuation. This way we can have them flickering by changing color table. You can disable them by setting attenuations to 0, and ideally setting index to 0 so it doesn't run off the table.
Ambient occlusion - we normaly uesd the one from indicies. The one from attenuations is used only for animated blocks - we interpolate between these two as the animation progresses. So it's ambient occlusion of the block at the end of the animation.
4 - texture coordinates - stored as 4 bytes (color class, watch out for byte ordering), each coordinate is stored in 16 bits in fixed format, by looking at it it seems 2:14 format. (so -2 to 2 range, stored in bias form in 0..4 range so we don't have to store sign). This is code we use to decompress it back to floats (shader code, but should mostly compile in c++ if you replace types)
After this was some shader code which was requested to not be released
so we stop here.
Anyway, my problem is how do we recover this data which needs
to go back in the settlement file if we can't put it in Milkshape to modify it
but it still needs to be there? Showstopper? or am I just ignorant of
the solution>
Re: Settlements: File Format Investigations
Hmm what does occlusion mean in this context? Doesn't it mean to signify whether anything behind the mesh should be rendered or not? Could be an absorption coefficient or something.
Anyways I don't know the rest of the info so I'd rather shut up :laugh4:
As for the html code, it needs to be enclosed in [html] tags
Re: Settlements: File Format Investigations
Haven't had a chance to digest the information so I'm
also a little vague on what they mean by occlusion.
Re code blocks versus html block let me try that to see
if one works better than the other.
Code wrappers:
Code:
serialization::archive
3 4 4 4
264 0
267 0 0 0
2 0 184.550750732 0
40 0 -7.09150409698 1.21208238602 -2.87348890305
Trying html wrappers:
HTML Code:
serialization::archive
3 4 4 4
264 0
267 0 0 0
2 0 184.550750732 0
40 0 -7.09150409698 1.21208238602 -2.87348890305
Both looked ok in preview so I'll post and see how it looks.
Edit: Naw, it's still messed up either way. Am I the only one
who can't put tags around code that work?
Edit the second: Tried over at TWCenter and code tags work
ok there. Could someone try it here and if no go, reboot the
servers? Just a thought.
Re: Settlements: File Format Investigations
Hmm that's weird alright. It could have to do with the wiki thing; I'll ask Tosa what's up when I next see him.
Edit: Hmm
Code:
serialization::archive
3 4 4 4
264 0
267 0 0 0
2 0 184.550750732 0
40 0 -7.09150409698 1.21208238602 -2.87348890305
The problem seems to be particular to this thread... Might want to open another one lol
Re: Settlements: File Format Investigations
@Alpaca,
Thanks for the test I'll try here and if still borked try a new thread.
Doing a code block:
Code:
serialization::archive
3 4 4 4
264 0
267 0 0 0
2 0 184.550750732 0
40 0 -7.09150409698 1.21208238602 -2.87348890305
Edit: Looked OK, I'll log out and come back and see.
.
Edit the second: Naw, same thing, linefeeds are gone.
Re: Settlements: File Format Investigations
Holy cow, I didn't realize KE you were already on settlements! Please don't think your work is unneeded! I've been away from the TW scene a little bit, if the animations and model importing already got finished, then the settlements are the last frontier. You don't understand, if it were possible for me to bring over Roman buildings into the M2 ground engine (with its impassable terrain and gorgeous texturing)...
Re: Settlements: File Format Investigations
Hi SignifierOne,
I haven't given up on settlements; it's just that this is like the mesh
format times 91 (the number of independent meshes I've seen so far
in just the village world model.) The village.world file is 3.9 Mbytes
and I'm at 3.85 Mbytes which sounds like I'm almost done except that
the meshes seem to be over and there are a bunch of strings here.
Maybe a few more days then its like, ok, now I've read it, what now?
As a first cut, maybe just something that lets a user mod an existing
settlement (have to come up with new terminology, the format seems
to define complexes of multiple structures) but not add new ones.
I.e. store all the granny strings I can't make heads or tails of in a
template file and then recreate the .world file with modified meshes.
Really ugly, though, save a template and then write out 91 .ms3d
meshes for every independent village model. Still haven't even thought
about what to do with the lighting information. Present for every mesh
but can't be stored or manipulated in a .ms3d file.
Oh well, if it was easy, it wouldn't be fun!:laugh4:
Just going to try a quick posting experiment, if this doesn't work I'll
start over in a new thread like alpaca suggested when I get to the
end of the .world file:
Code:
serialization::archive
3 4 4 4
264 0
267 0 0 0
2 0 184.550750732 0
40 0 -7.09150409698 1.21208238602 -2.87348890305
KE
Edit: Still no line feeds, I'll start a new thread when I've got something.
Re: Settlements: File Format Investigations
Doesn't Milkshape support any lighting info?
Re: Settlements: File Format Investigations
Keep up the exceptional work KE. I agree with S1, if I can bring over the many greek and roman buildings as well as make new ones, that would be awesome.
Keep it up and don't give up, we are depending on your hard work and your genius.
Take care,
AT
Re: Settlements: File Format Investigations
KE, I was thinking about the next steps forward, how to make it easy for you to add creation of new settlements. Because after all, it would be the sheer epitome of ludicriosity to ask you to create a brand new GUI within which to move around buildings to places you'd like them on the map.
So, here's a solution that worked real nicely in RTW: we used 3dmax as that 3d GUI! I made the following script, according to Caliban's suggestions: you import all of your settlement's buildings into 3dmax, and place them where you want them to be. Then you press a button, and 3dmax exports their locations into a text file, which you simply copy and paste into settlement text files for the game. RTW settlements worked from text files, which was a nice touch; you'd have to export the buildings back into the world file when the button is pressed, taking the coordinates of the buildings from where they are within 3dmax or milkshape. I'm sure milkshape can give you coordinates of objects within it.
EDIT: Script was really simple, here's the link (pardon the description which was corrupted upon TWC moving to a new database, not allowing me to edit): http://www.twcenter.net/forums/downl...do=file&id=692
Re: Settlements: File Format Investigations
KE, if you can make sure you can get these settlements to work in MilkShape. If not I will have to live without altering settlements. (I can't afford 3dsmax, nor do I want to steal it). But to be fair to the 3dsmax folks, it would be good if this can work in both programs... A lot to ask I am sure.
Like S1 said, if the coordinates can be exported from MilkShape, Awesome!!!!
Keep up the good work... If I could I would treat you guys to dinner or something...:laugh4: Such a little reward for a great accomplishment.
AT
Re: Settlements: File Format Investigations
Hi Alpaca, SigniferOne, and AT,
My apologies, I've sort of dropped out of the forums the last week.
Not from lack of interest I assure you; more that the dimensions from
the evil plane keep intruding on this plane. I.e. the real world sucks
and I've got to deal with it. Anyway, I'm still stuck near the bottom of
the village file. The strings are gone but now I've hit some float blocks
that have varying length granny strings between them and it is taking so
much time to get through them. I'd like to ask if people could take a look
at them and see if they spot anything but I need to start a new thread
to see if I can post with line breaks working.
I'll do this: take a look for Settlements: File Formats, Take 2.
