M:TW maps: imports and exports

[Kraellin]

ok, based on some short conversations with tosa and barocca, i decided to start looking at the file structure of the mtw maps and the possibility of altering existing ones (before the full game comes out with editor) and with importing existing stw/we/mi maps into the demo for use. like i say, tosa and barocca have both had some success in this already.

so, last night, i opened up the jaffa map and decided to see if i could tweak a couple things. the long and the short of it is that i couldnt....so far. it's been a while since i played with this stuff, but i believe these maps are using a checksum count for integrity, so i might have to use some other tools before i can alter existing mtw maps. just not sure yet...was only a cursory look last night.

however, i started this post for anyone and everyone to post data concerning the maps themselves. if you've tried stw/we/mi maps in the demo, post your results here, good or bad. if you can read and alter the file structure, post what you've found here. some of this may well become moot once we have the full game and editor, but let's start now anyways.

and a note to CA, if you havent already included a converter for making stw/we/mi maps playable in mtw, could you possibly do so? seems a shame to simply toss away all those old maps.

K.


------------------
The only absolute is that there are no absolutes.

[barocca]

DryRiverbed1 works
some minor graphical glitches
30x30 map

Ridges2 works
minor graphical glithces
30x30 map

===
by minor graphical glitches i mean some tiles are not correct,
no models/buildings,
some terrain elevations different,
but if i mention them here they are playable and not too weird


[This message has been edited by barocca (edited 08-07-2002).]

[Choco]

It would be great to have new maps from STW to use to do demos for medieval

The demo's maps are getting old quickly

[barocca]

I have been experimenting,

so far up to tile 86

tiles 4, 5 and 6 are the same between WEMI and Medieval,
all others are different
http://www.totalwar.org/ubb/frown.gif

[barocca]

and using tiles 4,5 and 6 - plus Trees which also work ...

Totomi, with a few more trees, to compensate for the tiles i had to delete,
http://doragonbarocca.homestead.com/files/mtw/totomi3.jjm

and Waterloo - minus the models,
http://doragonbarocca.homestead.com/files/mtw/waterloo3.jjm


both are best for arid or lush terrain

If you want trees for a rocky desert terain you have to place them manually, the tree tiles 4 and 5 showup as 'dirty' terrain in desert conditions,


Now if anyone wants to trawl through the 2 tiles sets and tell us which tile number to use from stw to get valid effects in mtw...

------------------
Clan Doragons Medieval Website
Mods, Unit Descriptions and more
DoragonBarocca of Clan Doragon (http://doragon.cjb.net)

[Kraellin]

i'm afraid i kinda gave up on this project, bar, after target's post. seemed easier to just wait for the editor and try and load and tweak the old maps in there.

K.


------------------
The only absolute is that there are no absolutes.

[barocca]

http://www.totalwar.org/ubb/smile.gif
they moved the release date back,
6 September here,
and i have serious Deja Vu'
(remember Warlords? - every time they got to 14 days before release they tacked 2 more weeks onto it...)

what i did was opened a map in stw editor,
then placed a row of consecutive numbered tiles - mark every 5th tile with a double,
saved the map.
did not close stw,
copy map to MTW, then edit a bdf to 'load' the map,
run mtw,
then i can tab back and forth and 'see' the differences,
my problem is i haven't made ANY maps, so it's not clear to me exactly what i am looking at,
but tiles 4, 5 and 6 are the same and the trees work,
http://www.totalwar.org/ubb/smile.gif
time is one thing i have in short supply at the moment,

if i could edit the tiles that are displayed for selection along the bottom in STW editor, but i can't find them...
(I was thinking of taking a screenshot and making a 'representaion' of the MTW tiles for the selection thingy in stw editor)

alternately someone who is far more familiar with the editor could write a list,

[Kraellin]

bar,

stw or we/mi? we do have two editors currently.

the textures for the we/mi editor are all in one large proprietary compressed file. i forget the name. at least, that's my best guess.

what i was going to do was write a program to convert the actual file on a more automatic basis. the file structure is fairly straight forward and shld be fairly easy to write a simple conversion program without ever having to load a map in the an editor. if you know where each texture is stored and what texture number it is for we/mi and then find the appropriate texture in mtw and it's corresponding number, you'd simply make a converter to change the we/mi numbers to mtw numbers.

the models, however, simply wont translate. they just arent the same thing. but, they would be fairly easy to erase from the file on an automatic basis, since they are in ascii form in the file. i suppose a few would translate over ok, in which case you could write a converting routine for that, but most are going to be lost.

K.


------------------
The only absolute is that there are no absolutes.

[barocca]

the wemi editor,

there are plenty of new textures in MTW,
and quite possibly some of the wemi textures were 'discarded',

(when i experiment as above, i get some interesting and unexpected results)

http://www.totalwar.org/ubb/frown.gif
why could they not have simply left the tiles as they were, and simply added new ones...

[Kraellin]

hehe, i aint touching that last line, bar ;)

oh, and tosa made a texture display map at one time. i forget the name, but that might help ya out.

i also see you've converted totomi and waterloo. nice :)

K.


------------------
The only absolute is that there are no absolutes.

[barocca]

WEMI Tiles 4, 5 and 6 work 100%
WEMI tile 7 has to be rotated 180 degrees to work,
WEMI tile 8 works 100%

So for tile 7, once you have placed the tile, simply rotate it 180 degrees and it will work nicely in mtw!

The following tiles WILL look weird when placed using the WEMI editor, the description is what MTW displays them as

WEMI tile 143 also works as a substitue for tile 8, giving a reduction in forest density,
note 143 is a 'blend' tile, meant to blend in with a dark rough grass texture across one corner, but as long as you block it with 5's, 6's and 7's the effect is fine.

WEMI 145 and 144 are also possible to replace 8, also giving a reduction in trees and the increase of dark rough grass texture giving an interesting forest floor effect.

(WEMI 145 lots of rough - sporadic tree density)
(WEMI 144 half rough - light tree density)
(WEMI 143 one corner rough - medium tree density)

waterloo and totomi will be updated again soon,

any requests for maps you'd like me to try?
I will edit in only the above tiles so far.
If I have more time i will try to work out more,
but the tile orientations are painfull to get correct.
B

[This message has been edited by barocca (edited 08-09-2002).]

[TosaInu]

Konnichiwa,

We need to be able to tell MTW jjm maps to use a custom set of folders. All that's required then are STW WE/MI textures in TGA format, the STW WE/MI models (already available) and a simple convertor that adds the custom paths in hex format to the STW WE/MI jjm.

Converting maps by hand takes too much time.
The proposed method also opens the road to real custom maps.

------------------
Ja mata
Toda MizuTosaInu
Daimyo Takiyama Shi

http://www.takiyama.cjb.net

[Kraellin]

bar,

you're getting me curious again. i may have to take up the gauntlet once again. good work :)

K.


------------------
The only absolute is that there are no absolutes.

[GilJaysmith]

I've got a tool that dumps out a .JJM file as text. If you're very nice then I'll give you the source.

IIRC the .JJM format didn't change from the Shogun v5 format, so, subject to my having forgotten something crucial here, you'll only have to deal with missing models and the different texture numbers. You probably know this already, but:
- Unrecognised models will just get dropped (I think!)
- The different texture numbers will have implications for cliffs - most likely they'll vanish - and forests - which will vanish, with new forests springing up on other textures.

Gil ~ CA

[barocca]

GilJaysmith,
the utility would be very handy,

can you send the utility itself?
(zip file format please)

If you can only send the source what would be required to compile it?

please send to barocca_x@hotmail.com

[Kraellin]

gil,

i'm assuming here that you dont just mean something like notepad which can display a .jjm in assci.

and yes, i'd appreciate this also. email is starfire@apex.net

K.


------------------
The only absolute is that there are no absolutes.

[GilJaysmith]

Quote Originally posted by barocca:
If you can only send the source what would be required to compile it?
[/QUOTE]

A C++ compiler. We use MSVC++ for everything at CA, but it should need only minor changes, if any, to build under other compilers.

But I'll send you the EXE too.

Gil ~ CA

[barocca]

Kraellin,

this is a sample of what i am working on,

http://doragonbarocca.homestead.com/files/mtw/Tiles1.jpg

The first row are from Warlords editor,
the second row are the corresponding tiles as displayed in mtw, they are taken from screenshots of a specially constructed map...(thats why a little blurry)

(up to tile 40 so far...)

and
GilJaysmith
http://www.totalwar.org/ubb/smile.gif
that would be very neat http://www.totalwar.org/ubb/smile.gif - i'll keep checking my mail..
http://www.totalwar.org/ubb/smile.gif


[This message has been edited by barocca (edited 08-10-2002).]

[TosaInu]

Konnichiwa Gil Jaysmith sama,

Please enlighten me here, what's the use of having the jjm file in txt format?

Is it possible to change the txt and then convert it to a working jjm file again?

Barocca san, very nice. But as you can see some textures are totally different. If you want STW maps to be used with MTW you've to remake them. A tedious job, making a tool to do that requires quite some skill and is limited to only STW maps. While the method described above, also allows custom texture directories.

------------------
Ja mata
Toda MizuTosaInu
Daimyo Takiyama Shi

http://www.takiyama.cjb.net

[barocca]

Konnichiwa Tosa san,

sadly you are correct,
however we have no means at our disposal to implement your directory control idea,
also we have no tool available to convert maps,
therefore we must convert them by hand and/or make 'fresh' maps from scratch.

Tiles 1 and 3 in stw/wemi produce a tile (in mtw) very similar to stw tile 35, tile 1 is a little bit 'dirtier than tile 3.
neither mtw1 or mtw3 match stw35 exactly!
NOTE many of the stw/wemi tile have no exact match in mtw that i can 'see', a converter will have to do a 'best guess'.
In order for anyone to write such a converter they will need a visual guide, in order to 'code' the tile changes.
Hence my current project.
http://www.totalwar.org/ubb/smile.gif

Knowing what tile in stw/wemi will produce what terrain in mtw will allow (i hope) more creativity in maps for use with the demo,

Trying to create or convert maps and remembering which tiles produce what from memory is very error prone, being able to 'see' tiles and default orintation will at the very least make life easier for me to convert maps.

While i acknowledge your idea for directory control is far superior, and that the release may be mere weeks away, I admit to wanting a little more variety in maps for custom battles.

ja mata
B.

[TosaInu]

Konnichiwa Barocca san,

I agree that your method is the best, and only, short term solution.

The directory control is what MTW maps should have, only the developers can make this. A game like UT has maps that control custom textures, models and sounds. Red Alert maps allowed scripting.

The special map Kraellin mentioned is simply 'iron bord' having all STW textures (old STW). You can load this map (error message pops up but you can click this away) and see all textures at once (grouped in series of 5).
Unfortunately loading green like maps in MTW fails. I believe you know a workaround for that?

Good luck.

Edit:http://www.totalwar.org/maps/Tools.shtml
4th file from the bottom, Texturetool.

------------------
Ja mata
Toda MizuTosaInu
Daimyo Takiyama Shi
http://www.takiyama.cjb.net

[This message has been edited by TosaInu (edited 08-10-2002).]

[GilJaysmith]

Quote Originally posted by TosaInu:
Konnichiwa Gil Jaysmith sama,
Please enlighten me here, what's the use of having the jjm file in txt format?
Is it possible to change the txt and then convert it to a working jjm file again?
[/QUOTE]

The main use will be to anyone who feels like modifying the source to do interesting stuff to the map representation and then write out a new .JJM file. With a tool like this, a man... could rule... the world! Or at least automatically convert an STW map to look OK in MTW without having to load it into the editor.

Gil ~ CA

[TosaInu]

Konnichiwa Gil Jaysmith sama,

Is this used at the CA office too? What's the benefit of this tool over the mapeditor that CA uses?


------------------
Ja mata
Toda MizuTosaInu
Daimyo Takiyama Shi

http://www.takiyama.cjb.net

[barocca]

watch this space

17 tiles to go and I will have the visual tile map completed,
i'm just taking a quick break to rest my eyes,
Then anyone will be able to see which tiles in stw represent which in mtw!
http://www.totalwar.org/ubb/smile.gif

[barocca]

Done

I suspect there are more than 161 tiles in mtw, at this moment i cannot access these additional tiles, all I have displayed are the 161 tiles i can currently 'reach'


sample image
http://doragonbarocca.homestead.com/files/mtw/lite.jpg

download here
http://doragonbarocca.homestead.com/files/mtw/TileMap_mq.zip
(268k)

a very high quality version is available, but it's a full meg...

http://www.totalwar.org/ubb/smile.gif

===
this took quite a while to make, if i've made any errors - due to lack of sleep - let me know asap!, and i'll fix them
http://www.totalwar.org/ubb/smile.gif

------------------
Clan Doragons Medieval Website
Mods, Unit Descriptions and more
DoragonBarocca of Clan Doragon (http://doragon.cjb.net)

[This message has been edited by barocca (edited 08-11-2002).]

[This message has been edited by barocca (edited 08-11-2002).]

[barocca]

mapdump command line,

msdos prompt >>

mapdump mapname.jjm >output.txt


http://www.totalwar.org/ubb/smile.gif

[This message has been edited by barocca (edited 08-12-2002).]

[Kraellin]

yup bar,

just checked my mail and got it also. unzipped and ran just fine. this is cool. this shows the entire file structure, especially when compared to a hex dump of a map. very easy to discern what everything is now.

shld be fairly easy to make a VB converter to turn the .txt file back into a .jjm, or is that what some of the c++ files are for?

also sent a copy to tosa. and gil, i was right. he had it running before i did ;)

even without a VB converter, i can now see all of the file structure and could do edits from a hex editor or even notepad, prolly, though that would take considerably more time.

gil said: 'With a tool like this, a man... could rule... the world! Or at least automatically convert an STW map to look OK in MTW without having to load it into the editor.' he's right. the height data is all there, the texture array is there and the model data is there. even the header info is there. with a byte by byte read > edit > write routine, it shld be possible to alter maps on an automatic basis.

and, it ..might.. be possible to alter map sizes as well. alter the header, add in height data, alter the array...a bit tricky, but may be possible.

K.



------------------
The only absolute is that there are no absolutes.

[barocca]

Well Kraellin,

creating such a VB program is far beyond my skills,

after all I just spent a day and a half taking screenshots of the stw/mtw tile sets,
only to find the mtw tile set tucked away in a folder...
i could have saved half a day if i had found them first,

Note
Tile to be displayed is refered to numerically,
Is it possible to create a VB program that will allow us to use tiles numbers BEYOND the current range?

If mtw does not allow custom tile sets, then perhaps we can create and load tiles outside the current numerical range of the tilset.
(3 sets of 170 tiles are included in the demo)
If we cannot get mtw to load tiles from a custom folder my other hope is to go 'beyond' tile 170 (or what ever is the final number of tiles in the mtw release) and create our custom tiles there.

We would temporarily replace a current tile set, create our map, and then renumber our custom tiles, drop them into a terrain folder and use such a utility to adjust the tile reference numbers in the map file!

Don't try and create such a utility just yet, just tell me if it wold be feasable,


Tosa
I know How to create custom tile sets,
I beleive I can even get mtw to load them,

the problem is they will (at this stage) be at the expense of an existing tile set,

there are 3 tilesets with the demo!
all of them currently 170 tiles (more or less), Lush tTile set tries to look for tile 171 when loading, and fails - it then stops looking for more tiles,
perhaps we can place more tiles beyond 170,


I'm just going to test my load custom tile set theory,
i cannot test my place/number tiles beyond 170 theory, because I don not fully understand the jjm file format, and thus cannot tell the jjm to look for tiles beyond 161,
Kraellin?
help?

[barocca]

http://www.totalwar.org/ubb/smile.gif

We can force any tile set into any folder

http://www.totalwar.org/ubb/smile.gif

ALSO
We have 2 more terrain folders that MTW Demo can recognise and use,
(it took a few loads and deciphering the failures to get these to work, thank heavens for the watcher utility)

The Folders are Temperate and Desert, I have not tested thoroughly for backdrops, but for summer they are both working fine.

what this means is if anyone wants to create custom Tile Sets, they can
http://www.totalwar.org/ubb/smile.gif

If the difference is going to be dramatic then I would suggest sending me a copy of the tile set and I can create a custom TileMap for that set to allow anyone to create their own maps,

ALSO any custom tile set will have to be made available for download if you want people to be able to play on it.

However the standard tile sets zip to 25Meg,
so you need to be concious of that and try to make your set as small as possible,

Here's How to Implement the additional Tile Set folders

Temperate Tile set

Create the folder here
..\Medieval - Total War (Demo Version)\Textures\Ground
call the folder Temperate
in the BDF call the Terrain::TEMPERATE

copy a set of the tree files in this folder
D:\Games\Medieval - Total War (Demo Version)\Textures\Trees
(or create your own)
rename them
Conifer0.lbm
Conifer1.lbm
Conifer2.lbm
Conifer3.lbm


Desert Tile Set

Create the folder here
..\Medieval - Total War (Demo Version)\Textures\Ground
call the folder Desert
in the BDF call the Terrain::SAND_DESERT

The Sandy Desert uses the Desert Palms tree files
despalm0.LBM, despalm1.LBM, despalm2.LBM, despalm3.LBM
so no need to create them, unless you want to change the palms
(NOTE:- This Terrain Choice automatically deletes tree models from forest terrain types)



one theory proven, one to go...

====
edit
====
I forgot to mention,
if you look back at the sample TileMap earlier in this thread you will see 13 White Tiles - these are Actually BLANKS,
meaning you can create 13 NEW tiles without having to create the new folders!

Arid and Lush both contain 13 Blank Tiles,
numbered 88 to 96 AND 98 to 101.
RockyDesert is a Full Tile Set


Tiles 162 to 170 of all 3 sets contain aditional Tiles that I have not yet figured out how to get the WEMI editor to reference.


[This message has been edited by barocca (edited 08-13-2002).]

[Kraellin]

bar,

lol. i thought you were using the textures in the demo to do your map, otherwise i'd have spoken up. btw, nice job!

ok. lots of stuff there. lemme see what i can cover.

texture numbers beyond the current range... can only guess on this one. they're prolly using a single byte for referencing the textures, so # 255 is prolly as high as it goes. in fact, when researching map making during the stw demo/editor, i ran into a phenomenon where the texture numbers 'rolled over'. in there, the numbers only went up to 127 and then rolled over to a new set of 128 numbers, but the second set was completely unusable. still, that made 256 textures, so i suspect it's still the same...256 max textures for any one set. but, only a test will prove whether we can access the higher numbers or not. if the .exe isnt set up for it, i'm not sure. just have to try it.

the same, based on what you've found out about 'temperate' and 'desert', may be true for additional texture sets. it ..may.. be possible to have an unlimited number of texture sets or it may be something hardwired into the .exe. i suspect those additional sets you found are projected sets that will be released with the full game, so, it depends on how things get called and referenced. CA's track record suggests that it will be hardwired, but, of late, they've been loosening up on the hard wiring, so i dunno. it might be possible to tweak something and get addtional sets to work.

the thing is, something has to call up the proper textures for the given map. if that data is in the .adf then we're good to go. we could just name our sets anything we wanted and make the maps that would store the texture set internally............

oh hell, i just looked at the .bdf. they're way ahead of us on this one. the .bdf stores both the map name and the terrain type (texture set). that ..shld.. mean that we can make any number of sets, store them the textures folder and reference them from the .bdf. so you ..shld.. be able to make a texture set named 'ice cream', if you wanted to, stuff it in the texture folder and simply alter your .bdf to call it up.

now, backtracking a bit, it's the map editor that's going to determine if this can be done on an automatic basis or not. if the editor allows for importing custom sets then it's a piece of cake. if not, then we shld still be able to edit by hand. you'd make your map in the normal way with a default set of textures, save the map and then edit the .bdf.

but, that only works for historical and any other forms of map that use and are referenced by the .bdf. so, it may not work for multi and custom games. i dont know where the call is for pulling up maps for those. i'm afraid those may be hardwired in the .exe and CA would have to pull that out and make it available to us to use....unless....i havent done a map dump on an mtw map yet. if they put the texture reference within the mtw maps, then we may be good to go.

bah, i just did a map dump of jaffa.jjm and i dont see a texture set reference. however, some of these byte numbers dont make sense, given a single byte system. some of the texture bytes are double bytes, which might indicate both a texture number and a reference to a texture set. what i would expect to see is simply a 2 digit hex number, representing numbers from 0 to 255, but a number of the textures are 4 digits, or 2 bytes and the first 2 digits are very limited in what they are, indicating a possible reference to a texture set?

i dunno, i'm too tired right now to work it out.

i can tell you what the map dump data means, however. the stuff at the top is header information. simple stuff. i'll explain that in more detail later. the next stuff is the height information, the 'contour' stuff. this is laid out in rows, prolly starting in the upper left hand corner of the map. next is the actual texture numbers and below that is the model data. btw, this map dump stuff matches up just fine with a hex editor result, except the map dump reverses the ibm style of reversing bytes in 'words'.

yeah, i saw the blank textures also. i'd be careful of those. CA may have some use later on for them or they may do something weird or they may have just not been finished at the time of release. dunno, but i wouldnt get your hopes up of being able to use those.

K.


------------------
The only absolute is that there are no absolutes.

[barocca]

the bdf Terrain names for tile sets are hardcoded into the demo,
so are the tile set folder names - note they are different http://www.totalwar.org/ubb/frown.gif
(I did an ascii dump of the executable)

All we can do is HOPE they change that to allow any folder names at all
(Tosa and I already tried new folder names - no joy)

All I had to do was decipher the folder names and tree names by 'watching' the demo try to load a map using the temperate and sand_desert terrain types.

===
EDIT
scratch that, just dmper jaffa and model locations appear,
still unsure of tree location data though...

ALSO
if the numbers within the texture table were the tile numbers only they should never exceed 2 digits,?
most are 4 digits,

?

[This message has been edited by barocca (edited 08-13-2002).]

[This message has been edited by barocca (edited 08-13-2002).]

[This message has been edited by barocca (edited 08-13-2002).]

[barocca]

just used wemi editor to create a small flat map - perfectly flat,
all tiles came up as number 4
correct

will have to test to see what cause the
4digit code we are seeing for some tiles,

I have less than 60 minutes before i have to go out
B


GOT IT
Top left as you load map is first tile in array

4004 8004 c004 4

Tile one - type 4 rotation 90 degrees
4004
Tile 2 - type 4 rotation 180 degrees
8004
Tile 3 - type 4 rotation 270 degrees
c004
Tile 4 - type 4 NO Rotation
4

helps?

ALSO
before and after smoothing
before

Contour North Contour East Contour South Contour West
( 0,fffffef5, a000) ( 0,fffffef6, a800) ( 3000,ffffff43, 7800) ( 5000,fffffef4, 4800) ( 7000,fffffea7, 2800)
( 800,fffffef5, a000) ( 800,ffffff43, a000) ( 2800,fffffef4, 7000) ( 4800,fffffea7,

after

Point Contour North Contour East Contour South Contour West
( 0,fffffef5, a000) ( 0,fffffef6, a800) ( 3000,fffffef5, 7800) ( 5000,fffffef5, 4800) ( 7000,fffffef5, 2800)
( 800,fffffef5, a000) ( 800,fffffef5, a000) ( 2800,fffffef5, 7000) ( 4800,fffffef5, 5000) ( 8000,fffffef5,

!

HOWEVER
The location of tree models does not appear in the output,
???
I just placed 15 trees on tile number one,
and i can't spot any changes between the 2 dumps!!!

[This message has been edited by barocca (edited 08-13-2002).]

[GilJaysmith]

Quote Originally posted by barocca:
The location of tree models does not appear in the output,
???
I just placed 15 trees on tile number one,
and i can't spot any changes between the 2 dumps!!!
[/QUOTE]

I never did get around to dumping out the trees.
There's a block of data just before the models where the hand-placed trees are stored. 11 bytes per tree as follows:
- tree x and z coords, 4 bytes apiece
- scale, 2 bytes
- sprite index, 1 byte

Spot on with the tile rotation...

Gil ~ CA

[barocca]

http://www.totalwar.org/ubb/smile.gif
So the Tree data is there, just not output to file,

btw, is there any way to 'access' the stw/wemi tileset?


(other silly news)
I have been messing around with a custom tileset,

i took a 4 quartered photo of a moon crater, plus one of a plain,
and cut them to tile size,

after i figured out the demo won't load a tile unless it is 16M colours, and changed my tiles accordingly, I then discovered the demo views gray? tiles as impassable,

which is quite strange, as the demo was letting the AI march freely up and down cliffs, which were gray, and should have been impassable,
see the custom map i made, remove the pond below the pass, restore the elevation to the corner tile (curently 'under' water) and watch the AI climb the walls...

Anyway, i took my 'moon' terrain and 'greened' it, but the demo still views the tiles as impassable...i think I have some more experimentation to do....

http://www.totalwar.org/ubb/smile.gif
(it was an experiment, ok?
i just wanted to see how radical i could get...)

[This message has been edited by barocca (edited 08-13-2002).]

[GilJaysmith]

Quote Originally posted by barocca:
btw, is there any way to 'access' the stw/wemi tileset?
[/QUOTE]

We aren't shipping a secret copy of them, if that's what you mean.
I don't know if the graphics format has changed but if not then I don't see why you couldn't reuse them.

Quote Originally posted by barocca:
my tiles accordingly, I then discovered the demo views gray? tiles as impassable,
which is quite strange, as the demo was letting the AI march freely up and down cliffs, which were gray, and should have
[/QUOTE]

IIRC a cliff obstacle is made when you have a big gradient and the underlying texture numbers are recognised by the code as cliffs. The cliff texture numbers are:

46-50
63-67
171-178

Any other texture will lead to a steep but climbable hill.

Gil ~ CA

[barocca]

for the tiles i was playing with, they were right on the water line, meaning mtw was reading the map as surrounded by water,

the cliffs, i was using textures between 63 and 67, - some they could climb, most they could not,
a glitch of some sort, checked all the tiles in the range you specified,
simply - 'dunno'
(thats why i settled on 'water traps')

as for the stw/wemi tiles,
um,
i can't find them?
i am told they are packed away inside a file of some sort?


[This message has been edited by barocca (edited 08-13-2002).]

[Kraellin]

bar,

i had to go back and check and i believe if you look in the 'campmap' folder of we/mi you'll find the .bp8 files are the texture files for the map. i'm basing this mostly on the size of those files. and then in the 'splines' file you have seasonal files which might be overlays to the textures, not sure, but also .bp8 files.

i dont know if those .bp8 files are compressed or not, but it's the one format we never cracked and the file size is about right.

what is that program you're using to watch the stuff load and where can i get it?

no, the numbers in the texture array table arent tile numbers. remember, this is an array table. these numbers are simply laid out so that they will be entered, in sequence, into an array. the array is then called to see what textures go onto the map in order within the array. since, in some cases, they are using a double byte and not a simple integer, then there is more than just the simple integer data of the texture being looked at here. i can only speculate what that extra byte is being used for. if you look at the hex numbers of the first 2 digits in the map dump you'll see that it is often the same 2 digits being used over and over. in a dump of the aki map i noticed that numbers like c0, 80, and 40 were used over and over. c0 in hex is 192, 80 is 128, and 40 is 64.

now, a texture reference ..shld.. only have to be 1 byte, and in some cases it is, but in some it's not. so what's that extra byte for? ah...orientation! ok. the extra byte must be for rotation of the texture. d'oh. ok, i was tired last night :) and d'oh again, for i just realized that's what you were saying in your post above about rotation. ok, i'm a little behind here...gimme a minute to catch up :)

ok. so there's NO reference in the map data as to tile sets or 'terrain'. too bad. that means that the terrain sets are hardcoded, most likely in the .exe.

ok, gotta check some stuff out. back in a bit.

K.


------------------
The only absolute is that there are no absolutes.

[barocca]

K,

Quote Originally posted by Kraellin:
bar,

i had to go back and check and i believe if you look in the 'campmap' folder of we/mi you'll find the .bp8 files are the texture files for the map. i'm basing this mostly on the size of those files. and then in the 'splines' file you have seasonal files which might be overlays to the textures, not sure, but also .bp8 files.

i dont know if those .bp8 files are compressed or not, but it's the one format we never cracked and the file size is about right.

GilJay? can you help here?


what is that program you're using to watch the stuff load and where can i get it?

Send me an email and I'll send it to you, (94k) - includes makefiles, libraries and source code! in C!
(I don';t think everything is in the package I have, but there is a fair sized chunk of data)
I have had it for so long I no longer remember where it came from, it is slightly unstable, so when you're using it don'thave anything important happening,
by slightly i mean it does not like to be 'open' when you are trying to do 3 or more things at once.
I use it, save the file and then call an external process monitor to kill it, I think I can send you the process monitor too...(147k)


... d'oh. ok, i was tired last night http://www.totalwar.org/ubb/smile.gif and d'oh again, for i just realized that's what you were saying in your post above about rotation. ok, i'm a little behind here...gimme a minute to catch up http://www.totalwar.org/ubb/smile.gif...
- i read this last bit AFTER i made a huge explanation of rotation and tile numbering, i thought you must have missed that bit, THEN, I move on to answer your next point and i find the Doh!'s
my turn - doh! http://www.totalwar.org/ubb/smile.gif



ok. so there's NO reference in the map data as to tile sets or 'terrain'. too bad. that means that the terrain sets are hardcoded, most likely in the .exe.

It is indeed hard coded, in your email remind me to send you the ascii extractor as well (192k)...and get me to send the ascii output file i generated from the mtw_demo executable (228k)

What we can hope for though is for them to change it so the bdf file Terrain:: is the source of the address for the tile set, instead of them writing that into the code -
GilJay? any chance of that?



ok, gotta check some stuff out. back in a bit.
K.
[/QUOTE]

Hmm - we're getting close to 600k with the data i need to send you,
tell you what,
send me an email and i send you an addy where i will stash the whole 'kit and kaboodle' for you to download, ok?

barocca_x@hotmail.com

http://www.totalwar.org/ubb/smile.gif
B.
(now lets see if i managed to type all that without spelling error's shall we...)
http://www.totalwar.org/ubb/smile.gif

[Kraellin]

ok bar,

i've done a little playing around with the height stuff and here's what i've found out so far...

the height or contour data is by rows as stated in the map dump file. if you notice in a 20 x 20 map there are 21 x 21 map dump rows. this is because you have to add an extra point for the end square. for each 'point' there are 5 data sets, the numbers between each parentheses. like so:

( 0,fffff651, a000) ( 0,fffff651, a800) ( 3000,fffff651, 7800) ( 5000,fffff651, 4800) ( 7000,fffff651, 2800)

in any one 'point', that long middle number is the heigth of that point. i suspect the others are vector numbers to the other 4 cardinal points. two corresponding vectors would give you a curve or contour...i think :)

also, that middle number works backwards; the higher the number, the lower the heigth on the map.

that point set i just posted is from ironing board, so all the height data is the same. i suspect the 2 numbers on either side of the heigth data are references on the grid, but not sure of that yet.

the map dump DOES work from top left of the map to bottom right.

the data starts along the top edge of the map, and remember, you're defining the POINTS where you raise the tiles. so the first 21 sets of 5 are the points along the very top edge of the map (top being top of the screen when you first load a map). the next set of 21 of 5 are the next horizontal row going left to right again and so on down the map.

this was verified by making 4 copies of ironing board and raising 1 point only of each of the 4 points of the top left tile. the data in the map dump of each of those maps confirms this data.

Rob (or rob's map converter fame) had always said there were 5 points to each tile, but he never understood that 5th point. i think now we may have it.

this may be my last post today....storms in the area and it's getting dangerous to remain online for now.

K.


------------------
The only absolute is that there are no absolutes.

[barocca]

K,

Until we know wether or not we can 'rip' the stw/wemi tileset from the bp8
AND actually create and use custom tilesets for mtw,
All we can do then is make a best guess converter?


A best guess converter is a closest match converter, you pick a tile and it's orientation that matches as close as possible the original.


We can now select, orient and adjust the height of the tiles.
We can place models,
I am unsure if we can adjust the positioning of tree models (GilJay gives an explanation of the Tree data blocks earlier in this thread)

As you can see from the TileMap there are a vast number of tile equivalents missing!

Also we know the Mtw Tileset (currently 3 sets of 170 tiles) is definately going to be larger than the WEMI Tileset (161 tiles), (and the Demo tries to load Tiles that do not currently exist).
I suspect the Release Tileset's are going to be even larger than the demo's.


Before anyone makes an effort to write a converter to bring stw/wemi maps up to mtw demo standard we have to ask ourselves a few questions,

Is the release going to occur as expected on or around the 6th September? (my local EB still has that date in their System)

How large is the mtw tile set going to finally be?

How is the Release version going to handle custom TileSets? - if at all?

Which Tile numbers will the Release recognise as Forested? (GilJay has already given us the recognised Cliff Tiles)

What amount of effort will be required to convert the code for the stw/wemi to demo converter into a full blown stw/wemi(/demo?) to Final Release converter?

We must remember we may need to adjust the code to 'convert' to tiles beyond the current range of 161 available in stw/wemi and 170 available in the demo.


I would however suggest begin experimenting with coding now, when the full release version is in our hot little hands adjusting the code to suit will be a heck of a lot faster than beginning from scratch!


(note to K, my personal 'coding' skills can be graded squarely between pathetic and woefull, :/
my analytical and research skills can be graded as superb, although I will make assumptions on points i don't quite understand, and ask for comments or clarification http://www.totalwar.org/ubb/tongue.gif)


NOTE TO Developer Types :-
The easiest method to allow custom Tilesets is if the line
Terrain::xxxx
in the bdf is the ADDRESS of the tileset folder! and NOT hardcode such values into the executable!
Simple and Straightforward!
It should also require only a minor change to code!
Currently the Terrain:: "value"
is hardcoded, and there is a matching Terrain folder for that value which does not always have the same name!

If the executable simply read the Terrain::"value" line and then loaded the tileset from an exact match textures/ground/"value" folder it would simplify and reduce code and simplify implementation of additional or customised tilesets.

B.

[Kraellin]

hehehe...slow down, bar ;)

ok. a rip of the original texture sets would be nice, but not essential.

i'll explain that first statement later.

first off, we dont need to alter anything in the we/mi maps EXCEPT the texture data, hand placed trees and models. the heigth data is unimportant in the conversion, other than simply making sure it stays intact. we already know the overall structure of the files and we know where and how the texture data works. in the simplest form of a converter, then, it's only necessary to renumber the textures into something coherent when the map is used in mtw. it may not be a perfect conversion, but it shld be workable.

we know where the texture data is and how it's coded. if we also know what textures in we/mi maps match up with mtw maps then all we have to do is make a routine that recognizes and alters those textures that need changing to be displayed in an mtw map correctly. that's all. the converter would simply scan the map files, see X and convert it to Y, then save and maybe rename the file for use in mtw. or, it could simply save it to the mtw map folder. i'd suggest a rename, maybe add an 'M' or something at the end of file name, but before the extension.

in addition, the app could check the models and do the same thing with those, eliminating those that shldnt be there...OR, we could simply move the models folder from we/mi over to the mtw folders. has anyone tried this yet? they might just work.

back in the 8 bit days, i was learning Assembly and wrote a little app that did this exact thing to various files. it simply read the file, byte by byte, and if it encountered certain bytes it would change them to something else automatically. pretty simple stuff. i just dont happen to have any type of compiler installed currently, so someone else would have to write it.

for a more complex converter/editor, it would require knowing EVERYTHING about the map files. i know where the height data is stored, but not everything about it. i'm still guessing on some of that. tosa also suggested that we could currently make 100 x 100 maps with the data we know so far. but again, we'd need a FULL understanding of the files. hypothetically, it's possible to write an entire map from scratch in a text editor or a hex editor. liken it to the guys who could 'read' the matrix from scrolling machine language ;)

as for using a .bdf file for the terrain data, i dont think so. remember, multiplayer games and custom games do NOT use the batinit folders. however, a new folder could be made in the same wise as the stat folders were made for we/mi. you extract the hardcoded stuff from the .exe and simply have the .exe reference a textureset folder or, you put a pointer in the map file header referencing the texture folders. the .exe simply then looks for this pointer and pulls up the appropriate texture set. this would have an added advantage of making it easy to do the same map in many different texture sets, if wanted.

the tree data is easy. gil did explain that one, and yes, we could repostion the single tree models.

regarding how large the final release tile set is going to be, it doesnt matter for making a converter...well, it does, but, we can work around that for now, in the simple version of the converter.

"...What amount of effort will be required to convert the code for the stw/wemi to demo converter into a full blown stw/wemi(/demo?) to Final Release converter?..." doesnt matter for now. even if we had to re-write the entire thing later on, it's not that difficult a project.

i'm afraid i'm not much of a coder either, bar. i had a borland compiler around here somewhere, but it's quite old...win95 days and i'm one of those guys that looked at 'learning C in 21 days' and gave it up after 3 :) i did manage to write 'hello world' though ;) i loved assembly. very logical, very straight forward and you can tweak EVERYTHING exactly how you want it. a lot of 'high' level languages sometimes only let you come close, but not quite. but, everyone tells me to stay away from assembly in a windows environment. i think it's a conspiracy, myself ;) prolly started by micrsoft or borland ;)

i'll play around some more with the heigth data and see if i can figure out the rest of those numbers. but for that, i think we've mostly got it now.

btw, in case that first post about the heigth data wasnt clear, each 'point' on the map which is a heigth point (the place in the map editor where pull up or push down the elevation) comprises 5 of those sets of numbers, the ones in the parentheses. so this: ( 0,fffff651, a000) ( 0,fffff651, a800) ( 3000,fffff651, 7800) ( 5000,fffff651, 4800) ( 7000,fffff651, 2800) is the data for ONE point on the map.

when i made those 4 ironing board maps, i raised one corner of one tile only in each map, each successive map raising a different corner, which is how i verified the layout of the data. in one map i raised a corner just a tad too high and the resulting elevation 'spilled over' into the next tile. the data in the map dump showed this also, which is how i derived the bit about the vectors.

K.




------------------
The only absolute is that there are no absolutes.

[barocca]

K,

I need you to email me so i can send you the link to the tools,
barocca_x@hotmail.com


I would have assumed that the online battles would ask you to select the terrain?
Otherwise how will it know which set of tiles to use?

Naturally I would expect it to 'look' for terrain folders in the same way as the bdf,
wether that be through encoding such information into a map - seems painfull,
(have to change the way the maps are made as well)

or by the executable doing a search of the 'ground' folder and reporting which terrains were avaiable.


As for Assembly, it is stable in windows,
also the program size will be minute compared to all the useless odds and ends that Borland compiler will add to the program.

An assembly program is 'stand alone' requiring no shared files or resources.
(If it works in 95 it will work in 98 etc.)

If you do not have the resources to write a converter avaiable I would strongly suggest compiling everything we know and learn about the jjm file into a document and making such a document available to rsw, tm and tosa,
and see what magic they can come up with!

Now if you can find, beg, borrow or 'aquire' an assembly compiler...

here http://download.com.com/3000-2069-895343.html?tag=lst-0-4

here http://download.com.com/3000-2352-4433199.html?tag=lst-0-9


and lots and lots here http://downloads-zdnet.com.com/3150-2069-0.html?tag=dir
(clouded the future is, difficult to see, hmmm?)

B

[Kraellin]

bar,

yer prolly right about the texture sets. so what you're saying is that in a multi game there will be an option to choose the texture set but that it's a hardcoded option, limited to the existing texture sets. and if that's the case then we'd need an option like we have for picking maps. fair enough.

heh, i can tell you now i'm not learning assembly just so i can code this little app. it'd be like learning to drive a semi-truck in order to ride a bicycle ;)

K.


------------------
The only absolute is that there are no absolutes.