BIF

[TosaInu]

Konnichiwa,

I found a BIFreader on the internet. It's a viever but it has some editing capabilities too. It's a 3.6 mb download, there's also a guide for download and there is a forum for this tool. I made a BIF file myself. When I tried to open a STW BIF file, I got the message access violation. Did CA 'protect' the image files? Is it a different BIF format? Is my computer not configured correctly (wrong windows/ missing files)?

The first url gives an overview page of some graphic tools, the 2nd gives the BIF reader 20/20. I hope someone can do something with this.
http://www.freebyte.com/graphicprograms/ http://www.hotfreeware.com/2020/2020.htm


------------------
Ja mata
Toda MizuTosaInu
Daimyo Takiyama Shi

http://www.takiyama.cjb.net

[Kraellin]

tosa,

yeah, i tried that reader too with some stw files. couldnt get any of the shogun .bif's to work or show as valid files or images. my estimation/speculation about the stw .bif's is that they are sort of supplement files of raw data. for instance, if you take the .lbm files for the unit images, you'll notice they are all one color and not varied for the different daimyo colors, so how do they do the coloring. my guess is that the .lbm files are the base images, while the .bif's do the coloring necessary for each clan. and since all you would need for that is raw data and not header info, i'm guessing that's what they're using the .bif's for.

there's prolly several examples of this sort of thing and they could be using them in other ways as well. some may not even be for images....dunno. but if a .bif is more or less just a raw data file, then it could be used for most anything where you wanted to amend some other file.

and too, i could be completely wrong and the .bif's are part of a secret society thing that we know nothing about :)

every once in a while i look at the .bif's and try out things like 20/20, but so far i've not really sat down to crack the things. one thing that could be done, if one had a lot of patience, would be a simple trial and error type testing. take one of the .bif's in the textures\men file that you know is associated somehow with that unit, like the ashi file. take the .bif in there and start altering it a few bytes at a time and load the game and see what happens to the ashi look.

i know some folks back in the early stw days did something like this and ended up with some odd looking units, but i dont recall what they did specifically, or that they even posted what it was they did to wind up with this.

the trouble with a reader like 20/20 is that it still has to know what the data is being applied to and the format that is being used in doing that. CA may well be using one routine to load one .bif and another routine to load another .bif. so, even if you found a reader that would work on one .bif type, it wouldnt necessarily work on another one that is being applied to something else. you'd have to know how each .bif is being used and on what and where and when.

therefore, the only way to alter them may well be a trial and error sort of thing and that gets very time consuming. so, i dont think it's impossible, but it's quite likely the last thing we'll be able to mod.

K.


------------------
I'm sorry, but i never apologize.

[Kraellin]

i'm more convinced than ever that the way .bif's work is actually fairly simple and really quite a smart way to program, at least from a programmer's point of view.

the concept seems to be analagous to something like a coloring book. the .lbm files are the book itself with the various images in them that are sort of generically colored. the .lbms are used for animations of the units. each .lbm file is a composite of the looks that each unit, for example, can have when it's on the field of battle. so, if you have a cav unit, let's say, it has just so many ways it will look on the field; it's turned this way or that way, etc, etc. the .lbm shows all of these looks in one composite file.

but, what the .lbm doesnt show is the colors for the different armies. that's where the .bif comes in. it 'colors' those generic .lbm units to fit the color used by X daimyo. and that's what i meant by a sort of coloring book concept. the .lbm's are the various pages of the book with generic pictures and the .bif's are the crayons or colors used to denote what unit belongs with what daimyo.

this is a very smart way of doing this, since you dont need any header information in the .bif file like other image files use and, the .bif can be used in any other situation like this in the game. it's just raw data, so, anywhere you need to add in raw data to images or anything else, a .bif works just fine.

the program calls up the .lbm's and puts them in memory for quick animation and the .bif's color them according to whom they belong. the .lbm's are very small files and the .bif's wouldnt even have to alter the image size at all, just change a few bits here and there and all done. for each game this then stays in memory as initially loaded and saves a ton of space on your harddrive, cd and in memory.

this also explains why the .bifs are not any sort of set file size and why they cant easily be read by a stand-alone reader. the data have to be applied to another file, not just read alone and displayed.

thus, it shld actually be possible to make a reader that would take the base file being colored or amended and apply the .bif to it to show the finished image. since the image data for each unit is provided in one file, both the .lbm and the .bif for it, one might be able to conceive a reader providing one knows one other thing....where the data in the .bif is placed, and that might take some work.

i see two possibilities on how these .bif's might be contructed and i havent studied this so dont take my word for it, but one, just the coloring data is in the .bif and two, the coloring data AND the location of where to color might be in the .bif. if the 2nd is true then you'd have to know the format for how color and location were arrange...did they do all the colors first and then do all the locations or did they do color/location side by side byte to byte and if the latter is true, is the location first or the color.

and, having just re-looked at the files, i see that there are 2 .txt files in those locations also. these may be the locations or the colors also...hehehe...confusing isnt it. if you want to study this on your own, the files are: textures\men\[unit name]. each unit name has its own separate folder with its own .lbm's, .bif's, and .txt files.

now, those .txt files may be something completely different. i'm really not sure. most places i've found the .bif's being used there is also a corresponding .lbm but not any .txt file. so i'm fairly sure i'm on track with the .lbm's and .bif's but not sure about the .txt files in the unit folders.

K.


------------------
I'm sorry, but i never apologize.

[BanzaiZAP]

Well, I'm back on island, and was briefly terribly excited about the burst of modding that's happening, but was soon horrified to see the omnipresent BIF problem has not been broken. *sob* Thanks, by the way for emailing me the information! (you're so good to me... *sniff*)

ANYWAY, here's my thoughts. Some of this is resaying what Kraellin is talking about, just in other words.

I agree with the Coloring Book Hypothesis, and I think the text file is mostly used to define the "pages." Reading some of the old links, it seems that the graphics file (in our case the LBM) holds the visual data, and the text file is used by the program to know where each "page" fits. In some types of BIF, the text is actually embedded as the header in the BIF file itself. So in Shogun, the text defines which parts of the LBM are used for which animations. The numbers define rectangles of pixels in the LBM, and the program plays those rectangles as a slide show, and presto, the archer fires his bow.

As for the custom colors, much of that is done by varying the appearance of partcular set colors in the unit's palette. For example, the archer book has a dark green background - the text file (or the program) sets that color to be transparent. The flags are done in certain shades of grey - the program turns everything that grey into Mori Red, or Shimazu Green. (Ah, but do you know Shimazu Green's Pantone number? How about hex number? "Get a life!")

So the main reason we can't crack into it ourselves, is that Creative Assembly has their own translator. Without that specific sequence of events, we can't make a BIF that fits the right pattern for Shogun's engine. All we really need is someone from CA to release a BIF compressor specifically for Shogun.

A "simple" one would just re-import the LBM. We would have to draw over the exact unit figures, not break any "rectangles" and use the exact color palette, but hey, that's what I've done already! Just replace the old LBM with your new one, activate the exe, and it saves as a new BIF. If you went over the lines anywhere, you'll get garbage, but that's programming. An advanced compiler would allow users to define our own rectangles and palettes, and also sequence our own animations! But building an interface for something like that would probably not be cleared by Marketing.

But I'd be happy with the simple one....

-- B)

[Kraellin]

banzai,

hey, i'm just guessing here. havent run any tests, so any insight you have on the .bif's is great.

couple questions though, in the 4th paragraph you mention, '...the archer book has a dark green background -...'. what are you referring to here...what book? and then in the same paragraph you mention, '(Ah, but do you know Shimazu Green's Pantone number? How about hex number?'. what is Pantone? what's that mean?

so, yer saying that the .txt files of numbers are the animation sequences for various applications of the .lbm's? could well be.

tosa didnt much like my coloring book analogy and it wasnt meant to explain all. basically what i'm theorizing is that the .bif's are only used to amend and alter other files or memory registers. by themselves, they are just meaningless strings of hex numbers, but when applied to another file or memory, that's when they work their magic.

as for the 'translator', tosa also argued that CA must have a tool like that also. sorta makes sense.

also, if you know anything about .buf's or .bp8's please let us know. one thing i've noticed is that anywhere there seems to be an actual texture file, it's a .bp8 extension. not sure if these are compressed files or not, but i do find that file extension everywhere there shld be a texture file.

i'm not really seriously attacking this .bif problem myself. i just poke around from time to time and post what i 'think' may be going on. so, plz, feel free to tear the whole thing apart. you wont be stepping on my toes :)

K.


------------------
I'm sorry, but i never apologize.

[BanzaiZAP]

I'm messing with it part-time myself. I've redrawn several LBM's, but have no way of telling the BIFs to get the new info.

When I said "book" I was comparing the LBM to the "coloring book" idea, which seems to have gotten out of hand. If you open the archer's LBM in a graphics program, the background is green. There are about thirty little guys, each in a new posture for a particular animation (about six guys "running to left" a bunch in "attack mode" etc. They also have grey flags. This means that the game engine knows to make that particular green transparent, and add the daymio colors to the grey areas.

The text file might dictate which figure gets used at which time in which animation. An example, if the series looks like:
4 12 135 189 125 158
this COULD (maybe not!) translate as
Animation 4, frame 12, rectangle defined by pixels (135,189) (125,158)

So the BIF compiler makes the BIF by going down the numbers in sequence, extracting those rectangles from the LBM, then saving in one file that the game engine plays the specific animations it needs.

So a custom translator would need to:
A) define how many animations, and how many frames each.
2) which parts of the LBM is used for each frame of each animation.

Ideally, we should be able to re-paint the figures, and as long as we have the same figure doing the same step of the same action, it should be just a matter or importing the graphic information into the BIF, and not need to revamp the text.

Pantone is a color-code mostly used in print. I was being faceatious! (sp)

Does this make sense? Do you think we can drag Target or another CA programmer into this discussion? Or are we trying to break several non-disclosure agreements?

-- B)

[Kraellin]

banzai,

yes, i'd seen those grey portions of the .lbm's also and figured pretty much the same thing. hadnt thought about the green much though. prolly right though. to get your 'pantone' just load the thing up in something like paint shop pro and look at the rgb numbers :)

i'm almost certain now that .bif's are indeed overlays to other image files. they 'might' be used to overlay data files, but i havent found an instance of this yet.

i like your idea of the text files, but in looking at a couple of them i noticed negative numbers in a few places, so that kinda makes me think not so. there IS a consistency in there though. all rows have 6 numbers in the row ranging from a small negative up to about 256. so we're talking a signed integer here. and one other bit of data that stirkes me as being interesting is that one of those text files has the same number of rows as there are mini-pics in the .lbm. so if the .lbm has 124 separate images of are arque unit, then there are also 124 rows in that one text file. i think...hard to count all those little arqu guys in one .lbm. so, i would guess that one row corresponds to one mini-pic in each .lbm.

but again, i've tested nothing. pure speculation at this point. tosainu IS testing some of this stuff though, so maybe you can get him to post some info on this as well ;)

K.


------------------
I'm sorry, but i never apologize.

[Yoshitsune]

Performed a little experiment that seems to support some of the theories expressed. What I wanted was to 'mod' the kensai and make him a horse-archer 'hero' a la Genpei War period.

No problem altering the troopstats file to give him an upgraded horse archer profile but I wanted to see if I could substitute the CA graphic for the kensai's. First problem is that the kensai doesn't have his own graphic files in the textures/men folder - he uses the NoDachi LBM and BIF except larger. Fair enough, the 12th Century wasn't known for ND regiments so I sacrificed them. Then I found all Jap Cav seems to be dealt with differently to other units. They use TWO sets of graphic-related files (lbm, bif and txt). These are prefixed 'Cav1' and 'Cav2' with the first lbm showing weaponless riders at the gallop etc and the second showing riders with bows. The program seems to determine whether 'Cav1' (files NOT folder - this is getting confusing ) are to be armed with yari, naginata etc. With this in mind I first copied the 'Cav2' archer files over to the NoDachi folder (textures/men/NoDa) and renamed them to replace the (backed up) originals. Here I noticed another difference between Jap Cav and the rest; Jap Cav don't have the larger '_H.bif' file. For the time being I left the original NoDa_H.bif intact.

Decided to test at this point to fired up the game. Well it didn't crash but the new unit graphic was now a rectangle, conforming to the size of the kensai, that flashed bits of horses legs etc at odd angles.

Exited and edited some more. I figured the the NoDa_H.bif must be the culprit so I just replaced it with a second copy of the 'Cav2.bif' suitably renamed even though its much smaller than other 'H' bifs. I also went into the 'actions' folder and found the kensai DOES have unique txt files here (MaSw). Replaced them with renamed 'Cav2' action txt files.

Restarted the game and found I now had a horse-archer hero instead of a kensai! Only problem is the 'firing bow' animation is playing constantly even when no arrows are being loosed. When the figure moves the gallop animation won't play so the horse appears to 'glide' (scary...). Seems to work OK when charging into melee though. This suggests that both 'Cav1' and 'Cav2' graphic files are used by CA depending on the action they are performing (shooting or moving).

I finally settled on using the 'Cav1' files instead which means, even though he will shoot arrows, I don't see the firing animation. Much cooler effect overall though.

Sorry that this does not provide any answers to the bif problem but I was pleased I found a little 'work-around' for my own purposes.

[BanzaiZAP]

Wow, thanks Yoshitsune! Gives us something mroe to chew on.

Question: Did you copy both the BIFand the LBM that first time? If so, then we still aren't sure if the LBM is necessary for the
BIF. It does seem show that the TXT file seems to indicate where the "rectangles" are in the graphic. I think the "_H" are used for the 8 bit graphics, maybe, or close-ups? But why would the cav not have them? I'll have to try this myself, but only change one file at a time, to see which exact file has which effect.

-- B)

[Yoshitsune]

Yes, Banzai, I copied the lbm, bif, txt and page.txt files. Those 'bits of horses legs etc' first time round were from the cav2.lbm. Didn't notice any problem when moving the camera close in for the final experiment.

Do you think it is the txt files that are defining the rectangle? I got the impression that the problem first time round was the NoDa_H.bif causing the wrong frames but I may be muddled in my thinking. Replacing it with a second cav2.bif seemed to select the 'correct' rectangles...

The txt files in the 'actions' folder seem to be duplicates of the ones in the troop folders but with more specific names. What if these aren't changed? Or these are changed and the ones in the troop folders are left alone? You're right about testing things one change at a time to isolate their effect! I was being too impatient to reach my objective though these things occurred to me

Still not satisfied with the final result as there is a graphic glitch when in melee. The rider appears to repeatedly die and resurrect instead of striking with sword Looks like both lbms are necessary for Jap Cav to be convincing. Same for Mongol Light Cav. Now the the Mongol Heavy Cav OTOH have only one lbm and no associated txt file...hmmm.

Will do some more experimenting and compare notes.

[RSW]

Sorry for my poor English.
I try to write down my findings in short and correct English.
Feel free to reply or correct my words, thanks !!!

I) 0000h ~ 01FFh 512 bytes, maybe a color table of 256 colors in 16 bits mode.
each BIF file has this area of same size.

II)0200h ~ 0203h, a double word (four bytes), width of this image (v1)
0204h ~ 0207h, a double word, height of this image (v2)
0208h ~ 0209h, a word (two bytes), unknown (v3)
020Ah ~ 020Bh, a word, unknown (v4)
020Ch ~ 020Fh, a double word, number of frames in this BIF (v5)
0210h ~ 0213h, a double word, unknown (always 1) (v6)
0214h ~ 0217h, a double word, the total length of this entry, always 4 * (number of frames) (v7)
0218h ~ 02??h, if 3 frames in this BIF, then we got 12 bytes (000Ch) in v7.
and we got 2 other fields for the first 2 frames' length.

EX: 0214h ~ 0217h 0C 00 00 00 -> 12 bytes for this entry
0218h ~ 021Bh 72 00 00 00 -> length of "first frame" from 0214h (114 bytes).
021Ch ~ 021Fh D7 00 00 00 -> length of "second frame + first frame" from 0214h (215 bytes).
0220h ~ 0223h 0D 00 FF 0A -> from here is the data of this image (first frame)

ps: In this example the data beyond 215 bytes is data of the third frame.
Each frame in one BIF has the same width and the same height.

III)One line of the frame are packed as one packet. (not sure)
The first two bytes (a word) of the packet is the length of
the packet in bytes, including the field of length.

EX: 0220h ~ 0223h 0D 00 FF 0A ->
0224h ~ 0227h 02 14 FA 0A
0228h ~ 022Bh 01 01 04 08
022Ch ~ 022Fh 0F 0D 00 F6

0220h ~ 0221h 0D 00 -> 13 bytes in this packet,
so 0220h ~ 022Ch is the first packet,
0222h ~ 022Ch is the data of this packet. (only 11 bytes)

IIII) I try to draw the data in the BIF file (like a raw data),
but I can't get the same result as I got in game.
So the data of one packet is not the raw data of one line.
Maybe it means how to draw this line.
We need more work to know how to draw one line. (more try and error)

EX: 00 means the pixel is transparent and 00 04 means we got 4 pixels of 00
00 04 -> 00 00 00 00 and so on.
I think there are some special meanings for FF and F6 (F? series).
but I can't figure out what their meanings are at so far.
I need help in this part.

Hope these findings can help to understand the BIF more quickly.
Maybe we can replace the game's BIFs for our own BIFs in near future...^_^

[Whitey]

doesn't this all mean we can have smaller unit animations (espeacially for Kensai) and as such...(runs off in exitement to try something)

[Swoosh So]

Nice one RSW this place needs guys like u thanks alot


Swoosh

Ps:i have no idea what that stuff means but im sure we just got a step closer to erm BIFFing! again thanks alot rsw

[Kraellin]

rsw,

what file were you looking at there? which bif? and i'm assuming from how you stated your results that you were reading this with a hexeditor, yes?

the color data makes a lot of sense, since i'm pretty sure the bif's do some coloring of the .lbm's. but, i'm not sure that the bif's dont just look through the lbm's for rgb numbers and then just replace information based on that. if you look at the unit lbm's you'll notice that there are areas in each of the unit pics that are in gray scale colors. i think those areas are at least one of the areas that are being replaced standardly in the little pics.

another thing we need in this, since many of us are calling things one thing and someone else is calling it another thing is a standardization of the terminology here. when we talk about an .lbm file it is one file that is being used as if it were several smaller files. it's not, of course, but the program is treating it almost as if it were. the lbm's seem to all be composite pictures or, a file that is a series of images all within the same image. some folks refer to 'pages' and i'm not quite sure if they're talking about the overall file or one of the smaller images within the larger file, like the pages of a book. if the large overall image is a book, then the smaller images could be considered pages, but if the overall image is the page then the smaller images would simply be pictures on the page. if you notice some of the text files in textures/men/[unit] you'll see that some of those files even have 'page' within the name. so is that referring to the overall image or the smaller images within the overall? if we're going to communicate coherently to each other about what we find then we shld also establish some common terminology. and even better if we can match up our terminology to what CA is using.

i'm glad to see some folks tearing into this stuff. i'm still playing with maps, but i do do some study on these files and am still interested.

yoshitsune,

there were some things i didnt understand about your findings. one, where did you get the data that the kensai were using nodachi images? and what do you mean by you 'sacrificed them'?

since we dont seem to be entirely certain yeet what all those text files are doing, let me add some possibilities. might just confuse things, but might also help. if we take the .lbm as the base file and that the bif's are essentially overlays on the bif, then there is one text file that seems to correspond to the number of smaller images in an .lbm. the relationship seems to be one line of text for each small image within the .lbm. so, what does that line of text do? the file is always the [unitname.txt] file, not the [unitname_page.txt] file. each line is ALWAYS 6 numbers. is this color data, positioning data, or is this a file for setting up a data base of that .lbm's image information for referencing later? the bif's themselves 'should' have the color data of what is being replaced. and rsw even suggests what data are being used for that. if one changes the data in [unitname.txt] what happens when you play the game. doing that might be a good first step in understanding these txt files. dunno. anyone wanna test that?

K.


------------------
I'm sorry, but i never apologize.

[RSW]

Ya...I use the "Ultra Editor" to open the BIFs. (under campmap/ and textures/Men/ and so on)

Then I try to find the same and different part of them.
In my point of view, the .lbm files is used
in software simulated mode or just a review.
Because I used the monk.lbm to replace the cav.lbm once. But I saw the same thing as b4.
Nothing is changed in hardware mode.Now, I can try to draw out the campmap/armoor_bonus.bif. Yes, it's
just a image file with 3 small images in it.
Every time when we enter the game, the game reads in all the needed bifs, and decodes them as many images. Then when it needs one, it just displays the needed one out...

BTW, I agree with Kraellin in the part of
the /texture/men/*.txt
These txt files is used to control the needed rectangles to display the actions of soldiers in battle.

When I try to draw the texture/men/*.bif,
Each of them consists of 12 frames,
and there are 10~15 small areas in each frame.These small areas just like the .lbm are occupied by the small soldiers' images.

If we cut out the same small areas of the 12 frames, we got one action's animation of one unit.

That's more easy than the Indexed Book method in the view of programming. But they do need the Indexed Book method to draw the colors of the flags on the soldiers' back.