PDA

View Full Version : Return of XIDX: with bug fixes and updates for M2TW.



Tellos Athenaios
10-18-2011, 00:01
The Europa Barbarorum Team is proud to announce the return of one of the most venerable modding tools dating back to the early days of RTW: XIDX, with bug fixes and updates for M2TW. The “XIDX” project provides modders with a tool to assemble and disassemble IDX/DAT pairs of files as used in RTW and M2TW (including Kingdoms). Originally developed by Vercingetorix for use with Rome Total War versions of these archives on Windows only, the tool now compiles and runs on Linux as well (and should run on any proper POSIX x86 or AMD64 environment).

The project is based on the source code of the XIDX program as published on the TWC (http://www.twcenter.net/forums/downloads.php?do=file&id=32). It has been heavily modified, the comprehensible change log is as follows:

Features:

Add -m option for building Medieval II versions of supported pack types.
Add -b option for transparent suffix ('.bin') manipulation (useful with skeletons for avoiding name collisions).
Add -e option for assembly/disassembly of event packs.


Bug fixes:

Port/fixes to build on Linux with GCC (tested with various versions >= 4.3.2)
Builds with GCC-Mingw32 and GCC-Mingw64 (tested with various versions >= 4.4.4)
Fix lurking segfaults on exit and inside the fork function of the XIDX shell.
Fix memory allocation, so the tool does not allocate lots of memory it never uses.


Misc:
Strip batch files, MS Visual Studio project file etc., added makefile.

The program is available as both a source (ftp://ftp@europabarbarorum.org/XIDX-src-dist.7z) and a binary (ftp://ftp@europabarbarorum.org/XIDX-bin-dist.7z) distribution from our FTP server. Both distributions are packaged as 7zip archives. Programs such as 7-zip can be used to extract these contents. (You can get 7-zip from here: http://www.7-zip.org/ or commandline equivalent p7zip from here: http://p7zip.sourceforge.net/ )

20555
20556

The program is made available under the GPL v.2 or later, copy of which is provided in licence.txt of either distribution. Below follows the summary:

Terms of Use
This software is free and there is no warranty what so ever. Use at your own risk.
See licence.txt for a copy of the program licence.
Copyright (C) 2004, 2005 Vercingetorix <vercingetorix11 at gmail dot com>
Copyright (C) 2011 The Europa Barbarorum Team <webmaster at europabarbarorum dot com>

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details at
http://www.fsf.org/licensing/licenses/gpl.txt



Comments, criticism, improvements and bug reports are welcome. Regards,

The Europa Barbarorum Team.

Tellos Athenaios
01-17-2012, 20:34
Update with the following bug fixes:

Bugs

Fully zero out the buffers before using them, instead of just assigning '\0' to the first byte. This makes valgrind a bit happier (quieter), which is a good thing.
Fix a bug in sound packing code. A 1 had crept in where a 4 should have been. Ouch.
Some code cleanup to have less code in a few macro's.


Misc

Generate statically linked binaries for Windows. This has the benefit of actually working if the user doesn't have the MinGW version of a couple of DLLs installed on his system (which is likely). The system I used for testing of the previous release masked the problem because these DLLs were present...

Tellos Athenaios
02-24-2012, 21:09
Minor update, containing a few miscellaneous fixes and decided to call it 1.0:

Fix bug in generated file names when extracting event packs caused by not dividing the log(position in the pack) by log(10). This ensures that when you list the files (dir, ls, find) they will appear in the same order as they should be placed in the pack because of how lexicographic sorting works. (Simplifies repacking.)
Fix version reporting.
Some minor improvements to the readme.

Tellos Athenaios
03-26-2012, 20:07
Version 1.1 brings:

Bugfixes
Fix memory leak resulting from exit() in the usage info function.

Misc
Change naming conventions for binary blobs extracted from event (EVT) packs. Numbering now starts at zero for every "frame", rather than just at the beginning of the pack. This change would allow other tools to guess the event (bank) that corresponds to a given file name based on the number before the bin suffix.

Tellos Athenaios
04-15-2012, 21:03
New version available.

Bugfix
Version 1.2 fixes a memory/file descriptor leak when opening archives with the wrong archive type flag. (For example specifying the -a flag for animations packs when trying to list the contents of a sound IDX/DAT pair.)

Tellos Athenaios
10-02-2012, 01:07
New version available:

Bugfixes

Fix non-virtual destructors
Skip packing of EVT blobs which do not conform to the filename conventions used by XIDX.
Fix error messages when packing an EVT blob fails
Improve file name parsing of EVT blobs (no longer limited to files from the current directory).


Feature

Add makefile flag to skip cross compilation (MinGW targets).


Reminder: beware of the data folder
XIDX rewrites paths so they are relative to the root of the mod (or installation if you do not use a mod folder), and it assumes that the "root" data directory is the last "data/" in the path. This is pretty much necessary to make packing work by default on Windows (unless you wish to try you hand at this sort of thing (http://stackoverflow.com/questions/8385454/batch-files-list-all-files-in-a-directory-with-relative-paths), but it does mean that your last "data/" must be the right "data/". Otherwise the path will be written to an invalid value. Example: "mods\EBII\data\sounds\data\SFX\effect.wav" would be rewritten to "data\SFX\effect.wav" instead of "data\sounds\data\SFX\effect.wav". So if the game tries to load "data\sounds\data\SFX\effect.wav" because that is what you put in your code, it would fail. Easy workaround: do not have data folders inside your data folder, at least not in the parts that you want to pack with XIDX.

This message does not apply to packing events, since the tool does not store filenames for those in generate IDX/DAT packs anyway. Note that for events you will want to sort filenames alphabetically so the filename convention for EVT blobs will guarantee that they appear in the right order. Example command for packing EVT blobs:

dir /S /A:-D /B /O:N "path/to/EVT/blobs/directory" "*.bin" | xidx -cbef "path/to/mods/my_mod/data/sounds/events"

Tellos Athenaios
10-13-2012, 17:20
Version 1.4 of XIDX is now available and it contains the following updates:

Features
Add support for creating RTW event packs (add support for the RTW file version stamp)
Extended the Readme with a lot more information on how to correctly generate packs, in particular event packs.


Fixes

Fix file version stamps for event packs (though the game seems to ignore them anyway)
Don't forget the Windows users: extend previous bug fix for sniffing out type numbers to paths with backslashes in them as well.

Tellos Athenaios
04-06-2013, 15:57
Version 1.5 of XIDX is now available.

Features
Add -B option for arbitrary suffix manipulation. The -b switch is now equivalent to using `-B bin'


Fixes
Fix entry size in generated animation packs when using suffix manipulation

Tellos Athenaios
03-24-2014, 00:04
Version 1.6 of XIDX is now available. It contains:

Features
Add -P and -p options for disabling implicit path mangling.


Bug fixes

Disable scaling for M2TW animation archives.
Write skeleton names instead of paths by default. (May be overriden using -P or -p.)


Misc

Add extra warnings and mention default file paths in --help output.
Add note on correctly packing skeletons, animation CAS file format(s) conversion.

Tellos Athenaios
03-24-2014, 12:43
Version 1.6.1 of XIDX is a point release addressing the following issues:

Bug fixes

Fix Windows builds to not complain about libwinpthread-1.dll (-static).
Fix -h option.
Report first of -p or -P as not affecting extracting/listing archives.

Tellos Athenaios
05-30-2014, 12:50
Version 1.6.2 of XIDX is available, bringing the following changes

Bug fixes

Fix build error on MINGW/GCC 4.9


Misc

Attempt to make XIDX print somewhat useful output to stdout and stderr when taken separately (useful when applying I/O redirection)

Varangon
02-01-2018, 03:40
Sorry for the necro, but I can't seem to find the XIDX download file. Can anyone provide it, please?

Ludens
02-01-2018, 10:55
Sorry for the necro, but I can't seem to find the XIDX download file. Can anyone provide it, please?

The download links go to the old EB site, but they reorganized their website sometime last year. Even the download link on the TWC is dead. Maybe ask one of the currently active EB members (https://forums.totalwar.org/vb/forumdisplay.php?235-Europa-Barbarorum-II)?

therother
02-01-2018, 22:01
Varangon, the links in the OP are still active.

Source: ftp://ftp@europabarbarorum.org/XIDX-src-dist.7z
Binary: ftp://ftp@europabarbarorum.org/XIDX-bin-dist.7z

Varangon
02-02-2018, 04:00
therother Thanks for the links, Theroter, but it says "This site can’t be reached europabarbarorum.org took too long to respond."

Is there another working link? Or could anyone provide me with this software?

Varangon
02-02-2018, 04:16
Ludens Thanks for the info, Ludens, I will try to do so.

Ludens
02-02-2018, 08:58
"This site can’t be reached europabarbarorum.org took too long to respond."

I had the same problem when trying to download it, yesterday.

therother
02-02-2018, 09:42
@therother (https://forums.totalwar.org/vb/member.php?u=1535) Thanks for the links, Theroter, but it says "This site can’t be reached europabarbarorum.org took too long to respond."
Sorry about that, firewall issue. Should be fixed now.

If not, please let me know and in the meantime you can download from TWC:http://www.twcenter.net/forums/showthread.php?488988-Return-of-XIDX-with-bug-fixes-and-updates-for-M2TW

Edit: I've also now attached them to the OP.

Varangon
02-02-2018, 10:25
Thank you very much! :D Saved a lot of my time! :D