Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

[Tellos Athenaios]

Update:

Fixes

• Fix missing include (stdio.h) in util.h
• Fix bad handling of the output pointer at slurp(), in util.c. Fortunately, in the case of the PACK tool this bug would not "manifest" itself during runtime.
• Fix opening sentence in the ReadMe, proving nobody ever reads those.

[Tellos Athenaios]

Another update. This one fixes a bug which caused the tool to crash (due to segfault) when unpacking uncompressed data (chunks).

[Tellos Athenaios]

Version 0.7 of the PACK tool is available now. It brings some major improvements to the tool to make your life a lot easier when using it.

Features

• Maintain a consistent ordering of the files in the PACK. Previously this depended on the order in which files are supplied, but it appears that Medieval II: Total War wants them sorted.
• Avoid packing duplicate files. Previously pack -a some.pack duplicate.txt duplicate.txt would pack the file duplicate.txt twice, no longer. It prints a warning about the second duplicate.txt instead. Duplicates are not removed if they already exist in the pack, however.
• Parsing logic for -t, --text, -0 and --xargs options is now more lenient. Empty strings (lines) are silently discarded instead of triggering a failure to pack. Closing delimiters for the last entry in the parsed file are no longer required, either.

Bug fixes

• Fix memory access warning in the parsing logic which deals with the -t/--text options.

Misc

• Output of the -l, --list commands is now sorted and in-order as well.

[Tellos Athenaios]

Version 0.8 brings the following new features:

Features

• Add -z/--no-compression and -s/--small-files options to disable compression. The --small-files option is meant to mimick behaviour of "Medieval II: Total War" PACK files, it disables compression for files of <= 1024 bytes (1kiB).
• Picked a more aggressive compression algorithm (lzo1x_999 vs lzo1x_1), yielding smaller results at the cost of (significant) extra run time and memory consumption while packing.

Misc

• Use the 'safe' LZO decompression algorithm variant which performs sanity checking of arguments passed by the pack program. This should make the program resilient against malformed input data. Also added some logic to print additional error messages indicating the specific error condition when compression/decompression fails.

[Tellos Athenaios]

Turns out there is a nifty optimisation feature built in to the LZO library. Let's use it, in version 0.85 of the PACK tool:

Features

• Use the LZO optimisation feature. This 'optimises' compressed data to make decompression slightly faster (1-3% according to the documentation).

Misc

• Add a section to the readme on compatibility with "Medieval II: Total War"

[Tellos Athenaios]

There is a bugfix for the LZO library used by the PACK tool available which addresses an issue that could be a potential security concern depending on how the library is used. The PACK tool appears not to be affected by this bug because it never attempts to decompress data that is large enough (16MiB) to trigger the bug.

Nevertheless I'd like to update the PACK tool to use the latest LZO. I ran into an issue trying to cross compile it using MinGW (to produce Windows builds of the library), however. As a result I am not able to produce Windows builds of the PACK tool with LZO 2.07 for now. This means that I'll have to hold off on this update to the PACK tool until the issue is resolved. In the meantime, I have filed a bug report.

Normal compilation of LZO using the lzo makefile still works fine. Since the update does not require modification to the PACK sources (only a rebuild with the new LZO is required), users wishing to upgrade to the latest LZO are advised to rebuild the PACK tool from source for now, until a proper update is available.

[Tellos Athenaios]

Version 0.86 of the PACK tool incorporates the latest version of LZO which address the build issue.

Misc

• Bump LZO version to 2.08.

[Gigantus]

I am such an utter noob when it comes to this command line stuff. What would be the entry in a batch file to extract all animation files from a custom pack file? (Medieval II\...\mymod\data\animations\pack.idx)
Batch file and IDX file in same folder as the pack.exe - I have got a 64bit system so I am assuming it's pack_amd64?