Results 1 to 12 of 12

Thread: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

  1. #1

    Default Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    Hereby I present the first release of the PACK modding tool, and new addition to the growing collection of modding tools created by the Europa Barbarorum team. This tool is the result of me converting johnwhile's research into the PACK file format into a usable little tool which has the following features (usage info):
    Code:
    pack <command> [opts] -- [<PACK> [file...]]
    pack <command> [opts] [<PACK> [file...]]
    
    Where <PACK> is the PACK file to operate on and pack the pack program
    
    Commands:
       -a, --add              :  add files to PACK file. If <pack> does not exist
                                 it will be created.
       -c, --create           :  create a new PACK file, overwriting <pack> if it
                                 already exists. --no-clobber is not supported.
       -h, --help             :  display this help
       -l, --list             :  list files contained by a given PACK file
       -r, --remove           :  remove files from PACK file
       -x, --extract          :  extract files from an existing PACK file
       -v, --version          :  display version information
    
    Options:
    
       -n, --no-clobber       :  do not overwrite existing files. When extracting 
                                 files from a PACK file this option guards against 
                                 overwriting files in the target directory; when 
                                 adding files to an existing PACK it prevents 
                                 overwriting files already present in the PACK file.
       -i, --case-insensitive :  [file...] is case insensitive, this option only 
                                 applies to matching files located in the PACK
                                 file   
       -g, --glob             :  [file...] may contain wildcards ('*' and '?'),
                                 this option only applies to matching files 
                                 located in the PACK file.
       -d, --details          :  list details about the contents of a PACK file
       -t, --text             :  read file names from given text file(s), when 
                                 creating a PACK file
       -0, --xargs            :  read NULL delimited file names from given file(s),
                                 when creating a PACK file
    The program is available as both a source and a binary distribution from our FTP server. Both distributions are packaged as 7zip archives. Programs such as 7-zip can be used to extract these contents. (You can get 7-zip from here: http://www.7-zip.org/ or commandline equivalent p7zip from here: http://p7zip.sourceforge.net/ )

    The program is made available under the GPL v.2 or later, copy of which is provided in licence.txt of either distribution. Below follows the summary:

    Terms of Use
    This software is free and there is no warranty what so ever. Use at your own risk.
    See licence.txt for a copy of the program licence.

    Copyright (C) 2012 The Europa Barbarorum Team <webmaster at europabarbarorum dot com>

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
    GNU General Public License for more details at
    http://www.fsf.org/licensing/licenses/gpl.txt

    Comments, criticism, improvements and bug reports are welcome. Regards,

    The Europa Barbarorum Team.
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  2. #2

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    New version, which brings a few code simplifications. Features & behaviour remain the same.
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  3. #3

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    New release. Fixed a crash when creating directories on Windows.
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  4. #4

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    New release, this one fixes some inverted logic (if/else) which causes packing certain files to fail. To trigger the bug you would have to have a file for which only the last "chunk" of data is incompressible.
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  5. #5

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    Update:

    Fixes
    • Fix missing include (stdio.h) in util.h
    • Fix bad handling of the output pointer at slurp(), in util.c. Fortunately, in the case of the PACK tool this bug would not "manifest" itself during runtime.
    • Fix opening sentence in the ReadMe, proving nobody ever reads those.
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  6. #6

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    Another update. This one fixes a bug which caused the tool to crash (due to segfault) when unpacking uncompressed data (chunks).
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  7. #7

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    Version 0.7 of the PACK tool is available now. It brings some major improvements to the tool to make your life a lot easier when using it.

    Features
    • Maintain a consistent ordering of the files in the PACK. Previously this depended on the order in which files are supplied, but it appears that Medieval II: Total War wants them sorted.
    • Avoid packing duplicate files. Previously pack -a some.pack duplicate.txt duplicate.txt would pack the file duplicate.txt twice, no longer. It prints a warning about the second duplicate.txt instead. Duplicates are not removed if they already exist in the pack, however.
    • Parsing logic for -t, --text, -0 and --xargs options is now more lenient. Empty strings (lines) are silently discarded instead of triggering a failure to pack. Closing delimiters for the last entry in the parsed file are no longer required, either.


    Bug fixes
    • Fix memory access warning in the parsing logic which deals with the -t/--text options.


    Misc
    • Output of the -l, --list commands is now sorted and in-order as well.
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  8. #8

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    Version 0.8 brings the following new features:

    Features
    • Add -z/--no-compression and -s/--small-files options to disable compression. The --small-files option is meant to mimick behaviour of "Medieval II: Total War" PACK files, it disables compression for files of <= 1024 bytes (1kiB).
    • Picked a more aggressive compression algorithm (lzo1x_999 vs lzo1x_1), yielding smaller results at the cost of (significant) extra run time and memory consumption while packing.


    Misc
    • Use the 'safe' LZO decompression algorithm variant which performs sanity checking of arguments passed by the pack program. This should make the program resilient against malformed input data. Also added some logic to print additional error messages indicating the specific error condition when compression/decompression fails.
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  9. #9

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    Turns out there is a nifty optimisation feature built in to the LZO library. Let's use it, in version 0.85 of the PACK tool:

    Features
    • Use the LZO optimisation feature. This 'optimises' compressed data to make decompression slightly faster (1-3% according to the documentation).


    Misc
    • Add a section to the readme on compatibility with "Medieval II: Total War"
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  10. #10

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    There is a bugfix for the LZO library used by the PACK tool available which addresses an issue that could be a potential security concern depending on how the library is used. The PACK tool appears not to be affected by this bug because it never attempts to decompress data that is large enough (16MiB) to trigger the bug.

    Nevertheless I'd like to update the PACK tool to use the latest LZO. I ran into an issue trying to cross compile it using MinGW (to produce Windows builds of the library), however. As a result I am not able to produce Windows builds of the PACK tool with LZO 2.07 for now. This means that I'll have to hold off on this update to the PACK tool until the issue is resolved. In the meantime, I have filed a bug report.

    Normal compilation of LZO using the lzo makefile still works fine. Since the update does not require modification to the PACK sources (only a rebuild with the new LZO is required), users wishing to upgrade to the latest LZO are advised to rebuild the PACK tool from source for now, until a proper update is available.
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  11. #11

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    Version 0.86 of the PACK tool incorporates the latest version of LZO which address the build issue.

    Misc
    • Bump LZO version to 2.08.
    - Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool - EB Install Guide - How to track down loading CTD's - EB 1.1 Maps thread


    ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  12. #12
    I know the vioces aren't real Member Gigantus's Avatar
    Join Date
    Jun 2011
    Posts
    4,839

    Default Re: Release of the PACK tool (for packing/unpacking/repacking M2TW PACK files)

    I am such an utter noob when it comes to this command line stuff. What would be the entry in a batch file to extract all animation files from a custom pack file? (Medieval II\...\mymod\data\animations\pack.idx)
    Batch file and IDX file in same folder as the pack.exe - I have got a 64bit system so I am assuming it's pack_amd64?



Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Single Sign On provided by vBSSO