Release of the CUF (font & text rendering) mod tool

**Tellos Athenaios** · 05-12-2010, 20:46

The Europa Barbarorum Team hereby releases the results of its efforts to open up the CUF file format for modding. The “CUF” project provides modders with tools to create their own CUF fonts as well as incorporate basic support for “rich text” in their mods. Originally developed for manipulating the CUF fonts bundled with these games, the “CUF” project has progressed so far we believe that others would want to use this tool for their own purposes as well. A screenshot or two may help to explain why we feel that such confidence is justified:

The tool allows you to convert TrueType or Type1 fonts to a CUF file; as well as make more advanced use of the strings.bin format in tandem with the CUF format to manipulate the way descriptions are rendered and produce multiple font styles alongside each other in the same “body” of text. The screenshot above shows how that might look.

The project can be downloaded from our FTP server right here: ftp://europabarbarorum.org/CUF-src-dist.7z. The download consists of a single compressed 7zip archive containing:

Java source code (requires Java 1.6 JDK to build) and resources
A COPYRIGHT file containing the licence under which the project is made available (see below)
A readme.html file containing instructions for building from source, running it and using it to get some modding done. The file also contains links to download additional required software: it is very much recommended that you read this file.
Ant build scripts to make building from source easy (requires Ant 1.7.1 or higher to build)
A POSIX sh script which you can use as example for creating your own scripts that run the program under specific settings. Note that it is not required to use such a script; also note that it is not a batch file.

Programs such as 7-zip can be used to extract these contents. (You can get 7-zip from here: http://www.7-zip.org/ or commandline equivalent p7zip from here: http://p7zip.sourceforge.net/ )

People who do not want to build from source can download the following compressed 7-zip archive instead: ftp://europabarbarorum.org/CUF-bin-dist.7z. This version contains:

The tool in the form of a single JAR file (dist/CUF.jar)
The help bundle (dist/cuf-help.jar). JavaHelp is required to use it, you can get JavaHelp from here: https://javahelp.dev.java.net/
A COPYRIGHT file containing the licence under which the project is made available (see below)
A readme.html file containing instructions for building from source, running it and using it to get some modding done. The file also contains links to download additional required software: it is very much recommended that you read this file.
A sample lib/nblibraries.properties file you can use to configure the tool to link against additional libraries (JAR files) at start up. This may be necessary for using the Help feature (depending on how you install JavaHelp).
Code documentation for use as reference material when building more complex scripts to automate the tool. (dist/help/javadoc)

Both versions (source and binary only) are made available under the following licence:

Copyright (c) 2010, The Europa Barbarorum Team
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of The Europa Barbarorum Team nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL The Europa Barbarorum Team BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Our knowledge of the CUF file format is by no means infallible; as is evident from the screenshot marking most CUF properties as `purpose unknown'. This obviously reflects on how the end result looks compared to a preview generated by the tool: while the preview is roughly accurate, it is necessarily a simplified render and thus does not reflect the precise rendering method of the game engine. Consequentially precise text alignment is at the moment somewhat hit & miss and best results come from using font sizes that correspond with the fonts used by CA.[*] Further research in the details of CUF font rendering as it is performed by the M2TW engine will be required to make the results more robust in this regard.

Comments, criticism, improvements and bug reports are welcome. Regards,

The Europa Barbarorum Team.

* Note that alignment does not refer to how individual glyphs align with each other; rather it refers to how a line of text aligns to its bounding box (e.g. how a label aligns on a button).

**Tellos Athenaios** · 06-18-2010, 20:58

Pushed out an update to the files on the EB FTP server (since the links in post #1 point to those, these are automatically updated as well).

A summary of the changes:

New Features

Use escape characters as a more robust alternative to source indentation or newlines which means less risk of damage by braindead text-editors and other ills. Also allows you to type in multi-line preview texts.
Tab base text alignment and text flow calculations. The tool is now able to generate Strings files which horizontally align pieces of text on individual lines.
FontStyle: removes the need to know about lower level Java packages in order to be able to work with font attributes (e.g. shape width and stroke weight, underline and strikethrough)
Support listening for compile events so it becomes easier to write the glue to manage scripts run through fork.
Made the Strings compiler a more modular structure so it becomes possible to compile Strings from other things than XML files.
Added a few different StringSources, the most notable of which is the EditorSource which allows you edit an existing Strings file.
(Improved) support for order strings files. Previously the only format that could be read & written was something I call “keyed”: it stores string keys and values, and it is used for most descriptions (e.g. export_units.strings.bin). However some files use a slightly different Strings file format, which omits the keys. The CUF program/libraries now support this variation, too.

Bugfixes

Fixed a fromFile() method which returned null when loading system fonts.
Fixed the same fromFile() method to strip the font family attribute from the delta map it uses. This is required for correctly handling font files that are not installed in the system environment.
Report script file names correctly when an error occurs.
Fixed some outdated Javadoc, added some more for previously undocumented elements.
Fixed misleading error message from SAXHandler. This bug would not ordinarily manifest itself in the previous version of the tool because the conditions for it to occur were not yet supported anyway... However with the introduction of tab support/layout you can now make additional errors which highlighted this bug during testing.

Misc changes

Changed structure of strings package. There is now a strings.impl package as a result of making the Strings compiler more modular.
Moved some parsing code around to enable better re-use, and improve modularity.
Changed CUFSource interface so it no longer requires you to work with ChartableEntry objects, if you don't want to. (Of course the built-in implementations still do.)
Enabled cuf.sh script to work when invoked from arbitrary locations.
FontToolkit/Shell/StringsToolkit etc. use string pathnames rather than java.io.File objects. The pathnames still expand as if processed with shell.pathParser().parsePath().
Some restructuring/fat-removal in the internals of the shell package. This should not affect existing scripts because existing scripts have about zero use for those internals anyway!
Changed how compilation is done. The StringsToolkit/FontToolkit now return an object which encapsulates a compilation task, rather than scheduling this task directly. This is required to take advantage of the ability to listen for compile events.
Removed FontSnapshot, it no longer had a purpose.
Updated readme.html file.

... and anything I might have forgotten. Also a screenshot that showcases tab support:

**Tellos Athenaios** · 06-29-2010, 00:08

Updated the tool with a fix to ensure order of records read from strings.bin files is preserved; added option to warn about empty strings in records.

Changed the compression method of the download, to XZ which promises better compression.

The binary-only download is now an XZ compressed archive, too: you can find the JAR file inside it as dist/CUF.jar.

Spoiler Alert, click show to read:

**Tellos Athenaios** · 07-14-2010, 23:32

Another update, another changelog. This one brings:

Added a dialog to compile Strings files from XML.
Added a dialog to preview Strings files, and added another to preview CUF files.
When compiling Strings files can specify an XSLT stylesheet to generate the actual compiled XML from a different XML format. People who want to simplify their XML code and know a bit of XSLT will appreciate the possibilities. (For those who do not know what XSLT is: w3schools.com has an excellent introduction to it.)
When compiling Strings files you can specify character encodings to use when the (XML) data is read & fed to the compiler. This is useful for people who insist on not using UTF-8, especially when you must work with UTF-16LE encoded files. (Notepad calls that Unicode.)
Fix for num entries field in strings.bin files generated.
Fix editControlCharacters() method in FontToolkit to work properly with fonts that have been run through compact() in FontToolkit first.
Improved exit handling means that fork'ed shells will run properly if you attempt to exit the program before they have started yet.
Misc fixes to show 1-based indices instead of 0-based ones in GUI labels and the like.
Compile with debug info by default.

**Tellos Athenaios** · 08-01-2010, 10:02

Updated the tool again, this time it contains the following changes:

Improved support for Windows:

Build scripts use native2ascii which means compiling the tool from source is considerably easier.
Tool forces the use of UTF-8 encoding internally, which fixes some visual glitches on Windows.
Added a Windows Script Host launch script, written in JScript. This means that unless Windows Script Host has been explicitly disabled or something similar, you can use that script on Windows the same way you can use the POSIX sh script with a sh interpreter.

Other changes:

Added filtering of output for some progress messages: turning the “ChattyShell” preference to off (false), and the number of progress messages will be considerably reduced for large fonts.\
Improved character encoding handling when compiling Strings files: due to some internal changes you can now leave encoding fields blank and let underlying parsers “guess” or (“detect”) the right character encoding in use. One particular benefit of this is that byte order marks are now mostly harmless in XML documents if you do not specify a character encoding yourself (previously you could get a content not allowed in prolog message or similar).
Added dialogs to the CUF shell GUI to run scripts.
Added support for wrapping multiple URI-resolvers in a single one, which is useful in conjunction with a KeyResolver.
Added support for compiling M2TW-style tagged files. Note that it is not at all as robust as compiling from XML files. In StringsToolkit, use the compile() method which takes input file name, output file name, a KeyResolver and a boolean flag to use this feature.
Added IdentityMapping (a kind of StringMapping) which allows you to compile without macros (Macro files): it simply passes its input text on to the other layers of the compiler.