Release of the CUF (font & text rendering) mod tool

[Tellos Athenaios]

I'm back with another update. Pretty big one this time around:

CUF files

• Preliminary support for kerning. This is something that people modding M2TW will probably not give much thought, but this should make the tool compatible with ETW CUF fonts too. Thanks to just at the TWC for pointing out to me how this feature is supposed to work in those files.
• Added the code to resolve the long standing todo of having some translation layer for CUFProperty objects to provide a better mechanism for checking the values you want to set.
• Added CUFSourceEditor which is a class that provides support for a declarative approach to editing individual glyph attributes like advance width and y-offset relative to some sort of baseline. This allows people to fix corner badly offset glyphs by injecting better values (this is mainly an issue with italic fonts where rounding errors do more damage to spacing between glyphs). All standard font classes (deriving from AbstractCUFSource) support this feature: you can use edit() and getEdits() respectively to work with it.

Font previews

• Better caching for glyphs. By removing the temporary solution with a proper cache anything that has a lot of the same characters being used (e.g. "Hello world") should see a significant speed up when a preview is rendered.
• Improved support for editing the CUF properties in the table of the preview window.

CUF Shell:

• Fix to allow prompting for user input from fork'ed scripts. In particular this includes a rewrite of the way the shells in the GUI obtain your commands and write output to the GUI.

Help files (need more work to be a proper reference documentation):

• The tool now contains support for the javax.help (JavaHelp) package which provides a Help browser. In addition the original readme.html file was cut into various pieces which are accessible through this browser (and bundled in dist/cuf-help.jar)
• Added necessary code to deal with the case where no JavaHelp is present (at compile time and/or at run time) so you can still build & use the CUF tool without it.
• Changed build scripts so that the Javadoc ends up as part of the help bundle too, which means you can use the search function of the JavaHelp browser to navigate it.

The Help feature requires JavaHelp, which you can get from here: https://javahelp.dev.java.net/

Misc:

• Rather than using an UnsupportedOperationException() stub, some of the code now throws an NotEditableException() when you attempt to do editing operations on read only data (something which would not work anyway).

[Tellos Athenaios]

Pretty big update:

Features

1. Support for JLine. This provides the program with some readline functionality when it runs directly from the command line.
2. Added a “Fontwizard” GUI which provides a for a simpeler way to produce fonts than by writing your own scripts. You can find it under the Build menu. It is a bit rough round the edges, but
   sufficient for quickly building a font or mocking up something. Use the Export buttons to pop up a dialog for previewing/saving your work.
3. Added more extensive logging capabilities to the program/software through IOHelp.Log.

Restructured code layout
Lots of the auxiliary code has been restructured due to the additions of Fontwizard, JLine support and better logging.

A particular item of interest is that the code which handles program settings has been moved from org.europabarbarorum.cuf to org.europabarbarorum.cuf.support, and
that it is now confined to the Setting class (the Prefs class merged with the Setting interface into the Setting class), and the specific program
settings moved to the modules that actually use them.

The upshot of these changes is that if you have script code which manipulates program settings directly you will need to update it.

Another item of interest are the various restrict() methods in FontToolkit: these require now an addition boolean parameter. Old code which used these operations
should use “true” here (this yields equivalent behavior to previous versions of these restrict() methods). Use “false” to invert the result of the old style
restrict() methods.

Bugs

1. CUFWriter catches a couple more errors you might make and fails with an error message of its own.
   So although the net result will be largely the same, at least you will have a useful error message to work wih.
2. The control character font now sets the number of glyphs it contains, along with a slight change in CUFWriter's behaviour this
   fixes two edge-cases: a) that CUFWriter previously could fail seemingly random (but see above) if no glyphs were declared on the font,
   b) that someone might actually want to use the control character font more directly.
3. Add missing check & default values in CUFRenderPanel when looking up certain CUFProperty settings.
4. Fix Classloader to not load classes that the JVM has already loaded. Prevents doing double work as well as duplicating shared resources.

[Tellos Athenaios]

Important update. This update contains a fix for a bug in the code which generated “keyed” strings.bin files. This fix involves generating proper lookup tables at the end of the file: without those Medieval II/Kingdoms suffer significant performance penalties. Mea culpa, the code that generated the strings.bin files was based off alpaca's bin converter which does not (nor does it need to) deal with this. I would like to thank my fellow EB members who tested and suffered the performance hit for their patience and help in debugging the problem, in particular Atilius and bobbin. Thanks, guys!

Apart from having a fix which brings performance back to something more reasonable, the silver lining here is that this fix also addresses a bug in the Medieval II/Kingdoms files. Apparently the data/text/export_units.txt.strings.bin files extracted from the game packs are not built correctly either (implying that the game does not convert export_units properly). The lookup table in this file does not refer to the source code, rather it contains a dump of 795 entries for the 265 RTW units from RTW Vanilla (795 = 265 *3, and there are 3 keys per unit in export_units data.)

This update also contains a simplification of the strings compiler, so the StringsSource types are no longer required. As a result if you were using StringsSources directly, you will need to update to use StringsFeeders. Furthermore setOrder(boolean) in StringsToolkit is gone and replaced with format(String). You can use “Ordered” where previously you would use “true”.

Furthermore this update incorporates some of the improved understanding of the CUF file format, with regards to certain CUF properties. These are “maximum width” and “maximum height” of glyphs. This means that the suitable values will be automatically inferred by the CUF compiler by default. This behaviour can be controlled/disabled per property through CUFWriter.SetVSize, CUFWriter.SetHSize and CUFWriter.SetGlyphDataSize options: see Javadoc for details.

This behaviour is new for VSize and HSize properties, both of which refer to cut-off values (if a glyph is wider or taller than either value it appears cut off in the game). The code rendering the previews in the tool do not take these values into account yet. For GlyphDataSize property it means you now have the final control over what is written to the CUF file.

[Tellos Athenaios]

There's a new important update available. This one fixes a few serious issues, and one less so.
[list]
New features:

• Add support for getting/setting macro names on CUFSources (MacroSource, AbstractCUFSource, Converter).
• Add support for hiding multiple fonts under a single macro (WrappedSource.MacroSource).
• Add support for exporting macros from the FontWizard (ExportDialog).
• Add support for editing CUF properties from the FontWizard.
• Add support for specifying range of characters in FontWizard.
• Add support for shifting character of fonts in FontWizard.

Bugs:

• Fix reference passing instead of copying arrays in AbstractCUFSource.
• Fix broken format message for missing macros
• Fix editing properties on the CUFSource (font) during previews.
• Fix occasional disabled next/cancel buttons in FontWizard

Misc:

• Merge various FontWizard pages to achieve same effect with less code.
• Move common FontWizard UI to its own package.
• Move create new font pages to own package.
• Move algorithms out of FontToolkit.
• Improve error handling in FontWizard pages when building fonts.
• Improve error handling when formatting localised messages.
• Tidy up a number of error messages.
• Move ComponentState from the fontwizard to the gui.support package.
• Use different labels on FontWizard next/cancel buttons depending on context.

[Tellos Athenaios]

There is another update of the tool. This update changes how sub shells interact with each other. Shells now live in their own thread (as managed by the JVM) rather than as task in a thread pool (as managed on top of the JVM). This update also fixes the long standing bug of output messages appearing out-of-order when running multiple shells in parallel.

The result is a tool which is able to tax the CPU more aggressively when a lot of shells are forked, and thus finish parallel tasks faster. Subjectively, the command prompt feels faster from the GUI, as well.

There are a few other minor changes:

• Added SAXHelp which contains the tools for XML crunching outside of generating strings.bin files, should you need it. (I did.)
• Added Context (right click) menu's on text fields and areas with copy/paste/cut actions (as appropriate). Of course you can also use Ctrl+C,Ctrl+V,Ctrl+X.
• Added alternative PlainKeys mode to FormatOptions for StringsWriter: this version retains the old behaviour where lookup tables would be omitted.

[Tellos Athenaios]

Update of the tool. The update itself is fairly small, but the main changes are worth updating for: a bug squashed and the tool is easier to use.

Main changes

• Update Macro, StringMapping and RawHandler to give better error messages.
• Fixed a bug in FontToolkit.fixChartableCompatibilityFor where it could end up inadvertently dropping characters from the chartable. The method was renamed into fixChartable.
• Use SortedMaps for chartables in the tool (most of the CUFSources already did this, but some used HashMaps). This ensure that the order of characters in a chartable is maintained for instance when compacting a font.

Misc changes

• Move algorithms out of FontToolkit, and together with WrappedSource to the font.impl package. Now there is an AlgorithmSource for these operations. (This change allows the code of the algorithms to be simplified and also makes the code elsewhere a bit neater.)
• Remove some redundancy in the error messages for bad file types.
• Fix cuf-impl.xml Ant script which suffered from bit rot. The update-licence task works again on Ant 1.8.1. This has otherwise no effect on the tool.

[Tellos Athenaios]

I've identified & fixed a new problem with the tool: the ScriptRunner component would kill its host shell too soon meaning that the exit code of the program could be 0 (indicating no errors) even if errors did occur. This is sufficiently serious for me to release a new version of the tool.

This update also contains a few other changes to the tool. The main ones involve the Shell and they are: fork always expects a boolean argument to accompany the CallBack argument (it indicates whether it the CallBack must run in the same thread as the shell on which you call fork), and there is a blockUntilExit method now if you need to block a Shell until another one exits.