Dol Guldur
08-07-2009, 13:45
Text Manipulation - Tips, Tricks and Tools
(Non-Platform-/Non-Game-Specific Resource)
Having coded RTW mods for a number of years I have picked up some tips and tricks for text manipulation along the way which has made life a lot easier. This post is to share some of the tools and techniques I've used. If you have other tips then please share them in this thread too. This post is not intended to be an exhaustive tutorial.
What I will address here are some resolutions to some of the deeper pains of text editing. All of the tools are free except for Excel (but most wil have that on their PC anyway).
1. How To Global Replace in Multiple Files
Tool: Crimson Editor
Link: http://www.crimsoneditor.com
Crimson Editor has become my number one modding tool since I started using it in late 2005/early 2006. I spent the first year using Notepad and wish I had found this earlier. It does so much. Currently a new version is being developed called EmeraldEditor which will no doubt do even more! There's an entry for CE on Wikipedia should the link above ever expire.
CE is a tabbed editor, meaning it can have more than one file open at a time. When you click the Search | Replace option from the drop down menus a dialogue box will open and you can change the default radio-button option to "All open files". CE will then replace your search term with your new word or phrase in ALL the files open in CE. This can save a ton of time when, for example, changing internal names of regions. You just open up every file that may refer to the internal name of a region and Bob's your uncle ;) As with all text manipulation you need to be savvy about it. You might, for example, choose to activate the "Match case" option when wanting to differentiate between internal and external names, or include curly brackets in the search term for some files so as to target only the internals for files that use curly brackets.
Notepad++, I am told, also has this functionality.
2. How to Join Lines
Tool: Crimson Editor
Link: http://www.crimsoneditor.com
I felt very stupid when I found this. I had been doing it manually for years! Sometimes you will want to include text in your mod and the lines will be broken rather than wrapped. CE's Edit | Advanced | Join Lines feature will get rid of these swiftly. Just place the cursor at the beginning of the block of text and then hit ALT J (you can hold it down to go faster) and it will repidly remove all line breaks.
3. How to Concatenate
Tool: Excel
Link: n/a
This can be useful for creating internal curly-bracketed tags once you have your "lookup" list. This is very easily done by Concatenation. Let's say you have George in cell D31. In A31 (or wherever you want, though I'd recommend this 2-column gap as it will copy and paste the right number of tab spaces into the names.txt file) you could type: =CONCATENATE("{",D31,"}") and this will cause {George} to display. Then just highlight the entire D column and copy the formula down and your internals are done!
4. How To Compare Changes
Tool: BeyondCompare3
Link: http://www.scootersoftware.com
This is very useful for Build Managers - those who maintain the current build of a mod in develpment. It has the ability to compare folders and files so you can compare not only different files but also entire versions of builds. It's quite intuitive to use. It colour highlights differences between files and folders. Howevev, it is not just a file/folder-comparision utility, it also allows you to merge or copy over parts of text from one file to another to "sync" them etc. Extremely useful for debugging ;)
Another free compare tool is WinMerge: http://winmerge.org/ but I have not used it so cannot comment.
5. How To Perform Multi-Line Editing
Tool: ElitePadLite
Link: http://www.editpadlite.com/editpadlite.html
This had me beat for a long time until I found this tool. Have you ever wondered how to search and replace terms than are on multiple lines but which are not unique? For example, some files list culture blocks like this:
variant
{
cultures
{
barbarian
nomad
hun
}
How would you go about adding a new culture to the end such as "celtic":
variant
{
cultures
{
barbarian
nomad
hun
celtic
}
You'd have to do it manually with nearly all free text editors. That's where ElitePadLite comes in! This is not entirely intuitive though so here's how to do it:
1. Open the Search Panel (Search | Show Search Panel) - it appears at the bottom, under the main window
2. Paste in the upper window of the Search Panel the lines of the text block to be searched for
3. Type what you want to add/change in the main window and then copy that to the lower window of the Search Panel
4. Press the Replace All button of the Search Panel
This is a powerful tool. For example, you can now add a building to all settlements in the descr_strat.txt file with one click of a button if you so chose!
6. How To Batch Rename File Names
Tool: LupusRename
Link: http://rename.lupasfreeware.org/download.php
This utility will allow you to mass-batch edit filenames in any selected folder. It has quite a lot of options for detailed renaming. This can, for example, rename all files beginning with "#roman_" to begin with "#barbarian_". Essential when setting up new cultures among other things!
7. Regular Expressions (Complex Replacing)
Tool: Crimson Editor
Link: http://www.crimsoneditor.com
Tool: Notepad++
Link: http://notepad-plus.sourceforge.net/uk/site.htm
If you want to replace a specific re-occuring element within variable other elements then use aqd's "regular expression" method in post #11 of the TWC version of this tutorial. This is extremely useful. You can find out more about regular expressions at http://www.regular-expressions.info/
(Non-Platform-/Non-Game-Specific Resource)
Having coded RTW mods for a number of years I have picked up some tips and tricks for text manipulation along the way which has made life a lot easier. This post is to share some of the tools and techniques I've used. If you have other tips then please share them in this thread too. This post is not intended to be an exhaustive tutorial.
What I will address here are some resolutions to some of the deeper pains of text editing. All of the tools are free except for Excel (but most wil have that on their PC anyway).
1. How To Global Replace in Multiple Files
Tool: Crimson Editor
Link: http://www.crimsoneditor.com
Crimson Editor has become my number one modding tool since I started using it in late 2005/early 2006. I spent the first year using Notepad and wish I had found this earlier. It does so much. Currently a new version is being developed called EmeraldEditor which will no doubt do even more! There's an entry for CE on Wikipedia should the link above ever expire.
CE is a tabbed editor, meaning it can have more than one file open at a time. When you click the Search | Replace option from the drop down menus a dialogue box will open and you can change the default radio-button option to "All open files". CE will then replace your search term with your new word or phrase in ALL the files open in CE. This can save a ton of time when, for example, changing internal names of regions. You just open up every file that may refer to the internal name of a region and Bob's your uncle ;) As with all text manipulation you need to be savvy about it. You might, for example, choose to activate the "Match case" option when wanting to differentiate between internal and external names, or include curly brackets in the search term for some files so as to target only the internals for files that use curly brackets.
Notepad++, I am told, also has this functionality.
2. How to Join Lines
Tool: Crimson Editor
Link: http://www.crimsoneditor.com
I felt very stupid when I found this. I had been doing it manually for years! Sometimes you will want to include text in your mod and the lines will be broken rather than wrapped. CE's Edit | Advanced | Join Lines feature will get rid of these swiftly. Just place the cursor at the beginning of the block of text and then hit ALT J (you can hold it down to go faster) and it will repidly remove all line breaks.
3. How to Concatenate
Tool: Excel
Link: n/a
This can be useful for creating internal curly-bracketed tags once you have your "lookup" list. This is very easily done by Concatenation. Let's say you have George in cell D31. In A31 (or wherever you want, though I'd recommend this 2-column gap as it will copy and paste the right number of tab spaces into the names.txt file) you could type: =CONCATENATE("{",D31,"}") and this will cause {George} to display. Then just highlight the entire D column and copy the formula down and your internals are done!
4. How To Compare Changes
Tool: BeyondCompare3
Link: http://www.scootersoftware.com
This is very useful for Build Managers - those who maintain the current build of a mod in develpment. It has the ability to compare folders and files so you can compare not only different files but also entire versions of builds. It's quite intuitive to use. It colour highlights differences between files and folders. Howevev, it is not just a file/folder-comparision utility, it also allows you to merge or copy over parts of text from one file to another to "sync" them etc. Extremely useful for debugging ;)
Another free compare tool is WinMerge: http://winmerge.org/ but I have not used it so cannot comment.
5. How To Perform Multi-Line Editing
Tool: ElitePadLite
Link: http://www.editpadlite.com/editpadlite.html
This had me beat for a long time until I found this tool. Have you ever wondered how to search and replace terms than are on multiple lines but which are not unique? For example, some files list culture blocks like this:
variant
{
cultures
{
barbarian
nomad
hun
}
How would you go about adding a new culture to the end such as "celtic":
variant
{
cultures
{
barbarian
nomad
hun
celtic
}
You'd have to do it manually with nearly all free text editors. That's where ElitePadLite comes in! This is not entirely intuitive though so here's how to do it:
1. Open the Search Panel (Search | Show Search Panel) - it appears at the bottom, under the main window
2. Paste in the upper window of the Search Panel the lines of the text block to be searched for
3. Type what you want to add/change in the main window and then copy that to the lower window of the Search Panel
4. Press the Replace All button of the Search Panel
This is a powerful tool. For example, you can now add a building to all settlements in the descr_strat.txt file with one click of a button if you so chose!
6. How To Batch Rename File Names
Tool: LupusRename
Link: http://rename.lupasfreeware.org/download.php
This utility will allow you to mass-batch edit filenames in any selected folder. It has quite a lot of options for detailed renaming. This can, for example, rename all files beginning with "#roman_" to begin with "#barbarian_". Essential when setting up new cultures among other things!
7. Regular Expressions (Complex Replacing)
Tool: Crimson Editor
Link: http://www.crimsoneditor.com
Tool: Notepad++
Link: http://notepad-plus.sourceforge.net/uk/site.htm
If you want to replace a specific re-occuring element within variable other elements then use aqd's "regular expression" method in post #11 of the TWC version of this tutorial. This is extremely useful. You can find out more about regular expressions at http://www.regular-expressions.info/