View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0006543 | The Dark Mod | GUI | public | 10.06.2024 18:06 | 13.11.2024 21:00 |
Reporter | Geep | Assigned To | |||
Priority | normal | Severity | normal | Reproducibility | N/A |
Status | new | Resolution | open | ||
Product Version | TDM 2.12 | ||||
Summary | 0006543: Change TDM codemap to add 2 new characters | ||||
Description | The current TDM char set has a redundant treatment of 2 characters: Ô appears at both 0x88 and 0xD4 ô appears at both 0x98 and 0xF4 It is proposed to end this redundancy, allowing 2 new characters, namely Ğ/ğ, with breve (cup), where Ğ uses TDM codepoint 0x88 ğ uses TDM codepoint 0x98 The Stone 24pt revision underway for TDM 2.13 now includes these glyphs and DAT changes. Additional steps needed for TDM 2.13 are: 1) Revise the wiki Charset page 2) Alter the DAT files of all existing fonts/sizes (except Stone 24pt) to handle missing breve accent, as follows: codepoint 0x88 in DAT --> glyph for ASCII G codepoint 0x98 in DAT --> glyph for ASCII g Since this is just a copy/paste operation, it should not take long per DAT file. 3) Alter 4 language-specific codepoint maps (tdm_base01\strings\czech.map, hungarian.map, polish.map, and slovak.map) to remove these transformations: source target ===== ===== 0x88 0xD4 // Ô 0x98 0xF4 // ô This will allow utilization of Ğ/ğ in Stone 24pt (and any future font upgrade that includes it.) 4) Include (2) and (3) in the 2.13 betas. | ||||
Steps To Reproduce | N/A | ||||
Additional Information | You can assign this work to me, except for (4). For more, including role of Ğ/ğ and alternatives in 8859-x standards and TDM-supported languages, see: https://forums.thedarkmod.com/index.php?/topic/22427-analysis-of-212-tdm-fonts/&do=findComment&comment=494855 Additional concerns - A) It is possible that some translators of the 4 languages used the 0x88/0x98 codepoints instead of 0xD4/0xF4 directly. The symptom would be - after change (3) above - "G" now appearing where a "Ô" was expected (and likewise lower case). This could affect: -- strings in the main menu system -- strings in a particular i18n FM (This is probably a low-level post-2.13 concern) Translator help may be needed. The fix presumably involves revision of all.lang and regeneration of the czech.lang file (etc), ideally by someone with a 8859-2 codepage PC. B) Existing main menu fonts don't extend beyond "top 50 European characters". Most FM fonts don't extend beyond ASCII. This doesn't address that larger problem. Short-term, it mainly is for the benefit of Stone 24 pt and its potential uses. | ||||
Tags | No tags attached. | ||||
Item (1) has now been addressed. | |
Item (2) is queued up, but it uses refont, and I need to further improve warning messages in refont first. | |
Here's a status report about work on part (2) of planned font conversion to use G-Breve. At this stage, refont was used to generate first-draft REF files from the full corpus of TDM font DAT files. As part of this effort, refont's problem detection was iteratively improved, to give better minor and major warnings of problems. An overview of findings follow, with categorization into 3 buckets, as to how ripe the REF files are for continuation to the next stage. As for refont itself, improvement of v 1.2 is on-going. Particularly problematic fonts are mason and mason_glow. These 48-pt-only fonts use per-character scaling that none of the other fonts do. Refont currently doesn't much support this, and won't for v 1.2; maybe later. Early candidates ---------------- These fonts gave no minor or major warnings as their DAT files were read by refont. They can be converted now by hand (e.g., hand-edit REF, then --> DAT). [Maybe do some by hand, then, once understanding of cases is reached, automate it.] 12 pt size: all except mac_humaine 24 pt size: lotharus [and presumably current 2.13 alpha of stone] 48 pt size: bamberg, carolingia, chrishand, ellianerelle, everett, gothica2, jd_hand, lotharus, medusa, nancy, popsies, rapscallionpirate, shoppinglist, summertime Midterm candidates ------------------ These gave only minor warnings. A minor warning indicates imprecision-from-integer that is less than 0.001. Extension of refont to optionally auto-correct these imprecise values (of which there are typically lots) is desirable. Then do G-Breve conversion as with early candidates. 12 pt size: mac_humaine 24 pt size: andrew_script, bamberg, carolingia, chrishand, ellianerelle, everett, gothica2, jd_hand, mac_humaine, medusa, nancy, popsies, shoppinglist, summertime 48 pt size: andrew_script, mac_humaine Last candidates --------------- These gave major warnings. One needs to proceed carefully. 24 pt size: camberic, carleton, carleton_bold, carleton_condesned, carleton_glow, rapscallionpirate, treasure_map 48 pt size: camberic, carleton, carleton_bold, carleton_glow, carleton_condensed, mason, mason_glow, stone, treasure_map |
|
July 20 Status Report about (2) Refont 2.1 Released; 2.2 in Beta ========================== 2.1 allows suppression of useless warnings due to scaling. 2.2 corrects a problem with buffer overflow of shaderName, encountered with rapscallionpirate font. Revised Workplan for 2.13 Font Improvements ====================================== Further analysis of all TDM fonts/sizes was done, to triage plausible improvements for 2.13, looking at two issues: A) Removal of minor and major warnings detected by refont. Minor warnings are due to inadequate precision, and can be easily auto-corrected by what will be referred to here as “touching” the DAT. (By this, I mean a round-trip in refont: DAT ==> REF ==> DAT.) Major warnings are best addressed by using datBounds and looking at the bitmap, fixing the REF file, then compiling the revised DAT. BTW, the TDM engine is robust in the face of these problems, but major warnings due to out-of-range metrics can cause odd glyph renderings. B) Substitution of redundant O-circumflex/o-circumflex codepoints by G-breve/g-breve. (Analysis here was aided by creation of "gfix", a hacked-up version of refont -stats focused just on this issue.) Broadly, the results suggest different treatments for two categories: System Fonts and FM-only Fonts. System Fonts ------------------- These are the carleton family (4 types x 3 sizes), mason family (2 types x 1 size: 48pt), and stone (2 sizes; but 24pt is already done, leaving 48pt). Generally, these fonts will need individual inspection, both to address major warnings (copious and nasty in some cases) and to understand how to implement G-breve/g-breve. A minimum REF-edit-only implementation of the latter would map to G/g. A fuller implementation would involve bitmap editing to provide the breves. FM-only Fonts -------------------- These 18 fonts are mostly ASCII-only. Other European characters are generally displayed as a hollow box or a “zero box”. This includes the existing 4 codepoints for O-circumflex/o-circumflex. Given that, the two codepoints to be considered G-breve/g-breve can continue to be displayed as a hollow box or a zero box at this time. That is, no change to the DAT file is needed for reason (B). For reason (A), almost all these fonts generate no warnings about any character font metrics, and so require no change to their 12pt & 48pt size DATs. Conversely, most generate minor warnings for 24pt size DATs, and should be “touched”. Special cases for (A): Major warnings are seen with camberic (all sizes), medusa (all sizes), rapscallionpirate (24pt), and treasure_map (all sizes). Most of these are about a negative value for a “top” metric, which should be a quick fix (unless it’s not). Special cases for (B): treasure_map in 24pt & 48pt has some support for O-circumflex/o-circumflex, so warrants similar support for G-breve/g-breve. It is more like a system font in that regard. In summary, across all the FM-only fonts in 3 sizes, there are around: - 28 DATs that are ok as is - 16 that need to be “touched” - 10 that need individual inspection and correction. Only treasure_map is a near-term candidate among FM-only fonts for bitmap editing to add G-breve/g-breve. |
|
For the FM-only/ASCII-only fonts, the 16 DATs that needed to be touched are done. I'll wait until later to release them, probably along with less-automated repair work on camberic, medusa, and rapscallionpirate. | |
Pausing any further work on 12pt for all fonts. Camberic 24 & 48 pt is done. Took a bit longer than expected, because needed a few poor-glyph touchups, plus moving a bunch of punctuation marks higher vis a vis baseline. Also prototyped a better test FM to review ASCII font chars within readables. Medusa next. |
|
Medusa 24 & 48pt are "done". Only a few glyphs moved. There are many that could be made more readable, but not a priority at this time. On to rapscallionpirate, and then planned release of this font trache. Related: New font-review FM "readablesASCII" further along. Has readables so far with revised camberic and medusa. (Also, as reported in forum, datBounds upgraded too.) |
|
Rapscallionpirate 24 & 48pt done, again in a limited sense... a few highly-unreadable chars improved. So next, I'll be better organizing and summarizing this font tranche, for release no later than mid-Sept. |
|
Now available is the first update in a series of font file improvements. This covers 14 fonts, and simply corrects minor internal rounding errors. It is thus a trivial update, but clears the deck for more substantive updates of remaining fonts. See the .docx file within the .zip for details. 2024_August_TDM_Font_Update.zip https://drive.google.com/file/d/1RNIDLmDkV_WT56kjtxSsXzqUxgYn5wh8/view?usp=sharing |
|
Here is the 2nd update in this series of font file improvements. This covers 3 fonts (camberic, medusa, and rapscallionpirate). See the .docx file at the top level within the .zip for specifics. 2024_Sept_TDM_Font_Update.zip https://drive.google.com/file/d/1t0AjCATPpFfSS3DQ-_YmC5Xs5r9H0jn-/view?usp=sharing |
|
In the previous two updates, improvements were made to most TDM fonts (of various sizes), but perhaps surprisingly, Task Item (2) in the OP was considered but skipped. This was because careful analysis showed that these fonts have codepoints 0x88 and 0x98 (as well as those of many other characters) all pointing to either a Hollow Box glyph or a Zero Box. So for those cases, for expedience, just skip Task Item (2) and keep the existing Hollow/Zero box. Today, the 3rd update is released, covering the remaining font families... - Stone - Treasure Map - Mason - Carleton Unlike the previous updates, Task Item (2) is applied to these fonts. No other improvements are undertaken (other than auto-correction of minor warnings). Consequently, the release affects only the DAT files (specifically, their contents for locations 0x88 and 0x98), not the bitmap DDS files. See the .docx file at the top level within the .zip for specifics. 2024_Nov_TDM_Font_Update.zip https://drive.google.com/file/d/1iimktc-M51xOidd9A0AnXD6ONy-kPXFy/view?usp=sharing |
|
Hmmm, I just thought of a concern about that release. DDS files are routinely 256x256, and refont & datBounds expect that. It turns out for carleton and carleton_bold (but not carleton_glow or carleton_condensed), their two 24pt dds files are 512x512. These are files fontimage_0_24.dds and fontimage_1_24.dds. Those pair of bitmaps are identical for carleton and carleton_bold. BTW, a quick look at the filesizes will tell 256x256 vs 512vs512. Speculatively, these bigger bitmaps were enlarged to allow non-ASCII characters to be added by hand-drawing... those glyphs are very roughly rendered. Current datBounds will definitely not handle 512x512 correctly, and refont (and so the REF and DAT files) is problematic. I'll need to research this further, revise the programs, and re-issue the suspect carleton family updates. |
|
It turns out that only the REF files need to change. An update of those and of refont are available. See: https://forums.thedarkmod.com/index.php?/topic/22427-analysis-of-212-tdm-fonts/&do=findComment&comment=498265 |
|
Date Modified | Username | Field | Change |
---|---|---|---|
10.06.2024 18:06 | Geep | New Issue | |
22.06.2024 00:37 | Geep | Note Added: 0016765 | |
26.06.2024 21:00 | Geep | Note Added: 0016779 | |
04.07.2024 19:03 | Geep | Note Added: 0016788 | |
20.07.2024 21:12 | Geep | Note Added: 0016801 | |
20.07.2024 21:13 | Geep | Note Edited: 0016801 | |
20.07.2024 21:14 | Geep | Note Edited: 0016801 | |
20.07.2024 21:16 | Geep | Note Edited: 0016801 | |
24.07.2024 15:10 | Geep | Note Added: 0016802 | |
05.08.2024 16:27 | Geep | Note Added: 0016808 | |
18.08.2024 21:03 | Geep | Note Added: 0016813 | |
28.08.2024 19:12 | Geep | Note Added: 0016817 | |
30.08.2024 20:52 | Geep | Note Added: 0016818 | |
24.09.2024 20:36 | Geep | Note Added: 0016842 | |
11.11.2024 04:09 | Geep | Note Added: 0016910 | |
11.11.2024 19:36 | Geep | Note Added: 0016912 | |
13.11.2024 21:00 | Geep | Note Added: 0016915 |