[Again]unable to display East Asian fonts

[kanetaka]

Recent svn can't display East Asian fonts. I doubt this has been since mod selector is introduced. East Asian fonts are distributed as mod(http://trac.wildfiregames.com/wiki/Installing_East_Asian_Locales) so we launched pyrogenesis.exe with -mod option in Alpha 16. Now we get the screenshot below.

I use a laptop with Windows 8.1 for your information.

Edited October 9, 2014 by kanetaka

[leper]

Following http://trac.wildfiregames.com/wiki/Adding_font_support on the wiki gives you a working font mod again, I'll try to upload it tomorrow new mod attached (only font files and a mod.json so that it shows up in the mod selector). This is actually related to us changing the caching formats a bit since A16.

EDIT: See below for the mod.

Edited October 14, 2014 by leper
Added mod.

[kanetaka]

Following http://trac.wildfiregames.com/wiki/Adding_font_support on the wiki gives you a working font mod again, I'll try to upload it tomorrow new mod attached (only font files and a mod.json so that it shows up in the mod selector). This is actually related to us changing the caching formats a bit since A16.

I tried the mod and it worked fine. Thank you.

[kanetaka]

My previous post was wrong. The problem still exists. But the mod works fine if it is unzipped. I am sorry to report late.

[leper]

The issue is the same as we encountered when testing the release packages, you should place the mod.json that is in the zip file next to the zip file. (Windows doesn't like multiple programs opening the zip file, so it doesn't find the mod which leads to issues when saving the config as the locale is still set to Japanese, but the fonts that support the needed characters aren't available.)

[leper]

I noticed that the po file for the mod selector was missing.

Also added a mod for zh_TW, and renamed the mod containing ja.

Using MicroHei for Chinese glyphs.

EDIT: http://sourceforge.net/projects/zero-ad/files/releases/locales/

Edited October 14, 2014 by leper
locale mods have been uploaded to sf

[llen_llen]

Thank you for providing font support!

However I believe East Asian font packs need proper subsetting as long as we carry on the current style.

There are several reasons:

1. The texture size is directly reflected in memory usage, so you'll suffer from severe memory overuse with huge textures.
   See http://trac.wildfiregames.com/ticket/2843
2. A great portion of characters actually only exist for (historical|regional) (terms|proper names) otherwise of no use (unless for translating Asian faction mod etc).
3. A "needed" character set for China/Taiwan/Japan is different from other ones by thousands of characters, due to linguistic/orthographical difference (not to mention Korea...). Covering all of them only adds a large number of unusable letters to every language.

At least Japanese community can offer a more dedicated JP font pack. How do you like it, leper?

P.S. Aside from chinese characters, I don't think the current implementation can handle Arabic or Indic alphabets while Persian translation is ongoing. Is there any plan to replace the architecture?

Edited October 14, 2014 by llen_llen

[leper]

However I believe East Asian font packs need proper subsetting as long as we carry on the current style.

Care to elaborate?

• A great portion of characters actually only exist for (historical|regional) (terms|proper names) otherwise of no use (unless for translating Asian faction mod etc).
• A "needed" character set for China/Taiwan/Japan is different from other ones by thousands of characters, due to linguistic/orthographical difference (not to mention Korea...). Covering all of them only adds a large number of unusable letters to every language.

That's one of the reasons why there are two mods now (and because some of the fonts used might provide glyphs for more than one language and collisions between those could look ugly). Those fonts are in a mod as others do not need those glyphs and the current way we handle fonts needs those huge files. (See also #2843)

Finding nice looking and compatible (license) fonts isn't really easy though, so if someone knows of a better looking font (or more complete one) just tell us.

At least Japanese community can offer a more dedicated JP font pack. How do you like it, leper?

It's a mod for a reason, and if you (the Japanese community, same for Chinese and other font/language families) can do a better job at providing a mod (which I assume is a lot easier as there is a lot more documentation about such fonts in those languages) please do so.

P.S. Aside from chinese characters, I don't think the current implementation can handle Arabic or Indic alphabets while Persian translation is ongoing. Is there any plan to replace the architecture?

If we have a font that has the needed glyphs we can display it. Scripts needing allographs will not work, that would need a new font renderer using Pango and HarfBuzz.

[llen_llen]

I had my post blown away for some reason so I had to rewrite this from scratch

Isn't there any way to save my draft...?

However I believe East Asian font packs need proper subsetting as long as we carry on the current style.

Care to elaborate?

I really ought to have

First, by "the current style" I mean the way we pre-render all glyphs we use into one sprite image and load it for displaying. This means an extra letter costs more memory than that of in a ttf file, particularly for East Asian characters which tend to be bulky than alphabets. To dump whole character set from an all-in-one font such as Hanazono or MicroHei results in enormous textures.

Meanwhile, many standardized language-specific subset(s) are known to suffice daily usage, which typically contains some thousands of glyphs (it sounds still big, but remember current official font packs contain some tens of thousands of them). We can choose ones for each locale mod so that substantially reduce textures' size. This is the general idea.

That's one of the reasons why there are two mods now (and because some of the fonts used might provide glyphs for more than one language and collisions between those could look ugly). Those fonts are in a mod as others do not need those glyphs and the current way we handle fonts needs those huge files. (See also #2843)

Yes, you are right for the most part, and I truly appreciate your effort to make them look more native. The only problem here is too many characters. The most of them are actually dispensable locale-wise, unless you take really seriously of interoperability, such as, you must be able to chat every character across locales or something.

Finding nice looking and compatible (license) fonts isn't really easy though, so if someone knows of a better looking font (or more complete one) just tell us.

Sure! The first choice for all Asian locales will be Source Han Sans family today, which contains locale-specific variations and 7 weights, under Apache 2.0 License. Our community build (see below) uses it too.

Other qualified fonts include:

Japanese - M+ (sans-serif; 7 weights) with many derivatives (Mgen+, Rounded M+, LogoTypeGothic...), Sawarabi (sans/serif), Nukamiso (sans), GT(sans/serif/script) etc.

Chinese (Taiwan) - official CNS 11643 fonts (script/serif; Trad.), Arphic 1 2 (script/serif; Simp./Trad.), cwTex family (Trad.) etc.

Chinese (China) - only WenQuanYi family (includes MicroHei) found free with explicit license.

Note that simplified Chinese font created in Taiwan may be inaccurate in detail, and vice versa.

It's a mod for a reason, and if you (the Japanese community, same for Chinese and other font/language families) can do a better job at providing a mod (which I assume is a lot easier as there is a lot more documentation about such fonts in those languages) please do so.

Thank you. Kanetaka and I are trying to make a lightweight community build, here are some screenshots:

Thanks to the official fontbuilder script, we were able to optimize font size and hinting, so it looks much better. But for some reason we are currently unable to build a correctly cached mod. Could you examine this original file?

https://www.dropbox.com/s/gqcvwia93s9w46y/jplocale%28uncached%29.zip?dl=0 (~30MB; zip)

And I believe that simplified/traditional Chinese font packs could be built in similar way.

If we have a font that has the needed glyphs we can display it. Scripts needing allographs will not work, that would need a new font renderer using Pango and HarfBuzz.

That's okay. I was just curious if font format will change in the near future.

Edited October 15, 2014 by llen_llen

[niektb]

What command are you using to package your mod?

[llen_llen]

What command are you using to package your mod?

I followed instructions on the following page:

http://trac.wildfiregames.com/wiki/Adding_font_support

except for some typos and directory delimiter difference seen in the example.

I think Kanetaka too.

EDIT:

As for me, I had problem installing the game in the first place.

Extracting of public.zip kept aborting at 64% whenever I tried, so I decided to decompress the exe and put every file directly into the target folder.

Edited October 15, 2014 by llen_llen

[leper]

I really ought to have

First, by "the current style" I mean the way we pre-render all glyphs we use into one sprite image and load it for displaying. This means an extra letter costs more memory than that of in a ttf file, particularly for East Asian characters which tend to be bulky than alphabets. To dump whole character set from an all-in-one font such as Hanazono or MicroHei results in enormous textures.

Meanwhile, many standardized language-specific subset(s) are known to suffice daily usage, which typically contains some thousands of glyphs (it sounds still big, but remember current official font packs contain some tens of thousands of them). We can choose ones for each locale mod so that substantially reduce textures' size. This is the general idea.

Yes, you are right for the most part, and I truly appreciate your effort to make them look more native. The only problem here is too many characters. The most of them are actually dispensable locale-wise, unless you take really seriously of interoperability, such as, you must be able to chat every character across locales or something.

Yes, the current system is far from perfect, but it is still working for most languages and there are quite some other issues that have higher priority. Something that could be done would be to stop including all glyphs (see my comment on #2843).

Sure! The first choice for all Asian locales will be Source Han Sans family today, which contains locale-specific variations and 7 weights, under Apache 2.0 License. Our community build (see below) uses it too.

Thank you! I'll take this into account when I have to build that mod for the next release.

Thank you. Kanetaka and I are trying to make a lightweight community build [...]

Attached.

Created using

./pyrogenesis -mod=mod -archivebuild-compress -archivebuild=/path/to/0ad/binaries/data/mods/jplocale -archivebuild-output=/path/to/0ad/binaries/data/mods/jplocale/jplocale.zip

You might want to update the information at the link in the mod.json (still points to A16, the fundraiser, and doesn't include any information about the lobby (at least none that I can find)).

[llen_llen]

Created using

./pyrogenesis -mod=mod -archivebuild-compress -archivebuild=/path/to/0ad/binaries/data/mods/jplocale -archivebuild-output=/path/to/0ad/binaries/data/mods/jplocale/jplocale.zip

Thank you! It worked for me perfectly! It seems that

-mod=mod

was the critical part, after all.

You might want to update the information at the link in the mod.json (still points to A16, the fundraiser, and doesn't include any information about the lobby (at least none that I can find)).

Sorry, but I'm not sure about what you meant in this part.

[feneur]

I had my post blown away for some reason so I had to rewrite this from scratch

Isn't there any way to save my draft...?

The forum software autosaves at certain intervals (every two mins unless I'm mistaken). There should be a text at the bottom left which says Last auto saved: and the time when it was saved. If you leave the page for any reason and then return it should say "View Auto Saved Content" + the date and time. Click that and you get a dialog with the saved text + a button that says "Restore content". Click that button to get what you have written back