"Not implemented" error with campaign button

[thekolian1996]

I have issue with hovering mouse cursor over campaign button. This problem exists only with two languages such as Bahasa Melayu and Ukrainian. This issue did not exist in A22.

Edited May 19, 2018 by thekolian1996
I am also translater for Ukrainian on Transifex . I'll try to help with this. [Update] Too late, you've done it on your own :)

[vladislavbelov]

12 minutes ago, thekolian1996 said:

I have issue with hovering mouse cursor over campaign button. This problem exists only with two languages such as Bahasa Melayu and Ukrainian. This issue did not exist in A22.

Hi @thekolian1996. The problem is in the translation string, it contains "[" and "]". They are special characters and should be escaped with the slash "\" one.

The solution is the translation fixing: https://www.transifex.com/wildfire-games/0ad/language/uk/. I think we need to ping translators and add a check in the translation script. @elexis?

[elexis]

Not sure if we can write a script to detect that. Some strings are used in GUI objects, others aren't. Those that aren't may use unescaped symbols, right?

[Imarok]

21 hours ago, elexis said:

Not sure if we can write a script to detect that. Some strings are used in GUI objects, others aren't. Those that aren't may use unescaped symbols, right?

What strings are not used in the GUI?

Maybe we could run the script only for some directories?

[fcxSanya]

On 5/20/2018 at 1:19 AM, elexis said:

Not sure if we can write a script to detect that. Some strings are used in GUI objects, others aren't.

We can compare tags in source / translated strings, I recall some translation tools giving warnings when they don't match (and in case of a custom script we of course can define yourselves what we consider a tag).

@Gallaecio or @GunChleoc may have some advice (having more experience with translation tools)

[vladislavbelov]

On 5/20/2018 at 1:19 AM, elexis said:

Not sure if we can write a script to detect that. Some strings are used in GUI objects, others aren't. Those that aren't may use unescaped symbols, right?

There is a little hack, we can just run a special generated page, that contains all translated text. It guarantees that all texts will be tested with the real engine.

[elexis]

2 hours ago, vladislavbelov said:

There is a little hack, we can just run a special generated page, that contains all translated text. It guarantees that all texts will be tested with the real engine.

Just calling the C++ function that parses all string should be sufficient.

On 5/20/2018 at 10:14 PM, Imarok said:

What strings are not used in the GUI?

Maybe we could run the script only for some directories?

Eh, you might be right that all strings end up in places that actively look for these gui tags. But there are still some differences. Some of the strings may use [ and other tags because they receive escapeText(), others don't. (right?)

[vladislavbelov]

6 hours ago, elexis said:

Just calling the C++ function that parses all string should be sufficient.

I think it requires more work (need to write JS to call C++) and has less checks, because no rendering there.

[elexis]

What's the benefit of rendering them? I don't think all strings that we have fit in a single page, also consider the cost of controling all of them manually, whereas a script can be ran from commandline and gives you almost instantly the results.

[Imarok]

7 hours ago, elexis said:

But there are still some differences. Some of the strings may use [ and other tags because they receive escapeText(), others don't. (right?)

I don't know, that's why I'm asking

[vladislavbelov]

1 hour ago, elexis said:

What's the benefit of rendering them? I don't think all strings that we have fit in a single page, also consider the cost of controling all of them manually, whereas a script can be ran from commandline and gives you almost instantly the results.

For full check. I.e. script doesn't guarantee existed symbols, you need to rewrite script for each change in the engine, it will test only small subset of functions.

And as I said it's a bigger work to write a validate script with right logic than the page generator script.

[elexis]

2 hours ago, vladislavbelov said:

it will test only small subset of functions

The aim is to test the strings, not the functions?

2 hours ago, vladislavbelov said:

And as I said it's a bigger work to write a validate script with right logic than the page generator script. 

Not sure. Loading all strings has to be done in both cases but we only need to create a CGUIString of each loaded string, so it seems less effort if we wanted to ensure that all strings contain only valid gui tags if they contain the special characters? (There is still no pattern that predicts when strings are escaped though)

[vladislavbelov]

24 minutes ago, elexis said:

The aim is to test the strings, not the functions?

I meant functionality (that process these strings), sorry. By hand checking we may loose many corner cases.

27 minutes ago, elexis said:

so it seems less effort

We don't want to run it for each commit, only for translation updates. So the performance difference is incomparable with the code problems:

30 minutes ago, elexis said:

There is still no pattern that predicts when strings are escaped though

 

[vladislavbelov]

@elexis I wrote a simple script: https://code.wildfiregames.com/P122 (it creates a test mod folder with the test page), it shows, that we have many errors in translations (including not included in the release).

[GunChleoc]

\[ isn't something commonly checked for, so we can't use existing tools - a least not was far as I am aware of.

The way we handle this in Widelands is that we send all strings through a "richtext-escape" function that will replace those elements that would cause crashes with appropriate entities.

Since you are using square brackets for font instructions in translatable text, you'll need to write some regex to automatically replace these. The superior solution would be to redesign the strings in such a way that all code is removed from the translatable strings, then you wouldn't need to escape [ in the source strings at all and could handle it by a simple string replace.

Handing code to translators is always fragile and should be restricted to placeholders, which translators are used to, and which can also be checked by tools.

[vladislavbelov]

On 5/26/2018 at 5:27 PM, GunChleoc said:

The way we handle this in Widelands is that we send all strings through a "richtext-escape" function that will replace those elements that would cause crashes with appropriate entities.

Since you are using square brackets for font instructions in translatable text, you'll need to write some regex to automatically replace these.

The main problem, that we can't check translations before the translator submit, other solutions are workarounds. 

On 5/26/2018 at 5:27 PM, GunChleoc said:

The superior solution would be to redesign the strings in such a way that all code is removed from the translatable strings, then you wouldn't need to escape [ in the source strings at all and could handle it by a simple string replace.

There's no code in strings, there are tags. We can use them for color, font and other style. So this isn't the solution.

[GunChleoc]

Well, I consider tags to be code, because they can break things - so, they should only be there if they are really needed. Do we really need to let translators change the font size and things like that in 0 AD?
 

Quote

The main problem, that we can't check translations before the translator submit, other solutions are workarounds. 

 

Agreed, this has to be checked at runtime.

[elexis]

17 minutes ago, GunChleoc said:

Do we really need to let translators change the font size and things like that in 0 AD?

Consider the english string "word1 word2 word3" that would be translated to "translation1 translation2 translation3 translation4". But its indeed better to avoid such things.

Another example are translated txt files and map descriptions where some words are colored.

[GunChleoc]

I have searched Transifex for [, and these are the current uses for tags:

1. Whole manual segments that have a font tag at the beginning only, and no closing tag
2. Some in-line headers that are separated by dashes in the manual
3. Warnings in map descriptions
4. Icons on the splash screen

Maybe this can be resolved by parsing [font, [color and [img rather than just [? I any case, I think the tags should be removed from entries like:
 

 [font="sans-bold-16"]Graphics settings