0AD27 can't build on Fedora 39 Workstation edition - Mozjs/Python related

[SolarEagle]

Hello,

I upgraded to Fedora 39 a few days after its release. Since then, I can't build Mozjs. Here's the relevant code, when doing:

./clean-workspaces.sh

then:

./update-workspaces.sh

I post only the relevant part, but I can post the full logs if needed.

Building SpiderMonkey...

SpiderMonkey build options: --disable-tests
           --disable-jemalloc
           --disable-js-shell
           --without-intl-api
           --enable-shared-js
           --disable-jitspew         
patching file js/src/moz.build
patching file js/src/old-configure
patching file python/mozbuild/mozbuild/virtualenv.py
patching file third_party/python/virtualenv/virtualenv/discovery/py_info.py
patching file python/mozbuild/mozbuild/action/process_define_files.py
patching file python/mozbuild/mozbuild/backend/base.py
patching file python/mozbuild/mozbuild/preprocessor.py
patching file python/mozbuild/mozbuild/util.py
patching file python/mozbuild/mozpack/files.py
patching file build/moz.configure/flags.configure
/home/nicolas/0ad/libraries/source/spidermonkey/mozjs-91.13.1/python/mozbuild/mozbuild/configure/__init__.py:915: SyntaxWarning: invalid escape sequence '\.'
  RE_MODULE = re.compile("^[a-zA-Z0-9_\.]+$")
Traceback (most recent call last):
  File "/home/nicolas/0ad/libraries/source/spidermonkey/mozjs-91.13.1/build-debug/../js/src/../../configure.py", line 22, in <module>
    from mozbuild.configure import (
  File "/home/nicolas/0ad/libraries/source/spidermonkey/mozjs-91.13.1/python/mozbuild/mozbuild/configure/__init__.py", line 13, in <module>
    from six.moves import builtins as __builtin__
ModuleNotFoundError: No module named 'six.moves'
ERROR: SpiderMonkey build failed

If I build it with the option

./update-workspaces.sh --with-system-mozjs

it builds, but with errors:

Linking Premake5
make : on quitte le répertoire « /home/nicolas/0ad/build/premake/premake5/build/gmake2.unix »

Premake args:  --with-system-mozjs --atlas
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Package mozjs-91 was not found in the pkg-config search path.
Perhaps you should add the directory containing `mozjs-91.pc'
to the PKG_CONFIG_PATH environment variable
Package 'mozjs-91', required by 'virtual:world', not found
Building configurations...
Running action 'gmake'...
Generated ../workspaces/gcc/Makefile...
Generated ../workspaces/gcc/pyrogenesis.make...
Generated ../workspaces/gcc/network.make...
Generated ../workspaces/gcc/rlinterface.make...
Generated ../workspaces/gcc/tinygettext.make...
Generated ../workspaces/gcc/lobby.make...
Generated ../workspaces/gcc/glooxwrapper.make...
Generated ../workspaces/gcc/simulation2.make...
Generated ../workspaces/gcc/scriptinterface.make...
Generated ../workspaces/gcc/engine.make...
Generated ../workspaces/gcc/graphics.make...
Generated ../workspaces/gcc/atlas.make...
Generated ../workspaces/gcc/gui.make...
Generated ../workspaces/gcc/lowlevel.make...
Generated ../workspaces/gcc/gladwrapper.make...
Generated ../workspaces/gcc/mongoose.make...
Generated ../workspaces/gcc/mocks_real.make...
Generated ../workspaces/gcc/mocks_test.make...
Generated ../workspaces/gcc/AtlasObject.make...
Generated ../workspaces/gcc/AtlasUI.make...
Generated ../workspaces/gcc/ActorEditor.make...
Generated ../workspaces/gcc/Collada.make...
Generated ../workspaces/gcc/cxxtestroot.make...
Generated ../workspaces/gcc/test.make...
Done (1795ms).

Then, doing in the relevant folder

make

fails:

nicolas@nt:~/0ad/build/workspaces$ cd gcc/
nicolas@nt:~/0ad/build/workspaces/gcc$ make
==== Building mocks_real (release) ====
Creating obj/mocks_real_Release
mocks_real.cpp
Linking mocks_real
==== Building network (release) ====
Creating obj/network_Release
precompiled.h
Dans le fichier inclus depuis ../../../source/network/NetMessages.h:27,
                 depuis ../../../source/network/NetMessage.h:26,
                 depuis ../../../source/pch/network/precompiled.h:26:
../../../source/scriptinterface/ScriptTypes.h:62:10: erreur fatale: jspubtd.h : Aucun fichier ou dossier de ce type
   62 | #include "jspubtd.h"
      |          ^~~~~~~~~~~
compilation terminée.
make[1]: *** [network.make:139: obj/network_Release/precompiled.h.gch] Error 1
make: *** [Makefile:79: network] Error 2

 I made a little research, and the error may be related to the Python six package, which is, if I've understood correctly, used for compatibility between Python 2 and 3. The package python3-six is installed on my computer.

Edited December 28, 2023 by Norse_Harold
Edited title to remove [Solved] because the issue isn't solved

[hyperion]

If it's not Fedora specific it might be related to python version. Could you post the version, so maybe others can try to reproduce it.

As for the system mozjs91 package, do you have it actually installed? I'm asking as the configure step (premake) can't find it in the first place and as a result the build error is fully expected. Premake / uptdate-workspace.sh not aborting early and creating broken makefiles seems a bug on it's own. If mozjs91 is installed, does it also install a pkgconfig file or just the shared library?

[SolarEagle]

 From what I do understand, mozjs115 is the default one on Fedora, but mozjs91 is available in the repos, and was already installed. Uninstalling it doesn't change anything.

[hyperion]

Not sure which package manager you use, there seems to be rpm yum and dnf for fedora, each having there own way to list installed files for packages. If https://packages.fedoraproject.org/pkgs/mozjs91/mozjs91/fedora-rawhide.html#files is correct then it doesn't install a pkgconfig file which I consider a packaging bug. Maybe file a bug against mozjs on the fedora bugtracker to get this rectified.

Also looking at the same page it looks like you are using python3.12, is that correct?

[Norse_Harold]

If SolarEagle wants to play 0ad multiplayer then he shouldn't be using anything other than the bundled mozjs, right? Otherwise, even a small difference in how the JavaScript engine functions, such as serialization and deserialization, could cause  an out-of-sync condition during multiplayer gameplay.
 

 

Edited December 27, 2023 by Norse_Harold

[hyperion]

16 minutes ago, Norse_Harold said:

If SolarEagle wants to play 0ad multiplayer then he shouldn't be using anything other than the bundled mozjs, right? Otherwise, even a small difference in how the JavaScript engine functions, such as serialization and deserialization, could cause  an out-of-sync condition during multiplayer gameplay.

Nah, if it builds it should be fine, quite a few distribution actually do this for the package 0ad. The last time runtime changes to the language happened should be well over a decade ago, tho don't take that as the ultimate truth, it's just what I remember. What we see more often is breakage in how we interface with the engine from C++, therefore we stick to EOL LTS releases. Ofc a distribution can patch it so it wont work but that is exceedingly unlikely.

[hyperion]

Apparently Fedora also uses devel packages so you'd need https://packages.fedoraproject.org/pkgs/mozjs91/mozjs91-devel/fedora-39.html, not mozjs91.

[SolarEagle]

 Many thanks, this is the solution. I mark the first post as solved.

[Norse_Harold]

19 hours ago, hyperion said:

Nah, if it builds it should be fine, quite a few distribution actually do this for the package 0ad.

In source/scriptinterface/ScriptTypes.h is a message that the only version of mozjs that works is the one that is bundled with 0ad. Either this message in the source code needs to be updated, or we need to double check the statement that use of a system mozjs is recommended for multiplayer. Also note the warning about the need for testing with any other version of mozjs, which distro maintainers aren't doing. They run the bundled FCollada tests, but they don't do testing of multiplayer or running replays and comparing the final hash.

In order for use of differing versions of mozjs to work properly in multiplayer, it needs to not only have the same intended functionality, but it also needs to have bug-for-bug compatibility.

Edited December 28, 2023 by Norse_Harold

[vladislavbelov]

9 minutes ago, Norse_Harold said:

Also note the warning about the need for testing with any other version of mozjs, which distro maintainers aren't doing. They run the bundled FCollada tests, but they don't do testing of multiplayer or running replays and comparing the final hash.

They're errors. So it won't compile if versions are mismatched. But indeed different versions (especially major) might potentially cause OOS.

[Norse_Harold]

20 hours ago, hyperion said:

quite a few distribution actually do this (use system-provided mozjs) for the package 0ad.

I would like to know which distributions specifically do that for their official binary package releases. I haven't found any so far. Debian doesn't do it. I assume that Ubuntu doesn't do it, since Ubuntu packages are based on Debian packages.

I just checked, and Fedora 39 doesn't do it. One might think that their 0ad.spec file uses system-provided mozjs by default, but it's a counter-intuitive syntax. It actually disables the build condition of system_mozjs by default. Proof is in the build logs that it's building the bundled spidermonkey.

Debian would like to use system-provided libraries in general because it makes it easier to maintain security of dependencies (ie. apply security updates promptly and in a centralized manner). But, from watching how they update the 0ad package, I see no evidence that the two maintainers are testing the game any more than verifying that it starts up after the package is installed. That's a conflict with the warning to developers and maintainers (yes vlad, I know that it is technically a compiler #error) that system-provided mozjs should be tested thoroughly. Debian maintainers are simply not doing that. They're doing the minimum to maintain the package. I assume that it's similar with maintainers of other distros.

We need to know which distributions and users are doing this so we can get feedback about whether it works reliably, and so that we know about a likely explanation for OOS problems. They're trailblazers and guinea pigs, because, as I understand it, use of a system-provided mozjs with 0ad is just not being done by the majority of users currently.

Edited December 28, 2023 by Norse_Harold

[Norse_Harold]

On 27/12/2023 at 10:57 AM, SolarEagle said:

I upgraded to Fedora 39 a few days after its release. Since then, I can't build Mozjs. Here's the relevant code, when doing:

Can you please try using the official Fedora method for building the package from source. This should ensure that you have the necessary dependency packages installed and distro-provided patches applied to the source code.

Specifically,

dnf download --source 0ad
cd 0ad
rpmbuild -bb

I haven't tested these commands because I don't have a Fedora system. The most likely point of failure would be the name of the directory to change to in the second command.

If the official Fedora method of building the package from source doesn't work, then Fedora would surely like to know about it. It's likely a bug that they can fix, or else it's user error. Based on the build logs it should build successfully on Fedora 39.

Edit: oops, the official Fedora method for building the package from source is only applicable to 0ad version alpha 26. However, it may at least help you ensure that you have the necessary build dependencies installed.

We need more information, such as the versions of python, python3 and python3-six that you have installed, in order to help you with troubleshooting building alpha 27.

How did you update to Fedora 39? Did you do a clean install, or did you upgrade from an existing installation of an older version of Fedora? You might still have a lot of python2 packages. The build of spidermonkey might be using python2 packages by default instead of the necessary python3 packages.

I think that you should try to remove as many python2 packages as you can without breaking lots of other packages. Replace them with equivalent python3 packages.

python2 packages are named "python-name".

python3 packages are named "python3-name".

Edited December 28, 2023 by Norse_Harold

[hyperion]

1 hour ago, Norse_Harold said:

We need more information, such as the versions of python, python3 and python3-six that you have installed, in order to help you with troubleshooting building alpha 27.

It's a python-3.12 issue, I created a trac ticket with a link to the upstream patch, no need for more info or experiments, it's easy enough to reproduce.

 

1 hour ago, Norse_Harold said:

I would like to know which distributions specifically do that for their official binary package releases.

Gentoo did for the longest time, then stopped because the restrictive header check became annoying for packaging, not because it could lead to OOS. Pretty sure once upon a time Ubuntu did the system mozjs thingy as well. The header test I consider a remnant of the sm-1.8 days. The pc file already ensures same major version.

[Stan`]

Spidermonkey also breaks API in minor versions sometimes (happened for 78). Another issue was the ICU problem where you need to have the same version between spidermonkey and 0ad else it will fail to link, but they do not pick it from the same place.

[hyperion]

On 28/12/2023 at 9:26 PM, Stan` said:

Spidermonkey also breaks API in minor versions sometimes (happened for 78). Another issue was the ICU problem where you need to have the same version between spidermonkey and 0ad else it will fail to link, but they do not pick it from the same place.

That incident with 78 affected at least a few distributions, so they must have used system spidermonkey at that time. Also that we are aware that some distros package sm wrt ICU in a way that causes issues is due to them using (or trying to use) system sm. Looking at the spec file linked by Norse we also see support for system sm.

So if they can they really would prefer that option any day, but the rigid check makes this somewhat tricky for packagers.

[Stan`]

16 minutes ago, hyperion said:

So if they can they really would prefer that option any day, but the rigid check makes this somewhat tricky for packagers.

Maybe the minor check could be omitted, @Itms or @wraitii would know, but the major check has to stay. I don't know if it got better, but our SM versions are usually heavily patched and that could lead to trouble maybe? The distro would just say the package is broken and give up on it ?
 

From my experience talking to maintainers SM is always the biggest pain, sometimes NVTT (no longer an issue now) and finally Fcollada when compilers decide to change things ^^

In the current situation given the missing parts I'd say we're not ready for 102.

 

 

[hyperion]

3 minutes ago, Stan` said:

but the major check has to stay

The major we ensure with pkgconfig already. So the the check in the header is redundant.

 

5 minutes ago, Stan` said:

In the current situation given the missing parts I'd say we're not ready for 102.

I tried fixing sm91 for python 3.12, that one looks really tricky, needs more than linked patch  in the bug and I ended up with binary patches for virtualenv before stopping for now. If we can't add git to the build deps I fear we won't add support for python 3.12 either.

[s0600204]

3 hours ago, hyperion said:

I tried fixing sm91 for python 3.12, that one looks really tricky, needs more than linked patch  in the bug and I ended up with binary patches for virtualenv before stopping for now. If we can't add git to the build deps I fear we won't add support for python 3.12 either.

Incidentally, I've also tried looking into this. I'm pleased to say that (after spending more time than I care to admit on it) I can now successfully build sm91 using python 3.12. The bad news that is that my solution as-is is a mess, and probably won't work for others, particularly Windows or macOS users.

It would be a lot easier, I'd imagine/hope, to patch SM102.

[hyperion]

11 hours ago, s0600204 said:

It would be a lot easier, I'd imagine/hope, to patch SM102.

I somewhat doubt it as the python 3.12 porting only really started with SM120, if you want me to try on a different distro feel free to share your patch. I also realized we could just pre apply the binary patches as we repackage the tarball anyway, so blatantly updating bundled virtualenv to a recent version via binary patch (18M) could be avoided.

 

@Stan` I played a match with SM91 vs SM102 and no OOS occurred, my guess the same would be the case for some more esr branches in both directions as we are no longer in the age of netscape where javascript was more like a toy. About we heavily patching SM, compared to distros we are close to vanilla . The reason for giving up is more likely that requiring a different SM minor than the one provided by the distro leads to "slot conflicts", ie a user needs two versions of SM with the same major which can't be installed at the same time. For a packager it's hard to tell if the header check for minor is meaningful, so if in doubt do as upstream insists.

[Stan`]

1 hour ago, hyperion said:

I played a match with SM91 vs SM102 an

Yeah usually those are non trivial bugs. Also I assume you tested on Linux Mileage may vary.

Still distros will other than arch and stuff will never write our migration patches between SM versions