Spidermonkey 68 & 78 upgrade

[wraitii 1.460]

Hello everyone,

I have just merged SM68. To compile the game, you will now require the Rust compiler.
You can follow instructions here: https://www.rust-lang.org/tools/install
It should be mostly straightforward.

I am now working towards SM78 which I hope to merge soon. I will detail this thread with what the changes mean later  

[wraitii 1.460]

And after a long month of upgrades, I have at last merged SM78 !

0 A.D. now uses the latest "Stable" Spidermonkey release. The next one will be SM94, I believe, which won't come for a little while still.

This version requires "LLVM-objdump" to compile. You will probably need to install LLVM binaries, if you're compiling using GCC (if you're using clang, you probably already have it). Depending on your distro, this could be a in various packages. If you can, update the https://trac.wildfiregames.com/wiki/BuildInstructions with the correct instructions  

As usual, use this thread to report issues.

----

NB: we know that there is an issue with GCC 7.5. Fixes include upgrading to GCC 8 or applying D3181 from Phabricator.

[Nescio 1.525]

Congratulations!

On 06/12/2020 at 3:10 PM, wraitii said:

This version requires "LLVM-objdump" to compile. You will probably need to install LLVM binaries, if you're compiling using GCC (if you're using clang, you probably already have it). Depending on your distro, this could be a in various packages.

Does it? Fedora 33 with gcc 10.2.1 here; I believe I don't have llvm installed. I did a clean build and everything seems to work when I run the game.

[wraitii 1.460]

10 hours ago, Nescio said:

Does it? Fedora 33 with gcc 10.2.1 here; I believe I don't have llvm installed. I did a clean build and everything seems to work when I run the game.

You must have an "objdump"  or "llvm-objdump" program. Possibly LLVM's, possibly something compatible. I'm interested if you can find out more.

[hyperion 27]

objdump is part of binutils

[Nescio 1.525]

2 hours ago, wraitii said:

You must have an "objdump"  or "llvm-objdump" program. Possibly LLVM's, possibly something compatible. I'm interested if you can find out more.

1 hour ago, hyperion said:

objdump is part of binutils

Yes, that one I have:

Spoiler

[~]$ sudo dnf install binutils
Package binutils-2.35-15.fc33.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!

[Nescio 1.525]

@wraitii, could you specify what dependencies are needed for SpiderMonkey and what the requirements are for the rest of the game?

https://trac.wildfiregames.com/wiki/BuildInstructions lists a --with-system-mozjs option. I have the relevant package installed (mozjs78-devel-78.6.0-1.fc33.x86_64) and tried compiling 0 A.D. with it, but it failed, unfortunately. Being able to use the distribution's shared package would save disk space and compilation time.

[asterix 418]

6 minutes ago, Nescio said:

--with-system-mozjs option

This currently is problematic on some linux systems because of https://code.wildfiregames.com/D3127

[Stan` 7.035]

If it's not fixed by using ^ then it would be nice for you to share the logs

[Nescio 1.525]

40 minutes ago, Stan` said:

If it's not fixed by using ^ then it would be nice for you to share the logs

So I did a make clean and a clean-workspaces.sh, then ran update-workspaces --with-system-mozjs and make, which ended with:

==== Building pyrogenesis (release) ====
Creating obj/pyrogenesis_Release
main.cpp
Linking pyrogenesis
/usr/bin/ld: ../../../binaries/system/libscriptinterface.a(ScriptInterface.o): in function `JSStructuredCloneData::~JSStructuredCloneData()':
/usr/include/mozjs-78/js/StructuredClone.h:459: undefined reference to `js::SharedArrayRawBufferRefs::~SharedArrayRawBufferRefs()'
/usr/bin/ld: /usr/include/mozjs-78/js/StructuredClone.h:459: undefined reference to `js::SharedArrayRawBufferRefs::~SharedArrayRawBufferRefs()'
collect2: error: ld returned 1 exit status
make[1]: *** [pyrogenesis.make:81: ../../../binaries/system/pyrogenesis] Error 1
make: *** [Makefile:71: pyrogenesis] Error 2

When trying to run binaries/system/test or pyrogenesis the response is “No such file or directory”.

Then I did a clean again, applied D3127, and ran update-workspaces --with-system-mozjs again, which quickly stopped, ending with:

string_sha1.c
string_startswith.c
term_textColor.c
zip_extract.c
Linking Premake5
make: Leaving directory '/home/b/Projects/0ad/build/premake/premake5/build/gmake2.unix'

Premake args:  --with-system-mozjs --atlas
Error: /home/b/Projects/0ad/build/premake/pkgconfig/pkgconfig.lua:89: attempt to call a nil value (global 'aftersysincludedirs')
ERROR: Premake failed

 

[Stan` 7.035]

@wraitii
^

[wraitii 1.460]

19 hours ago, Nescio said:

So I did a make clean and a clean-workspaces.sh, then ran update-workspaces --with-system-mozjs and make, which ended with:

That's https://bugzilla.mozilla.org/show_bug.cgi?id=1644600

19 hours ago, Nescio said:

Then I did a clean again, applied D3127, and ran update-workspaces --with-system-mozjs again, which quickly stopped, ending with:

Seems the diff is a little buggy then.

I'm afraid the system-mozjs won't work on your distribution unless you ask them to patch https://bugzilla.mozilla.org/show_bug.cgi?id=1644600

[Frantisek Zatloukal 2]

On 22/12/2020 at 3:51 PM, wraitii said:

That's https://bugzilla.mozilla.org/show_bug.cgi?id=1644600

Seems the diff is a little buggy then.

I'm afraid the system-mozjs won't work on your distribution unless you ask them to patch https://bugzilla.mozilla.org/show_bug.cgi?id=1644600

Hi, I am maintainer of mozjs78 in Fedora. Can you add a patch to that ticket? Or provide a link to that patch if it's somewhere in 0ad's repo?

I'll happily include a patch for this in Fedora package.

Thanks!

[Stan` 7.035]

13 minutes ago, Frantisek Zatloukal said:

Hi, I am maintainer of mozjs78 in Fedora. Can you add a patch to that ticket? Or provide a link to that patch if it's somewhere in 0ad's repo?

I'll happily include a patch for this in Fedora package.

Thanks!

Hey thanks for reaching out,  The patch is here https://github.com/0ad/0ad/blob/83e81362d850cc6f2b3b598255b873b6d04d5809/libraries/source/spidermonkey/FixSharedArray.diff

You can see all the patches we apply here

https://github.com/0ad/0ad/blob/83e81362d850cc6f2b3b598255b873b6d04d5809/libraries/source/spidermonkey/patch.sh

[Frantisek Zatloukal 2]

15 hours ago, Stan` said:

Hey thanks for reaching out,  The patch is here https://github.com/0ad/0ad/blob/83e81362d850cc6f2b3b598255b873b6d04d5809/libraries/source/spidermonkey/FixSharedArray.diff

You can see all the patches we apply here

https://github.com/0ad/0ad/blob/83e81362d850cc6f2b3b598255b873b6d04d5809/libraries/source/spidermonkey/patch.sh

Thanks a lot for quick reply, I've included the patch in Fedora package: https://src.fedoraproject.org/rpms/mozjs78/c/15ae20d569f6d11894a1e2fc29169d10432dfb6a?branch=rawhide . However, fun doesn't end here, it seems something has changed in between mozjs78.6 and mozjs78.8:

 

./../../source/scriptinterface/ScriptContext.cpp: In member function 'void ScriptContext::UnRegisterRealm(JS::Realm*)':
../../../source/scriptinterface/ScriptContext.cpp:136:46: error: cannot convert 'JS::Zone*' to 'JSContext*'
  136 |         JS::PrepareZoneForGC(js::GetRealmZone(realm));
      |                              ~~~~~~~~~~~~~~~~^~~~~~~
      |                                              |
      |                                              JS::Zone*
In file included from /usr/include/mozjs-78/js/Value.h:25,
                 from /usr/include/mozjs-78/js/CallArgs.h:74,
                 from /usr/include/mozjs-78/jsapi.h:31,
                 from ../../../source/scriptinterface/ScriptTypes.h:63,
                 from ../../../source/scriptinterface/ScriptContext.h:21,
                 from ../../../source/scriptinterface/ScriptContext.cpp:20:
/usr/include/mozjs-78/js/GCAPI.h:539:55: note:   initializing argument 1 of 'void JS::PrepareZoneForGC(JSContext*, JS::Zone*)'
  539 | extern JS_PUBLIC_API void PrepareZoneForGC(JSContext* cx, Zone* zone);
      |                                            ~~~~~~~~~~~^~
In file included from /usr/include/mozjs-78/js/TraceKind.h:12,
                 from /usr/include/mozjs-78/jspubtd.h:18,
                 from ../../../source/scriptinterface/ScriptTypes.h:62,
                 from ../../../source/scriptinterface/ScriptContext.h:21,
                 from ../../../source/scriptinterface/ScriptContext.cpp:20:
/usr/include/mozjs-78/js/TypeDecls.h:55:21: note: class type 'JS::Zone' is incomplete
   55 | class JS_PUBLIC_API Zone;
      |                     ^~~~
../../../source/scriptinterface/ScriptContext.cpp: In member function 'void ScriptContext::PrepareZonesForIncrementalGC() const':
../../../source/scriptinterface/ScriptContext.cpp:254:54: error: cannot convert 'JS::Zone*' to 'JSContext*'
  254 |                 JS::PrepareZoneForGC(js::GetRealmZone(realm));
      |                                      ~~~~~~~~~~~~~~~~^~~~~~~
      |                                                      |
      |                                                      JS::Zone*
In file included from /usr/include/mozjs-78/js/Value.h:25,
                 from /usr/include/mozjs-78/js/CallArgs.h:74,
                 from /usr/include/mozjs-78/jsapi.h:31,
                 from ../../../source/scriptinterface/ScriptTypes.h:63,
                 from ../../../source/scriptinterface/ScriptContext.h:21,
                 from ../../../source/scriptinterface/ScriptContext.cpp:20:
/usr/include/mozjs-78/js/GCAPI.h:539:55: note:   initializing argument 1 of 'void JS::PrepareZoneForGC(JSContext*, JS::Zone*)'
  539 | extern JS_PUBLIC_API void PrepareZoneForGC(JSContext* cx, Zone* zone);
      |                                            ~~~~~~~~~~~^~
In file included from /usr/include/mozjs-78/js/TraceKind.h:12,
                 from /usr/include/mozjs-78/jspubtd.h:18,
                 from ../../../source/scriptinterface/ScriptTypes.h:62,
                 from ../../../source/scriptinterface/ScriptContext.h:21,
                 from ../../../source/scriptinterface/ScriptContext.cpp:20:
/usr/include/mozjs-78/js/TypeDecls.h:55:21: note: class type 'JS::Zone' is incomplete
   55 | class JS_PUBLIC_API Zone;
      |                     ^~~~
make[1]: *** [scriptinterface.make:141: obj/scriptinterface_Release/ScriptContext.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: Leaving directory '/builddir/build/BUILD/0ad-0.0.24b-alpha/build/workspaces/gcc'
make: *** [Makefile:113: scriptinterface] Error 2

I can try and take a look, but no promises with my really basic C/C++ knowledge. Shall I report this to trac?

[wraitii 1.460]

I think that's because of https://hg.mozilla.org/releases/mozilla-esr78/rev/9fad778aa3291ef630338bc568453a79faa37089

It's already been reported at 5986 but for pratical reasons I haven't done the update for this release.