Bellaz89

Hi @Stan` I have the same problem. And it is weird Wouldn't this block every player that try to join a lobby game? I have some(but not all!) of my friend in different countries/different ISPs that reported this issue. This problem also started to happen to me after I moved to another house. I mantained the same ISP, but I needed to switch from cable DSL to a 5G modem. I don't know what kind of connection my other friends have. I can ask if this can be useful. I am quite sure that we all run on linux (I have Ubuntu 20). I have tried to enable/disable the TLS encryption without success. After this change, this problem starts to happen. I am still able to connect to some 'golden' player, but many of them are out of my reach. I saw this on A24 and on the A25 prerelease I am also open to help and make tests with my network if needed

Hello @user1 my lobby name: Bellanzio offending user name: troll The opponent was the host and they ended the game without resigning when it was clear they were going to lose commands.txt

Interesting. So probably some VM knobs have really to be optimized.. What kind of feature does SM 91 include that makes the code run faster? I am not able to find the changelog (it seems that FF 91 is not even released).

@wraitii Just out of curiosity: did you see an improvement in JS execution performance after the SM update? I remember that I didn't get very impressive results with tests done by myself.

I don't know the 0ad developers plans, but I think that some dependencies of 0ad would be very difficult to compile with emscripten. For example, there should be a limitation in JIT generation from webassembly. This is a limitation for the Javascript engine of 0ad, Spidermonkey (https://stackoverflow.com/questions/53760444/is-it-possible-to-compile-javascript-enginelike-spidermonkey-and-run-on-webass). Even without that, making something as complicate as a modern JIT engine to run on an unsupported platform is extremely difficult (if possible at all..). IMHO even getting rid of Spidermonkey to make 0ad run on webassembly/emscripten would require a too big effort. This would mean to rethink all the foundations of the 0ad engine (pyrogenesis). At this point, it might be even easier to rewrite all 0ad C++ codebase from scratch... For comparison, just porting the JS engine (so, most of the code was already written) from version 45 to version 78 took ~1 year 'only' to fix all the API changes.

I don't know if this is intended or not, but the "source" install page is still referring to A23 https://play0ad.com/download/source/

Congratulations to the 0ad team and thank you very much for your efforts!!

AH cool! I was not aware that it is possible to enable them! This simplifies the SM68 -> SM78 upgrade a lot .

Hi @wraitii Just a question: since in SM78 uneval() and .toSource() were removed, how did you get around that? did you modify the JS scripts or did you write a wrapper?

Good job! I really like your solution of using the context through Request . I wonder if in Spidermonkey V52 there is already the bug in StructuredClone API I encountered in SM68 and SM78. Back then I had to patch Spidermonkey itself to make it possible to use the StructuredClone API (a symbol was not exported). Is this the case also for SM52?

@happyconcepts Yes, there was. It is fixed in https://code.wildfiregames.com/D2869

Hi there! Probably you noticed that there were not updates for some time. Unfortunately I was quite busy with other stuff and in the future I don't think I will have more time to dedicate on this (at least for ~8months). Therefore I would say that this experiment finishes here and I am happy about the outcomes. Some WFG members are already working on porting the main branch of 0ad to more recent version of Spidermonkey, so don't worry about this. Last micro update. I tried to do a very rough and inaccurate performance test just looking at the framerate. It seems that my modifications reduced a bit (not dramatically) the FPS. In my opinion this is due to the fact that many optimizations of the GC parameters were done for SM45 and are not useful anymore or even counterproductive for most recent versions of Spidermonkey. I think that there will be the need to spend some time in profiling the VM to get the best achievable performance.

That's a good question. I didn't profile the game and for now I just did debug builds so far, therefore I cannot tell. However I expect some improvement since many changes were made on the underling VM since V45. Also, for the future, having the possibility to use WebAssebly (and WebWorkers as suggested on #0ad-dev) could have for sure an impact. Stay tuned for benchmarks!

Update: Fixed error handling issue. Now 100% tests are running It was found that non-ASCII characters in game schema resulted in a crash. Since the presence of these characters is non-essential and limited, these characters were temporarly removed (Identity.js). The problem is under investigation. Probably this is due Game is running with Spidermonkey 68 Todo: Clean up. (better naming.. remove failed attempts to correct issue in the code) Benchmarking/Profiling. See how much the performaces increase/decrease after the upgrade Try link the game with SM78 beta () A big thanks to the WFG team for helping me in this task even though it doesn't follow the 'normal' 0ad development process. I hope this will be helpful in porting the official branch to newer versions of Spidermonkey Running game:

Update: Corrected a cross-compartment bug in CGame::ReallyStartGame, line 329. Does this affect the current 0ad version too?? Added checks to consume error messages I was able to start a game! ...but almost nothing (except the gui!) works properly

I tried to link a debug version of Spidermokey. This undercovered a lot of nasty bugs Update Patched Spidermonkey 68 source code. the class SharedArrayRawBufferRefs in ./js/public/StructuredClone.hStructuredClone.h (firefox source directory) was tagged as JS_PUBLIC_API to export its symbols. Please note that since Spidermonkey 68 clang is required to compile the codebase (at least on linux). Use instructions in https://github.com/mozilla-spidermonkey/spidermonkey-embedding-examples/blob/esr68/docs/Building SpiderMonkey.md. PersistentRooted is initialized with .init() Applied https://code.wildfiregames.com/D2768 ObjectToIdMap is based on GCHashMap I am almost able to run a game ...

Hi, It seems that you have my same problem.. (doing a PhD ). Don't worry. As I previously stated, I expected my effort not to be 100% compatible to the direction you want to follow. And also last days I found many issues to solve that popped out when I tried to link a debug version of SM68. I fear that my efforts will require quite more time to result in an usable version on 0ad. I think that going through SM 52 is the most reasonable way to have a stable code without wasting too much time. I will continue to try going directly to SM 68 and see what are the issues to face to port 0ad to that version of Spidermonkey. Of course I would be happy to discuss together and give as much help I can. Thanks for 0ad and see you soon!

Update ArrayBuffer tests fixed modifying the serializer/deserializer. Fixed UTF test removing U+D800 (high surrogate) unicode character. All tests run, 99% pass. The last failing test is on exception catching. For the moment it can be ignored. Starting to understand why the GUI doesn't show up when launching pyrogenesis Update 2 Fixed GUI with https://code.wildfiregames.com/D2768 Still crashing when changing page. (see below for test screenshots) (see below for gui screenshots)

@Sturm Hi, As @Stan` pointed out, the main technical point is that SM 68 is the latest ESR version (SM 78 ESR is going to be released at the end of this month) There are some other points: There is at least some documentation for SM 68 (see links in the first post). This helps a lot in understanding the direction that has to be followed to port the code without getting crazy. SM 68 is the latest packaged version of spidermonkey on my linux distribution. Since SM stability depends a lot on compilation flags, I prefer to use something already tested by someone else. Also IMHO there won't be an huge performance difference between version 68 and 76. The main reason to switch, apart from support, is going to be some Javascript feature that could be beneficial to use. I think much more performances are going to be obtained taking advantages of the C++ and Javascript API additions since SM 45. Of course if someone will decide to port 0ad to a version of SM newer than 68 is going to be good and beneficial (like I decided to jump directly to SM 68). It is the beauty of open source . In this regards e.g. Webassembly could be something to be explored to improve performances of pyrogenesis without changing the runtime code (or even making the runtime lighter moving part of the non-IO code, like the IA, the mapgen and the pathfinder, to WASM). This would be very beneficial from the modding/hacking perspective. But this is going to be decided by someone else

Hi! In fact none. It is just that SM 68 defines some values in its headers that usec++ 14 digit separators ( see https://riptutorial.com/cplusplus/example/31798/digit-separator ).

Update All tests run, 98% pass. The failing tests are on serialization/deserialization (see below)

I think he meant that spidermonkey 68 requires at least c++ 14 to be used. This is unfortunately unavoidable.. I will add this change to the changelog.

README First: The official topic about 0ad spidermonkey update is here: ==================================================================================================================================== Hi there! DISCLAIMER: This is a work in progress. The game starts but might be subject to instabilities, crashes or other imperfections. Just for fun and to learn something new I started adding SM68 support to 0ad. Here you can find the link to my 0ad fork at https://github.com/bellaz89/0ad/tree/rework I don't know what kind of development milestones are planned for 0ad. This is a pure one side project that doesn't pretend to be merged (but of course, if it is useful to someone I am going to be happy) Also note that I do not assure that everything I did is reasonable since this is my first approach with 0ad source base and spidermonkey. Some code could easily be incorrect or redundant. To compile it a system version of mozjs68 has to be installed. use the --with-system-mozjs68 workspace generation option. Tested and developed only on linux (Debian Bullseye 11). Some changes I did: Switched to C++14 due to Spidermonkey 68 requirements JSRuntime direct access is deprecated in favor of JSContext. Therefore ScriptRuntime create a new JSContext. All the methods/functions that used JSRuntime now use JSContext. the Compartment is now JS::Realm Since only a single JSContext is allowed per-thread, ScriptInterface just generates a new global object and populates it. The environment separation is done changing the Realm rather than using a different JSContext. Now the Realm holds the ScriptInterface private data. A macro to switch the Realm was added (CX_IN_REALM) Removed JSAutoRequest Starting adding explicit JS error checks since JS_SetErrorReporter was removed. A lot of API fixes, some are just name changes See links below to get a partial (quite long) list of additional API changes. Increased runtime size to 512 MB to make the tests run without segfault (to optimize). Why increasing the runtime size helped? Support of ArrayBuffer in Serialization/Deserialization is explicitly added since ArrayBuffer is matched by JS_IsArrayBufferViewObject anymore. The used API can be found in js/ArrayBuffer.h Open questions: How to pass data between Contexts. The StructuredClone API was changed and there are some problems continuing using it https://groups.google.com/forum/#!topic/mozilla.dev.tech.js-engine/ViE6Yg8yCvY Script sources are passed with JS::SourceText. Doubts about the right conversions between string format to be done Are the optimizations on SM45 GC still valid in SM68? How passing data efficiently between Realms Update June 8 2020 Basic error tests fixed (updated to mozjs style error output) All tests run, 98% pass. The failing tests are on serialization/deserialization Update June 9 2020 ArrayBuffer tests fixed modifying the serializer/deserializer. Fixed UTF test removing U+D800 (high surrogate) unicode character. All tests run, 99% pass. The last failing test is on exception catching. For the moment it can be ignored. Starting to understand why the GUI doesn't show up when launching pyrogenesis Update 2 Fixed GUI with https://code.wildfiregames.com/D2768 Still crashing when changing page. Update 17 June Patched Spidermonkey 68 source code. the class SharedArrayRawBufferRefs in ./js/public/StructuredClone.hStructuredClone.h (firefox source directory) was tagged as JS_PUBLIC_API to export its symbols. Please note that since Spidermonkey 68 clang is required to compile the codebase (at least on linux). Use instructions in https://github.com/mozilla-spidermonkey/spidermonkey-embedding-examples/blob/esr68/docs/Building SpiderMonkey.md. PersistentRooted is initialized with .init() Applied https://code.wildfiregames.com/D2768 ObjectToIdMap is based on GCHashMap Update 18 June Corrected a cross-compartment bug in CGame::ReallyStartGame, line 329. Does this affect the current 0ad version too?? Added checks to consume error messages I was able to start a game! ...but almost nothing (except the gui!) works properly Update 22 June Fixed error handling issue. Now 100% tests are running It was found that non-ASCII characters in game schema resulted in a crash. Since the presence of these characters is non-essential and limited, these characters were temporarly removed (Identity.js). The problem is under investigation. Probably this is due Game is running with Spidermonkey 68 Todo: Clean up. (better naming.. remove failed attempts to correct issue in the code) Benchmarking/Profiling. See how much the performaces increase/decrease after the upgrade Try link the game with SM78 beta () A big thanks to the WFG team for helping me in this task even though it doesn't follow the 'normal' 0ad development process. I hope this will be helpful in porting the official branch to newer versions of Spidermonkey (New) Spidermonkey 78 Since the next firefox 78 ESR is going to be released this month. I also started to work on the SM78 port using the beta release. Here you can see the branch https://github.com/bellaz89/0ad/tree/rework78 I am able to run a game but I get a lot of failing tests. So expect even a more instabilities. Important: In SM 78 uneval() and .toSource JS methods were removed. Therefore a workaround has to be decided Changes done for SM 78: Lot cosmetic API changes (different method names, method moved to different workspaces..) Seems that it is not possible anymore to disable the dynamic VM grow (to confirm) Needs C++17 to compile Useful links (contain a more in-depth list of changes): https://github.com/mozilla-spidermonkey/spidermonkey-embedding-examples/blob/esr68/docs/Migration Guide.md http://mozilla.6506.n7.nabble.com/Success-with-embedding-ESR52-spidermonkey-td369034.html https://groups.google.com/forum/#!topic/mozilla.dev.tech.js-engine/Mu1arXbBhi8 https://github.com/mozilla-spidermonkey/spidermonkey-embedding-examples Happy Screenshots: Tests. all passing Finally the game runs! mainlog.html entity.zip interestinglog.html