Itms

Completely agree. The only questionable thing we find in a few mods is the code that removes the mod itself from the list of enabled mods. Players should be aware that their opponent is using autociv or any other mod (in theory: as I said, they can't know if their opponent changed the code locally).

In this situation, it's you who doesn't understand me: for the last time You need to use Linux to run the packaging script BEFORE using nsisbi on Windows. I believe there is no problem in the code, it's just that you don't use or don't have the correct tools. I could test but I'm working on SpiderMonkey at the moment, and however much time I loose repeating things to you it's still quicker than packaging the game, which takes a couple hours. If you make the effort of understanding what's blocking you, then I can answer your questions if they are clear. No problem with that.

No, it doesn't need any update!

You need to run source/tools/dist/build.sh on Linux, then you can use nsisbi on Windows (instead of makensis on Linux) to create a Windows installer. What do you not understand? Can you stop pinging people all the time? Can you formulate questions instead of harassing us?

Maybe but why do you say that? The big difference is in the unlogged code. at first sight it could be anything, no?

I agree it's going out of topic. In the staff forums we have been saying that I will stop signing mods that change the mod detection code. For instance autociv "hides itself" from the list of enabled mods, specifically in multiplayer. I can't "unsign" it, but all mods will need a new signature for A24, at which point autociv and others will have to remove this hiding feature in order to appear on mod.io. As odalman mentions, this is not going to fix the underlying problem. Players can install non-signed mods manually, they can even modify the engine code if they have the skills. Out of sync errors will prevent players from directly modifying the simulation to their advantage, but they can always tweak the UI, implement auto-train and other things... The change to signing is mainly made in order to be fair and more open to players using mod.io who might be unaware of autociv's (and others) internals. I'm not sure we can do more without resorting to complex measures like using servers.

gameboy did you run the packaging script on Linux first as needed? NSIS is only the last step of packaging.

I think mistakes or inconsistencies can be easily fixed by just adding Balancing as reviewers manually.

Sounds good. I suggest not touching the O11 code ownership. Code owners are sufficiently complicated as they are. I think we should keep restricting code owners to team members @Stan` I will create instead a Balancing user group with the Herald rule Nescio proposes

In this case the best thing to do would be to create a Herald rule which is triggered when a diff title contains [gameplay]. Anyone can create such a rule, to be notified whenever they want The so-called "Code owners" (O11 and the like) are only determined by paths of modified files. I think it's still interesting to have that.

Ah! When seeing your last message I thought you had discovered a bug. But I just discovered that Stan has added you to the list of owners of the Balancing changes. You are basically an automatic reviewer for all those changes. @Stan` you probably shouldn't do that before explaining to people that it will flood their notifications/inbox

You're not subscribed to this diff, you shouldn't receive any notification about it. Are you talking about the public Activity stream? You cannot possibly catch up with everything. You should rather manually subscribe to what you are interested in, and monitor your emails or notifications depending on your preferences (by notifications I mean the small bell icon on the top left). This is a bit confusing indeed. It is a summary of the pending actions you would perform if you clicked "Submit". By default it would subscribe you. If you write inline comments, or choose actions in the Add Action.. menu, they will be added to the summary, waiting for you to press Submit.

I don't understand. When you click on the link above, you get to this page, right? As written there, your notifications should already be limited to what you are subscribed to. Do you have an example of a change that you would not have wanted to receive a notification for?

Here is the page for your account For others, click on your avatar on the top-right of the page, then Settings, then Email Preferences (which also handles notification preferences).

(I deleted the duplicate topic) The build restrictions are handled in binaries/data/mods/public/simulation/components/BuildRestrictions.js. It is there that you can find the logic behind where buildings can be placed. You would need to add some information to the templates of the buildings (in binaries/data/mods/public/simulation/templates) In the BuildRestrictions component, you will need to query the engine's TerritoryManager. Maybe you'll need to tweak it as well. It is implemented in source/simulation2/components/CCmpTerritoryManager.cpp.

Ah, I see it as well. Which one should I rather sign? Signing both sounds like a big overlap.

@Stan` @andy5995 I think I managed to "undelete" Hyrule Conquest and Vision Sharing. However I cannot see the Korean pack anywhere.

Hi andy, sorry I hadn't seen this topic, so I didn't know I had to review the mod. It's better to send me a PM (even though I can be long to answer, at least I know I have a mod to sign). I haven't deleted/rejected any mod, so I don't know what happened with yours. EDIT: I think one of the mod.io developers cleaned up all pending/incomplete mods from the whole platform.

There is a confusion here: we do use the same engine as Firefox. However, we need to build the game with it, so we need the development files of SpiderMonkey. When one installs Firefox, they only download the compiled Firefox which includes (or comes alongside) the compiled SpiderMonkey. Similarly, when we release 0 A.D., we distribute the compiled SpiderMonkey which is quite small. 0 A.D. end-users can download the compiled SpiderMonkey to run 0 A.D.; on the other hand, they cannot build 0 A.D. For that, one needs the development files which are heavy. If you want to take a look, in some package managers, the former is called mozjs52, the latter is called libmozjs52-dev. That is because you built SpiderMonkey. If you checkout a fresh SVN copy, that folder is 98 MB (which is already heavy: it's the development files). It is worth mentioning that SpiderMonkey devs improved the size of the source SM tarball so in SM52 that folder is now only 35 MB. When you build a big project such as SpiderMonkey, the compiler outputs a huge lot of intermediate build binary files that take up a lot of space. If you want to reclaim that space after building, you can delete the folder libraries/source/spidermonkey/mozjs-45.0.2/. The built files, needed by 0 A.D., are copied elsewhere at the end of a build. Not really. It is SpiderMonkey that constrains which C++ version we support. Right now, it's a lot of work to migrate to C++14/17: that work is upgrading SM Once SM is upgraded, supporting new versions consists of a three-line change in premake and possibly some small adjustments. Upgrading SM also means dropping some old compilers, which is work, but it's communication work with package managers rather that programming work.

Sorry I'm only looking at this now, but you haven't installed any mods by any chance? If not, can you delete all the folders that are in "Documents/My Games/0ad/mods/"?

Not necessarily. I am going to try to update SpiderMonkey as far as possible in A24. I don't want to unnecessarily delay the release of A24, but the less obsolete SM is, the fewer problems we will have.

Thanks! No, I didn't need to patch SM to use the clone API. It probably breaks in a more recent version.

Removal of ObjectToIDMap In source/scriptinterface/third_party/ObjectToIDMap.h, we have some code, taken from Firefox, allowing us to keep a map where JavaScript objects are the keys, and the values can be anything (integers or strings in our case). This code was upgraded in Firefox 45 and the old version, which I had forgotten to update, stops working with SpiderMonkey 52. However, I discovered recently that the new code doesn't work either! SpiderMonkey developers tell me it is never actually used in Firefox. In fact, this code is not needed. If we have an object to which we want to attach an ID, we can use Symbol properties. JS symbols act as unique property keys that allow us to safely assign an ID to an object. Additionally, they are ignored when iterating over an object's properties, so the rest of the code won't know about them. f1fa7a1 (up for review at D2897 in Phabricator) In the AI manager, we used object maps in some code about "serializable prototypes" which is completely dead: no code calls into it. So I removed it. @wraitii has written D2746 which actually implements prototype serialization, but he tells me he would rather rebase his work after the upgrade. e9f2161 : I replaced the other use of object maps, in the serialization code, with symbol properties. 727f26c : This allowed me to remove that code entirely. This means less external code to maintain! Sometimes what we actually need is a map with custom keys, and JS objects as values. Currently, we do that using std::map<KeyType,JS::Heap<JSObject*>>, but I believe we should use JS::GCHashMap. I'll look into that in the future. Other cleanups 43bb449 : Previously, in SM, objects rooted on the stack did not have a default constructor, so we had created our own class wrapping them with such a default constructor. Nowadays, this wrapper is not needed anymore, so I removed its remaining use and deleted the class. Also less code to maintain. 58ba02e : I performed an optional cleanup that would make the refactoring below easier. c148184 : We have a ScriptInterface::IsExceptionPending function that just calls a SpiderMonkey function with an unneeded GC request, so I removed it. TODO: I did that under SM52 and I don't know if it works under SM45. If it does, it should be committed before the upgrade. Refactoring of runtimes, contexts, and compartments This is the main part of the upgrade Until SM45, there were three levels of containers in the SpiderMonkey API. Each thread of the engine needed to have a runtime to execute some JavaScript. In each runtime, there could be several isolated contexts. In our case, we had one context for the simulation, another one for each page of the GUI, etc. The isolation of contexts allows us to separate properly the different parts of our code. Inside contexts, there could be several isolated compartments, but we didn't use that. Each context had its own and only compartment. In our code, JSContext is abstracted by our ScriptInterface class. We also have ScriptRuntime for JSRuntime. In SM52, the runtime and context levels are merged. Runtimes are removed from the API. There must be one context per thread, and isolation of the different parts of a thread must be done using different compartments. This is summarized in the following picture made by @Bellaz89: The difficulty here is that calls to the SpiderMonkey API still use a context. Previously, it was enough to get the context from the script interface, and call SM: the target code would be in the correct compartment. Now, getting the context is not enough: we first need to make the context enter the compartment of the script interface, and only then, we can call the SM API on the context. In order to prevent mistakes, I decided to perform a big change. I think it's better to prevent getting the context from a script interface directly. I wrote a Request struct which allows one to get the context from a script interface, and which automatically enters and leave the associated compartment. In the past, we had many issues with JSAutoRequest, which is needed to properly handle the memory of the JS runtimes. In a lot of cases, we forgot to add JSAutoRequest, which created bugs. The new Request object allows us to prevent these mistakes, because it automatically opens a memory request as well. The changes go like this: c65093e and 25dc831 : Prevent getting the context, except through a Request that acts just like JSAutoRequest. I made two commits for readability but they should be committed together. This change removes quite some boilerplate code previously caused by JSAutoRequest. 5d4e4c4 : Rename "context private" to "compartment private" (this commit is huge but changes nothing, just the name of classes and variables). 8142e29 : Here I make the central change: each ScriptRuntime now has one single context, whereas each ScriptInterface is associated with a compartment. Getting the context through a Request automatically enters the correct compartment. The Request also provides the global object associated with the compartment. It is still possible to get the general context but this is now advertised as unsafe. 521fd88 : There is a check made on the creation/destruction of runtimes, I change it to be made on the creation/destruction of contexts. 17ce48e : Rename ScriptRuntime to ScriptContext to match its new role (this commit is big but changes nothing, just the name of classes and variables). 0 A.D. is now ready to use SM52! Actual SM upgrade be5b0ed and 3013473 : In two commits, I upgrade the SM52 files, including a prebuilt version for Windows, and I adapt the build and premake scripts to use the new version. Simple API changes A lot of commits are made to match the API changes. They are very simple. I will merge them as a single commit in the future, along with links to SpiderMonkey's bugtracker. Right now I keep them separate so that I have a list. Some of them deserve a note: c4fc364 : In this one I remove all uses of runtimes, which become "unsafe contexts" with no compartment barrier. 8c6476a : In this one I use the new API for structured clones. Those take a new argument which is a scope, depending on whether the clone will be used in the same thread, or even in the same run of the application. My choices of scopes have to be reviewed. Changes in the error reporting API cc31596: The heaviest API change is due to the error reporting made differently. Previously, both errors and warnings in JS code would make SM call the error/warning reporter, which we implemented ourselves. Now, only the warning reporter can be supplied. Errors are always handled by the exception system, in which all exceptions have to be caught. Sometimes, they are caught by the JS code itself, but sometimes they are not. In this case, the engine has to catch them and display them. If they stay uncaught, the engine can crash when a new exception is raised, or when performing a core operation such as creating a new compartment. Thus, I had to refactor a big part of our JS error handling. I think I have improved the design which was a bit hacky, but my code is not foolproof, as uncaught exceptions are maybe still possible.

Hi everyone In this thread, I am going to present the SpiderMonkey upgrade work I have dedicated myself to since around a month. The aim of the thread is to explain what this upgrade is, why it is such a big change, and how I organized it. Please don't hesitate if you have comments or questions on anything! What is SpiderMonkey? SpiderMonkey is Mozilla's JavaScript engine, used first and foremost in Firefox. As you probably know, 0 A.D. is written in C++ but a large part of the gameplay is programmed in JavaScript, in order to be easily modifiable by mods. In order to execute the JavaScript code of the game, we include SpiderMonkey inside 0 A.D. (this is also called embedding). We are not the only ones to embed SpiderMonkey: for instance, GNOME (a well-known desktop environment) also embeds SpiderMonkey. I will often use the acronym SM to designate SpiderMonkey. No kinks here, just laziness to type! All about upgrading The advantage of embedding SM is that we don't have to take care of bugs or complicated JavaScript interpretation quirks: the folks at Mozilla have us covered. The downside, however, is that we are often out of the loop on the evolution of SpiderMonkey. In order to get the improvements of the engine, we have to upgrade it to recent versions, and this is very complicated. A big part of 0 A.D. is built upon SpiderMonkey, so when the latter changes, we have a lot of work to do to adapt. On top of this, SM upgrades are big changes. SM evolves along with Firefox (which has a new release every 8 weeks), but a stand-alone SM version is only released for embedders approximately once a year, when Firefox releases an Extended Support Release (a kind of Firefox LTS, which is, for instance, the one available on Debian). See the Firefox release calendar. The difficulty of upgrading is that, between two ESR releases, SM accumulates a lot of changes, and, whereas Firefox is always on top of them, other front-ends like 0 A.D. are faced with numerous breakages which could create bugs. Additionally, at 0 A.D. we haven't upgraded SM for a long time due to the difficulty of upgrading. Thus, it is also difficult to find information to upgrade to SM releases that are already outdated. It is not much easier to jump to the current SM release: so many changes have happened that we would be unable to make sense of them all at once. Current status I upgraded SM to version 38 in the ends of 2016. At that point, we were mostly up-to-date: version 45 was already out but it was still getting some fixes, and the stand-alone SM45 was not officially released. Nowadays, stand-alone SM versions are almost never officially released, however the SM development team has started to put out more and more news (see their brand new website) so things are certainly getting better. I couldn't work on the SM45 upgrade until the summer of 2019, and even then I left a few bugs in, that we will discover in the next posts I am releasing today the upgrade to SM52, which is a big one (hence the forum thread). That brings us to "only" two years late, as SM52 stopped receiving fixes in June of 2018. It is now necessary to work on the upgrade to SM60, SM68, and SM78 (the latter is still getting fixes until the beginning of 2021). However, I am going to work on other urgent things in August. I need to improve a lot of things with the CI, Jenkins and the handling of patches, and I would like to allow contributors to use git instead of SVN. I hope these exciting news will make you forgive me for delaying the SM upgrades In any case, I want to get back to further SM work as soon as possible. In this other thread, @Bellaz89 did the daunting work of making SM68 work with no previous knowledge of 0 A.D.! Many of you probably saw it. Bellaz was then very helpful and answered all my questions concerning the upgrade. His spadework, which cannot be committed just as-is, made the actual upgrade a breeze; and apparently there will be no big work to perform on 0 A.D. between SM52 and SM68, apart from an encoding change. We will see... Overview of the upgrade The upgrade can be found at this git branch. It is very big. Not counting the changes to SpiderMonkey itself, in the engine of 0 A.D., it consists of 145 files changed, with 2667 insertions(+) and 3243 deletions(-)! The objective is to split the massive change into independent parts, which can be understood, tested and committed one by one. I tried to make a majority of changes before the actual upgrade. That way, the upgrade itself is leaner. There are a few changes that I will be able to do just after the upgrade. Right now they are not uploaded, but I will announce them here later. I will also keep the posts below updated whenever I update or commit a part of the upgrade. You are probably wondering about the performance. From my few tests, the performance is identical in SM45 and SM52. If I'm not mistaken, the structural changes in SM52 pave the way for performance improvements in the next SM versions. On top of this, I have written a patch for a huge performance bottleneck (D2919) that was very apparent under SM52, and I have identified a couple of easy improvements that I will list in a ticket soon. Finally, Bellaz thinks that we have a few JIT optimizations (i.e. whether JavaScript code should be compiled, which costs some time, but makes it faster) which have become counterproductive. We could try and tweak our JIT options and see if we can improve the performance that way. You can already checkout the branch and test it! Now, to the details for the curious programmers!