Jump to content

SpiderMonkey build changes


Recommended Posts

Short version: Please complain if the game no longer compiles properly.

Long version: I've updated the code and build system so that it uses a new version of SpiderMonkey (the JavaScript engine used by Firefox). We used to use version 1.6, from 2006; now it's a version that's about a day old (picked at random from the mozilla-central repository, plus a patch that's not been committed there yet). SpiderMonkey's build system has been redesigned since 1.6, and I've rewritten the way we compile and link it, so it's all somewhat different.

Hopefully things will work automatically. On Windows, there's nothing to do. On Linux, when you first run update-workspaces.sh it will try to build SpiderMonkey and ought to work and then carry on as normal, but errors are possible. OS X theoretically should work like Linux but I haven't tested it at all.

One simplification is that we no longer use a threadsafe version of SpiderMonkey (we do use it in multiple threads but each individual JS runtime is only used in one, and the patch mentioned earlier makes it safe to do that), so there's no need for NSPR when compiling. The other main benefits are that the interpreter is faster, it has a JIT to make things much faster (though it's disabled for now), it has better debugging tools, and it supports new language features (these and these). It'd probably be good to update to a newer version at some point in the future, particularly when the JägerMonkey JIT is integrated and stable, but this should do for now.

Link to comment
Share on other sites

Have you run update-workspaces.bat again and done a Rebuild in VS? Might be necessary to do that before it'll work. (It looks like it compiled successfully in the autobuilder so I believe I didn't just forget to add all the files, though that's always possible too (y))

Link to comment
Share on other sites

Hi all,

I am following the progress of this project for quite a while and finally found the time to check out the sources to give it a try and build it myself.

I just wanted to share my thoughts and findings.

Here is the diff for libraries/spidermonkey-tip/build.sh to get update-workspaces.sh to run fine on my mac.

I hope this will help someone else, so here it is.

--- libraries/spidermonkey-tip/build.sh	(revision 7370)
+++ libraries/spidermonkey-tip/build.sh (working copy)
@@ -61,6 +61,12 @@
DLL_EXTN=dll
LIB_EXTN=lib
LIB_PREFIX=
+elif [ `uname` = "Darwin" ]
+then
+ INCLUDE_DIR=include-unix
+ DLL_EXTN=dylib
+ LIB_EXTN=dylib
+ LIB_PREFIX=lib
else
INCLUDE_DIR=include-unix
DLL_EXTN=so
@@ -72,8 +78,8 @@
mkdir -p ${INCLUDE_DIR}/debug/js
mkdir -p ${INCLUDE_DIR}/release/js
mkdir -p lib/
-cp -uL src/build-debug/dist/include/* ${INCLUDE_DIR}/debug/js/
-cp -uL src/build-release/dist/include/* ${INCLUDE_DIR}/release/js/
+cp -L src/build-debug/dist/include/* ${INCLUDE_DIR}/debug/js/
+cp -L src/build-release/dist/include/* ${INCLUDE_DIR}/release/js/
cp -L src/build-debug/dist/lib/${LIB_PREFIX}mozjs-ps-debug.${LIB_EXTN} lib/
cp -L src/build-release/dist/lib/${LIB_PREFIX}mozjs-ps-release.${LIB_EXTN} lib/
cp -L src/build-debug/dist/bin/${LIB_PREFIX}mozjs-ps-debug.${DLL_EXTN} ../../binaries/system/

Cheers,

ToBe

Link to comment
Share on other sites

(y) Is there anyway to automate that? Perhaps some script that reminds about updating workspaces when I try to compile? Or do I just need to change my desktop wallpaper with update workspaces in big letters?

[edit]

I suppose I should add here that I got it to compile on vc2005 after updating workspaces. Also, it runs alright.

Edited by WhiteTreePaladin
Link to comment
Share on other sites

update-workspaces is "only" required when files are added/removed to the project. Since it does take a little while (VS isn't great about reloading solutions), it's best not to run this before every build.

I guess hooking into SVN would be possible for this purpose, but definitely overkill. The most economical solution is to get into the habit of checking for relevant SVN messages (whether a file was added/removed) and/or to try updating the workspace whenever build failures are encountered :P

If there's a nicer solution, I'd love to know about it.

Philip: build indeed succeeded on Windows here (VC2005) (y)

Link to comment
Share on other sites

No.

SpiderMonkey isn't designed around regular stable releases - the versions that have been released are very old, the version that some people are planning to release soon is also very old, and the developers aren't interested in changing that (since they're mainly focused on Firefox; the recommended process for embedding SpiderMonkey is to just pick a recent snapshot and bundle it with your own code, like we do).

The behaviour between versions often changes in incompatible ways, so using a different version to what's in SVN is quite likely to introduce test failures and subtle bugs and network out-of-sync errors. The particular version we're using also has a patch to avoid some thread-safety issues - using a different version means the game will randomly crash or corrupt memory in rare situations, which is not good. So you shouldn't do that (y)

Why do you want it to use the system library?

Link to comment
Share on other sites

No.

SpiderMonkey isn't designed around regular stable releases - the versions that have been released are very old, the version that some people are planning to release soon is also very old, and the developers aren't interested in changing that (since they're mainly focused on Firefox; the recommended process for embedding SpiderMonkey is to just pick a recent snapshot and bundle it with your own code, like we do).

The behaviour between versions often changes in incompatible ways, so using a different version to what's in SVN is quite likely to introduce test failures and subtle bugs and network out-of-sync errors. The particular version we're using also has a patch to avoid some thread-safety issues - using a different version means the game will randomly crash or corrupt memory in rare situations, which is not good. So you shouldn't do that (y)

Why do you want it to use the system library?

Packages provided by linux distributions usually prefer to depends on a shared library from another package rather than providing the same library in different packages (e.g. when there is a security problem the distribution only need to update 1 package; also save space on drive).

Debian provides spidermonkey inside xulrunner packages that are already used by other packages. This is what I get on an Ubuntu 10.04:

$ apt-cache rdepends xulrunner-1.9.1
xulrunner-1.9.1
Reverse Depends:
zekr
xulrunner-1.9.1-testsuite
xulrunner-1.9.1-gnome-support
xul-ext-flashgot
|xul-ext-flashgot
xul-ext-firecookie
|xul-ext-firecookie
xul-ext-firebug
|xul-ext-firebug
videolink
python-hulahop
|mozzemberek
mozilla-nukeimage
miro
libgtk2-mozembed-perl
libgtk-mozembed-ruby1.8
libgjs0
libgjs0
kazehakase-gecko
galeon
xulrunner-1.9.1-dev
xulrunner-1.9.1-dbg
$ apt-cache rdepends xulrunner-1.9.2
xulrunner-1.9.2
Reverse Depends:
xulrunner-1.9.2-testsuite
xulrunner-1.9.2-gnome-support
xiphos
tuxguitar
prism
mozzemberek
moonlight-plugin-mozilla
google-gadgets-xul
gnome-web-photo
fennec
eclipse-rcp
conkeror
chmsee
yelp
xulrunner-1.9.2-dev
python-gtkmozembed
libswt-mozilla-gtk-3.5-jni
couchdb-bin

IIRC some time ago 0ad was able to link to the system spidermonkey.

Link to comment
Share on other sites

Packages provided by linux distributions usually prefer to depends on a shared library from another package rather than providing the same library in different packages (e.g. when there is a security problem the distribution only need to update 1 package; also save space on drive).
Okay - I agree that's usually good, but I'm not convinced it's the best tradeoff in this specific case. In its current state the game is very insecure, and bugs in SpiderMonkey are the least of its worries - it'd be better to focus on reviewing and fixing the game's code first, so it never uses null pointers or overflows buffers etc. The disk space impact is minimal - it's well under 1% of the size of the game's data files. In most cases we have no reason to prefer a specific library version, so we use the standard system packages as much as possible, but I think the drawbacks for SpiderMonkey (the risk of different bugs and features and incompatibilities between versions, given that the game relies heavily on its precise behaviour and performance, and the time wasted on debugging those problems) outweigh the benefits now.

Using xulrunner sounds like a better idea than using the standalone JS releases, since it's much more up-to-date, but it's got sub-optimal build settings (we want non-JS_THREADSAFE builds now to avoid complexity and performance overhead) and we'd still have compatibility problems whenever a new major version is released, and it would delay us adopting new or experimental SpiderMonkey features (e.g. the new JIT).

IIRC some time ago 0ad was able to link to the system spidermonkey.
I consider that to be a bug (y)
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...