[Discussion] Spidermonkey upgrade

[Radagast.]

Interesting to see this more closely. I'm afraid my knowledge of Assembly is limited too. My guess is that the optimizations have also stripped the segfault version of the push to stack command of the JS::Value because the optimizer realized there was no longer a need for it.

So if the compiler gets the order wrong the code changes drastically if optimization is turned on ..

Your comments are helpful. Especially as assembly differs a lot inbetween hardware types.

Have not really used that tool so far ... essentially even a profiler can't be trusted on a low level. Tools are needing a lot of time to set up to work properly. Currently I have lost my eclipse to an update (why did I initiate this update, I had a bad feeling with it anyway), revert is failing too, so this will cost me some time to fix. Have I ever mentioned that I don't really like this endless virtual programming world? Spending time into nowwhere, you know what I mean as you are the poor guy to deal with SpiderMonkey updates. :/

[Yves]

Philip said he had a similar issue with VS2008 when upgrading to SpiderMonkey 1.8.5.

He had to define JS_NO_JSVAL_JSID_STRUCT_TYPES and build SpiderMonkey with that definition on Windows.

I've tried this now but unfortunately it doesn't work here.

During testing I noticed that the error only happens in debug mode in my example program. In 0 A.D. it happens in release and debug mode though.

I've asked on the SpiderMonkey newsgroup.

I also found something interesting about the different ways the function is called here.

The return value is saved depending on its size:

32-bit or smaller values
Widened to 32 bits and stored in the EAX register
64-bit values
Stored in the EDX:EAX register-pair
Values larger than 64 bits
Stored in memory, which EAX then points to.

The working version uses the convention for values larger than 8 Bytes / 64 bits.

The segfault-version uses the convention for 64 bit values.

It still looks like the first version only reads 8 bytes though (otherwise it would need more lines that copy from an offset of eax like eax+8 and eax+12):

  004117FC: 8B 10              mov         edx,dword ptr [eax]        ; expect the pointer to the JS::Value data in eax. Copy the first four Bytes to edx  004117FE: 8B 40 04           mov         eax,dword ptr [eax+4]      ; expect the pointer to the JS::Value data in each. Copy the second four Bytes to eax

[Yves]

Problem solved!

It looks like a bug in VisualStudio, but passing the JS::Value as reference instead of using a return value works around the problem.

It was a bit of work to adjust all implementations of ToJSVal and change all call-sites, but I don't see any disadvantages for the final result.

I've registered at Github too and created a branch for the SpiderMonkey upgrade.

You can get it here: https://github.com/Yves-G/0ad

It should build as normal on Windows and Linux now and shouldn't require any additional steps. Feedback is appreciated but be aware that it still contains some known issues too.

EDIT: On Linux you need to install libnspr-dev

[Radagast.]

Hey, well done!! Interesting ... a bug in an IDE compiler? oh gracious, I don't wish to debug that ...

This means the SpiderMonkey is history? No more mess with it? Oh, and the JavaDebugger will work now again (if we really need it) or am I just mixing it all up again?

[Yves]

This means the SpiderMonkey is history? No more mess with it? Oh, and the JavaDebugger will work now again (if we really need it) or am I just mixing it all up again?

Yes, you're mixing it all up!

I'm upgrading SpiderMonkey, so only version 1.8.5 will be history for us.

The JS Debugger will have to be reimplemented with the new version, that task isn't part of the upgrade.

The upgrade isn't done yet but it's close to completion.

After this upgrade we'll already have to start with the next one. It should be much easier now but there are still some time consuming tasks involved like preparing for exact stack rooting and a moving garbage collector.

[agentx]

After this upgrade we'll already have to start with the next one.

Do I have to change from this module pattern? It is excellent!

HANNIBAL = (function(H){// do stuff with Hreturn H; }(HANNIBAL));

And what is the issue with the debugger? It might save me some time, so if there is anything I can do except compiling, let me know.

[Yves]

Do I have to change from this module pattern? It is excellent!

No, I'm quite sure this module pattern will continue to work in the future because it doesn't use special Javascript features or anything like that.

The pattern is so commonly used that it will have to be supported for future versions.

With the ES6 specification they even try to integrate the idea of modules directly into the language.

So maybe we could simplify the notation or get some additional benefits at some point. I haven't yet checked how this new type of modules will work exactly.

Get some information about these new modules here.

And what is the issue with the debugger? It might save me some time, so if there is anything I can do except compiling, let me know.

The issue is that the C++ debugger API changed even more than the rest of the JSAPI and the whole server component of the debugger has to be rewritten.

There's a Javascript API which is more stable and it would probably be a good idea to use that instead.

I also wondered if there would be a way to somehow connect existing debuggers more directly to avoid maintenance work, get additional features and have a UI that people are already used to.

[Yves]

The SpiderMonkey upgrade is ready for review.

Check the comments on trac and get it from my github repository if you want to help testing and reviewing.

[agentx]

The issue is that the C++ debugger API changed even more than the rest of the JSAPI and the whole server component of the debugger has to be rewritten.

There's a Javascript API which is more stable and it would probably be a good idea to use that instead.

I also wondered if there would be a way to somehow connect existing debuggers more directly to avoid maintenance work, get additional features and have a UI that people are already used to.

I've tried to understand the SciptInterface C++ code in 0AD, but I'm far away from understanding what's going on. Let aside the changes you've mentioned. Apparently it is now possible to connect a gdb? Not sure whether this provides JS variable inspection. What I understand very well is your proposal to connect the firefox debugger to 0AD and start JS remote debugging, like with an Android device. That's the no questions left solution. Is it more than a dream?

I'm not especially fond of every feature introduced in upcoming EcmaScripts. The modules rationales are rather unspecific, the performance impact unknown and whether IonMonkey can deal with a cloudy issue. So far it looks like the only gain are a few bytes per file. It may take a long time until third party AI modules are ready to plugin in via this extension. However, there is a notable exception: fat arrows.

(() => "Hello World")() // try in browser's console

Will they land in Alpha 16?

[agentx]

Is it more than a dream?

Yes: https://github.com/cocos2d/cocos-docs/blob/master/manual/framework/native/scripting/javascript/js-remote-debugger/en.md

[Radagast.]

Don't really know where they have their implementation of it. Can't even locate this ScriptingCore class ... is it within SpiderMonkey?

Searched almost all their repositories for this class. I think Yves has better chances. This is where they have their JS-bindings ... spidermonkey. But which version they use?

https://github.com/cocos2d/bindings-generator/search?q=ScriptingCore&ref=cmdform

Edit: Interesting, they also have a 3D engine with physics. Perhaps we can inspire at some points as they can inpire form pyrogenesis. Though the target is a bit different.

Edited March 27, 2014 by Hephaestion

[agentx]

Don't really know where they have their implementation of it. Can't even locate this ScriptingCore class ... is it within SpiderMonkey?

Improve your Chinese

http://casperise.com:81/casper/cocosino/raw/8532c58fddd659e2dc4f757dd656ee704cd3fb67/cocos2dx/scripting/javascript/bindings/ScriptingCore.cpp

[Radagast.]

You know what I am thinking? Trying to learn Chinese for a long time.

Crazy, you found it in the same directory I were. Didn't know GitHub's search function doesn't examine the filename. That's interesting, but not good.

Would be cool, if we could sort it out for Yves. So that he hasn't to find it.

Btw. Did you see we now have Petra AI? Will have a dozen AIs round soon. Yours. My humble one. It's getting interesting. (if we could get a decent debugger ... I really fear javascript, it's costing hours to find the odds especial in asynchronous affairs.)

[Stan`]

My humble experience in Javascript lead me to JSFIDDLE and JSLINT, can they help you ? Also VS2012 supports javascript.

[Radagast.]

Have to wait for master Yves, see no other chance.

Hm, I could set up eclipse. If it were not used a major other project currently. I really don't fear more than breaking it at this moment ... I had some fun with it already, electricity failure, plugin failure, revert failure, dependencies mixed up like magic. No.. really not. Though I could install another eclipse nearby. How could I forget about that? Have to give it a try, would make things a lot easier, currently I am using Vim, it's ultra-powerful .. if you have the knowledge .. so GDB could be used. Still for debugging eclipse is a real speed-up. Thx for the input.

[Stan`]

Have to wait for master Yves, see no other chance.

Hm, I could set up eclipse. If it were not used a major other project currently. I really don't fear more than breaking it at this moment ... I had some fun with it already, electricity failure, plugin failure, revert failure, dependencies mixed up like magic. No.. really not. Though I could install another eclipse nearby. How could I forget about that? Have to give it a try, would make things a lot easier, currently I am using Vim, it's ultra-powerful .. if you have the knowledge .. so GDB could be used. Still for debugging eclipse is a real speed-up. Thx for the input.

You're welcome. Didn't know Eclipse supported javascript.

Edited March 27, 2014 by stanislas69

[agentx]

You know what I am thinking? Trying to learn Chinese for a long time.

Crazy, you found it in the same directory I were. Didn't know GitHub's search function doesn't examine the filename. That's interesting, but not good.

Would be cool, if we could sort it out for Yves. So that he hasn't to find it.

Btw. Did you see we now have Petra AI? Will have a dozen AIs round soon. Yours. My humble one. It's getting interesting. (if we could get a decent debugger ... I really fear javascript, it's costing hours to find the odds especial in asynchronous affairs.)

Actually it might be even Japanese what I've seen in Google's result list around the hit. I'm not an expert in Asian languages. I wish I had the time to learn one. It would be Burmese, I love the symbols.

Yeah, more bots please. I'll check Petra this weekend, seems like Aegis sources have some potential.

Don't fear JS! It is such a little dirty language. Once you learned not to use the, ummh, bad parts it is fun.

>asynchronous affairs: Have you found the entrance to callback hell?

Btw, I've solved the resource flow math, it is just a ringbuffer and a simple linear regression, very cute.

[Radagast.]

Don't know but I expect it as it's capable of almost everything (even PHP). Why not JavaScript too then?

That is good news, agentx! It sounds really brilliant. I'm happy wraitii and Mimo put in so many AI efforts in the last months, the API rewrite, Aegis rewrite, Petra AI, ... your scripts, decision tree, today night I figured out some essentials about how to let units' dreams (yet another part to add to our plan) influence the daily decisions.

Callbacks are interesting ..

I recently heard Czech is the hardest language because they decided to keep old grammar and written language from centuries ago. While the spoken language has changed so much that now one has to learn two languages.

[Yves]

The SpiderMonkey upgrade is now committed to SVN and an autobuild is done.

You can easily test it now.

I've tried to understand the SciptInterface C++ code in 0AD, but I'm far away from understanding what's going on. Let aside the changes you've mentioned. Apparently it is now possible to connect a gdb? Not sure whether this provides JS variable inspection.

GDB can only be used for the C++ JSAPI code and not for Javascript code.

I think they added some pretty printers that now show more about the values inside the JS types.

Some code changes are needed to attach an external debugger, but I hope there's a way that allows us to use an existing debugger as client and only a little bit of JSAPI code.

[agentx]

The SpiderMonkey upgrade is now committed to SVN and an autobuild is done.

Impressive job! JavaScript now with Set, Map, WeakMap, fat arrows and more. SM 31 will probably bring array comprehensions. Only nitpick are the new warnings with "reference to undefined property". It is a common pattern to exit loops by testing against undefined, no warnings needed.

[historic_bruno]

Impressive job! JavaScript now with Set, Map, WeakMap, fat arrows and more. SM 31 will probably bring array comprehensions. Only nitpick are the new warnings with "reference to undefined property". It is a common pattern to exit loops by testing against undefined, no warnings needed.

I think the warnings are a good thing, as they help raise the awareness that undefined is a special value in JS and should be treated as such. The programmer should know if a variable can ever be in the undefined state without it being a bug, and in that case they should test for it explicitly. I would argue it's bad practice if a variable can either evaluate to false in some other way (e.g. false, 0, "", null, NaN) or be undefined, it should be one or the other by choice, so the type+value test is more appropriate: "if (foo !== undefined)". That's also part of our coding conventions. I think a lot of subtle bugs can be caught by warnings like that (and a good understanding of the peculiarities of JS).

[Stan`]

Can you tell me a specific case were a variable has to be undefined ? I thought this was a bug source even in recursive functions.

[wraitii]

Ais use undefined variables to check for stuffs that are… undefined. Such as a garrisoned unit has no position, so it's position is undefined, and the variable will be too.

I too really like the fact that spidermonkey is stricter.

[agentx]

Ok, then all the warnings I get will be removed soon?

[FLHerne]

Can you tell me a specific case were a variable has to be undefined ? I thought this was a bug source even in recursive functions.

You may be confused with languages like C, where an 'undefined' variable simply has a random value?

In Javascript 'undefined' is a specific state that can be tested for.