WIP: Javascript debugging - Web developer wanted

[Yves]

I've uploaded a new version that fixes some more crashes to the wiki.

There were some problems with rooting Javascript values and some parts with threading problems.

Do you use Codeblocks with JS? Does it do syntax highlighting etc.?

I didn't use it for JS yet and I don't know if there are any JS plugins.

I wanted to develop a Codeblocks plugin first because it's the only supported IDE that runs on all systems at the moment and because I know it.

[zoot]

I didn't use it for JS yet and I don't know if there are any JS plugins.

I wanted to develop a Codeblocks plugin first because it's the only supported IDE that runs on all systems at the moment and because I know it.

Eclipse should be cross-platform too - though it is a bit bloated.

Anyway, I have a feature request: how about a way to set breakpoints through a command-line argument? For instance,

pyrogenesis -breakpoints=/path/to/breakpoints.txt

And then the debugger would set the breakpoints defined (somehow) in the file /path/to/breakpoints.txt.

This would make it feasible to use the debugger with the -replay argument (to debug a recorded commands.txt)

[Yves]

Eclipse should be cross-platform too - though it is a bit bloated.

I know that it's cross platform, but we don't generate projects files for eclipse with premake at the moment.

Premake doesn't even support it unfortunately.

Anyway, I have a feature request: how about a way to set breakpoints through a command-line argument? For instance,

I've thought about something like that too.

For example it should also be possible to debug the game's startup.

I'm not sure yet if I will read breakpoint definitions or if it will just be an option that stops the execution as soon as any JS-Code is triggered.

You could then add all the breakpoints you want using the normal ToggleBreakpoint command.

Could the Web-GUI somehow save the breakpoint definitions and load them? I'd prefer if as much as possible would be done with the Web-GUI because that's cleanly separated from the engine and the game.

[zoot]

I know that it's cross platform, but we don't generate projects files for eclipse with premake at the moment.

Premake doesn't even support it unfortunately.

Maybe someone can make a JS plugin for Codeblocks, then. Some of the code from the C plugin can probably be borrowed, since the syntax is similar.

I've thought about something like that too.

For example it should also be possible to debug the game's startup.

I'm not sure yet if I will read breakpoint definitions or if it will just be an option that stops the execution as soon as any JS-Code is triggered.

You could then add all the breakpoints you want using the normal ToggleBreakpoint command.

Could the Web-GUI somehow save the breakpoint definitions and load them? I'd prefer if as much as possible would be done with the Web-GUI because that's cleanly separated from the engine and the game.

I guess that could work. If the debugger prints some useful code to stdout when it breaks (like: "DEBUGGER: In break on line x in file y."), IDE plugins can parse that and know when to connect.

Edited January 20, 2013 by zoot

[k776]

Could the Web-GUI somehow save the breakpoint definitions and load them?

Sure, but it would be handy if the debug server provided a URL which listed all current breakpoints... /GetAllBreakPoints , and in simple JSON:

[{file: ..., line: ...}, {file: ..., line: ...}]

[Yves]

I've found a problem that seems quite difficult to solve.

Maybe someone has an idea.

Some explanation of the terms first:

Script: Is a script or a function, but it does not include any child functions. In a normal script-file you usually have multiple functions and therefor multiple scripts.

jsbytecode: It's some bytecode which (as far as I can tell) represents a single line of sourcecode. A jsbytecode belongs to a specific script.

I'm using a call hook set with JS_SetCallHook to get informed about each script that runs.

I check each script if it is associated to a filename for which a breakpoint should be set and if the line number specified by the user is between start- and end-line of the script.

Now the problem is that if there are multiple nested functions, there will be more than one script where this condition applies.

I'm using JS_LineNumberToPC to get the bytecode associated with the line specified by the user.

This function takes a script as argument. I only get the right jsbytecode if this script represents the innermost function.

Otherwise it could be a completely different line because JS_LineNumberToPC doesn't count lines of scripts/functions definied inside the current script.

Some things I've tried or thought about:

If I call JS_LineNumberToPC first and then pass the resulting jsbytecode to JS_PCToLineNumber, I get the line number on which the breakpoint will be exactly set. Unfortunately that doesn't really help to find out if it's the innermost script/function.

If for example the user has specified a line with comments or an empty line, the returned line number can be different too.

I could check all the breakpoints that are already set in each call hook. If they are more nested (start- line is greater than the last one and end-line is smaller than the last one), I could replace the breakpoint.

The problem here is that a wrong breakpoint could be triggered before the right call hook is executed that would correct it.

Here's the API documentation (unfortunately not much more than a list of functions):

https://developer.mozilla.org/en-US/docs/JSDBGAPI_Reference

I hope there's another solution than parsing the JS code.

[Yves]

I used the NewScriptHook to fill such a structure with information:

struct trapLocation
{
  jsbytecode* pBytecode;
  JSScript* pScript;
};
std::map<std::string, std::map<uint, trapLocation> > m_LineToPCMap;

I was able to toggle an example breakpoint correctly that was set on the wrong line with the last version.

I think this should be clean if I delete invalid entries in a DestroyScriptHook, but I'm a bit too tired now, maybe I have forgotten something important. I'll check it in depth tomorrow.

[Yves]

A new version is ready. You can get it from the wiki article together with the updated documentation.

http://trac.wildfire...scriptDebugging

Changelog

• Fixes/Adds Step command to step single lines of code
  
• Adds StepInto command to step into functions
  
• Adds StepOut command to step out of function
  
• Adds setting for stopping threads relatively simultaneously (check the wiki)
  
• Adds the ability to continue all threads with one command by passing threadDebuggerID=0
  
• Adds the ability to break in parent scripts that aren't enclosed by a function
  
• Fixes the output of GetThreadDebuggerStatus to return a JSON array instead of just multiple objects
  
• Fixes possible crash with null-functions
  
• Fixes threading issues with static hook-functions
  
• Fixes setting breakpoints in the current function
  
• Fixes issues with function definitions inside functions which could cause the debugger to break on the wrong line
  
• ... maybe I forgot something
  

I had to use a different approach (described in the post above) to solve certain problems.

I hope I didn't introduce too many new problems. I've tested a few hours on Windows and on Linux, but testing is very cumbersome without a GUI that supports all the features. Kieran's current version of the GUI was already very helpful, and I hope he'll be able to continue the work and add some of the other features.

[zoot]

Step: Steps to the next line of code and will step over functions that are called on the current line.

Hmm, so will this effectively skip the line? Or what will happen if the line is something like: "a = b()"? What value would a be assigned, if any?

Edited January 23, 2013 by zoot

[Yves]

Hmm, so will this effectively skip the line? Or what will happen if the line is something like: "a = b()"? What value would a be assigned, if any?

It will just continue the execution until the next line is reached. It should do exactly the same as normally.

[zoot]

Ah, I see. It just doesn't step through the functions themselves...

[quantumstate]

On irc your were asking about other debuggers and multiple threads. The Visual Studio C++ debugger will run other threads when you step through code. This means if you have a breakpoint which triggers in multiple threads you will switch between threads when stepping through code (you can restrict the breakpoints to only break in a certain thread).

[Yves]

I think we shouldn't use the same behaviour for this debugger because stopping other threads only works if they execute Javascript at the moment.

Stepping to the next line in one thread could continue the execution of a lot of C++ code in another thread.

I decided to allow continuing for all threads but send "step", "step in" and "step out" only to one thread.

[quantumstate]

That sounds sensible to me.

[mmayfield45]

Hi all, I thought this sounded like fun so I had a go

I have only tested this on the latest Firefox / Chrome on Linux.

http://mm45.co.uk/JSDebugger.zip

[k776]

Very nice :-) I don't have time to work on the one I started, so I'm glad someone did.

[zoot]

Hi all, I thought this sounded like fun so I had a go

I have only tested this on the latest Firefox / Chrome on Linux.

http://mm45.co.uk/JSDebugger.zip

Remember to do this (or something equivalent for a permissive license of your choice).

[Yves]

Oh my god this is so awesome!

I hope you don't mind if I post a screenshot:

I've only tested it a few minutes so far but it already looks very polished and the GUI works as I dreamed it could... someday. I never expected this to happen so quickly!

Now I have to fix the remaining problems on the serverside and it will be a very useful tool.

About the licensing:

Do you agree that your code will be committed to the project repository and that it will be licensed under the GPL v2 (or later)? I guess there are also some other options if you prefer.

What 3th party libraries and code did you use and how are they licensed? I see "jquery.easyui" which contains a license file but what about "ace"?

[zoot]

Oh my god this is so awesome!

I hope you don't mind if I post a screenshot:

Wow

[MishFTW]

What 3th party libraries and code did you use and how are they licensed? I see "jquery.easyui" which contains a license file but what about "ace"?

Its a JS code editor that can be embedded. More info here.

Overall, it looks really nice

[historic_bruno]

Impressive!

About the licensing:

Do you agree that your code will be committed to the project repository and that it will be licensed under the GPL v2 (or later)? I guess there are also some other options if you prefer.

What 3th party libraries and code did you use and how are they licensed? I see "jquery.easyui" which contains a license file but what about "ace"?

MIT license would also work, a lot of the scripts in source/tools are licensed under MIT. The Ace scripts appear to be BSD, so I don't see any problems assuming mmayfield45 chooses a free software license for their own code

[mmayfield45]

I'm happy for the license for my code to follow whichever license the 0 A.D. project uses.

[Yves]

I'm happy for the license for my code to follow whichever license the 0 A.D. project uses.

Very good.

I'll wait with committing the code until the server side is ready to commit too and until everything related to licensing is clear.

@all

Checking LICENSE.TXT, everything in source/ should be "GPL version 2 (or later)", but the web-GUI will most likely go into /source/tools which is listed seperately and uses "Various (unspecified)" licenses. I think we should make it clear somehow which licenses the web-GUI uses.

Should we put a seperate license.txt or something into that directory or how should we do that?

[historic_bruno]

@all

Checking LICENSE.TXT, everything in source/ should be "GPL version 2 (or later)", but the web-GUI will most likely go into /source/tools which is listed seperately and uses "Various (unspecified)" licenses. I think we should make it clear somehow which licenses the web-GUI uses.

Should we put a seperate license.txt or something into that directory or how should we do that?

See http://trac.wildfiregames.com/attachment/ticket/1682/license-updates.patch

[Yves]

See http://trac.wildfire...e-updates.patch

Thank you, this makes clear where to put the information about licensing that we have.

@mmayfield45

Just to be sure: Could you please confirm that you agree to release your own work under "GPL version 2 (or later)" like most of the game's code?