WIP: Javascript debugging - Web developer wanted

**MishFTW** · January 6, 2013

do you execute the pyrogenesis_dbg executable?

Ah... that explains it.

**Yves** · January 6, 2013

I think the only thing missing before a basic debugging UI can be completed, is a way to find out where the program is currently halted. For the moment this will be the only new feature I'm going to implement for the debugger (if the UI developers don't find anything else that's required).

It should be relatively easy to write a function that returns filename and linenumber of that location.

The debugging UI will have to poll for that information, I don't think it's possible to push data from the webserver (at least not with our very lightweight and limited mongoose server).

I think with that stage (including the feature mentioned above) the following should be technically possible:

Having a list of available file names
Opening files from that list
Setting breakpoints graphically by clicking on a source-line
Single stepping
Detecting where a breakpoint/step is triggered and displaying the correct file with an icon (or similar) at the correct line
Displaying the callstack when a breakpoint is triggered
Displaying variable content in a hopefully user-friedly way (maybe by searching the variable array for a variable with the name of the currently hovered word and having the array available as raw data if it's something more tricky). This part could be quite tricky if we want to get it perfect (resolving array elements and such things).

After implementing this last feature I'm going to concentrate on the threading aspects and hopefully I can make that stable what we currently have.

Please tell me if you think there are any other essential features for the first version.

**k776** · January 7, 2013

I compiled the patch and hit a breakpoint and stepped through successfully :-) So nice work.

For the UI, for speed, a single data endpoint would be good. Something that returns the current breakpoint (and it's null if the program hasn't hit a breakpoint yet), as well as the stacktrace all in one call. That way, when I make a request to /Step or /Continue, then I can poll /Status (or /GatherState or whatever name really) and it returns all data in one request for quick parsing. Possible?

Edit: Can you also please make sure you keep the first post up to date with callable end points and some basic [clipped] result examples.

I'm hoping to get started on something basic after work today. A form to submit a break point, and poll every few seconds to see if a breakpoint is reached, displaying current breakpoint and the stack trace.

Finally, a way to send commands to the game. e.g. POST a JS command like "unit.state" which evaluates in the context of the current breakpoint, so we can change something, run continue and see if it worked. It would be a nicer way of checking local vars than printing them all.

**MishFTW** · January 7, 2013

Perhaps I'm not doing it right. I do get a success message after patch; however after I compile I don't have debug mode. Can someone post a screenshot of the generated HTML or save a copy of the webpage?

**k776** · January 7, 2013

Ok, I have a very crude web UI here: https://github.com/0...script_debugger

You can set breakpoints, step, and continue. There wasn't a endpoint to see if a breakpoint has been hit, so the step/continue controls always show.

More feedback in the game would be good. When I set a breakpoint, there is no warning the game that it got applied. And it should print before it hits the breakpoint to say "Breakpoint reached", else the game just freezes with no apparent reason (not obvious that it hit a breakpoint).

Now, for bugs: I tried calling the /GetCallstack endpoint and I get a segfault. I tried setting multiple break points and I get a segfault.

Here is the "bt full" for the segfault when hitting a breakpoint and trying to call /GetCallstack: http://pastie.org/5634255

Here is the "bt full" for the segfault when setting multiple breakpoints and hitting the first one: http://pastie.org/5634267 and http://pastie.org/5634270

**historic_bruno** · January 7, 2013

Here's the updated patch:

That fixed it

Now I can set a breakpoint and continue, but there are some problems the first time it's hit. Like k776, I get a crash when attempting to get the call stack. Unfortunately I don't have debug symbols for the Spidermonkey lib so the crash call stack is somewhat useless:


 	ntdll.dll!_ZwRaiseException@12()  + 0x12 bytes	
 	ntdll.dll!_ZwRaiseException@12()  + 0x12 bytes	
 	mozjs185-ps-debug-1.0.dll!50f7160d() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for mozjs185-ps-debug-1.0.dll]	
 	mozjs185-ps-debug-1.0.dll!51062eb9() 	
 	mozjs185-ps-debug-1.0.dll!51004fa2() 	
 	mozjs185-ps-debug-1.0.dll!50f773dd() 	
 	mozjs185-ps-debug-1.0.dll!50f77523() 	
 	pyrogenesis_dbg.exe!CDebuggingServer::GetCallstack(std::basic_stringstream<char,std::char_traits<char>,std::allocator<char> > & response={...})  Line 98 + 0x2b bytes	C++
>	pyrogenesis_dbg.exe!CDebuggingServer::MgDebuggingServerCallback(mg_event event=MG_NEW_REQUEST, mg_connection * conn=0x02cb5c38, const mg_request_info * request_info=0x02cb5c3c)  Line 400 + 0xf bytes	C++
 	pyrogenesis_dbg.exe!MgDebuggingServerCallback_(mg_event event=MG_NEW_REQUEST, mg_connection * conn=0x02cb5c38, const mg_request_info * request_info=0x02cb5c3c)  Line 327	C++
 	pyrogenesis_dbg.exe!call_user(mg_connection * conn=0x02cb5c38, mg_event event=MG_NEW_REQUEST)  Line 526 + 0x37 bytes	C++
 	pyrogenesis_dbg.exe!handle_request(mg_connection * conn=0x02cb5c38)  Line 3364 + 0xb bytes	C++
 	pyrogenesis_dbg.exe!process_new_connection(mg_connection * conn=0x02cb5c38)  Line 3990 + 0x9 bytes	C++
 	pyrogenesis_dbg.exe!worker_thread(mg_context * ctx=0x02ca0968)  Line 4056 + 0x9 bytes	C++
 	msvcr100d.dll!__beginthread()  + 0x233 bytes	
 	msvcr100d.dll!__beginthread()  + 0x1c9 bytes	
 	kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes	
 	ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes	
 	ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes

For some reason the breakpoint is only hit once the first time game setup loads. I played around with it some more and it seems if I continue, exit match setup, and return to hit the breakpoint a second time, then getting the call stack works. I can also continue or step through and the breakpoint is hit many times.

I got this crash once when exiting game setup and returning to hit the breakpoint a second time, it doesn't seem as easily reproducible as the first:


 	ntdll.dll!_ZwRaiseException@12()  + 0x12 bytes	
 	ntdll.dll!_ZwRaiseException@12()  + 0x12 bytes	
 	mozjs185-ps-debug-1.0.dll!512549e4() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for mozjs185-ps-debug-1.0.dll]	
 	msvcr100d.dll!__unlock()  + 0x16 bytes	
 	mozjs185-ps-debug-1.0.dll!513b4aea() 	
 	mozjs185-ps-debug-1.0.dll!513b3bfe() 	
 	mozjs185-ps-debug-1.0.dll!51140927() 	
>	pyrogenesis_dbg.exe!CDebuggingServer::NewScriptHook(JSContext * cx=0x04f85d28, const char * filename=0x0b74eec1, unsigned int lineno=1483, JSScript * script=0x0f4f3800, JSFunction * fun=0x1cb142d0, void * callerdata=0x00000000)  Line 220 + 0x3a bytes	C++
 	pyrogenesis_dbg.exe!NewScriptHook_(JSContext * cx=0x04f85d28, const char * filename=0x0b74eec1, unsigned int lineno=1483, JSScript * script=0x0f4f3800, JSFunction * fun=0x1cb142d0, void * callerdata=0x00c69e28)  Line 167	C++
 	mozjs185-ps-debug-1.0.dll!51261f75() 	
 	mozjs185-ps-debug-1.0.dll!51261b21() 	
 	mozjs185-ps-debug-1.0.dll!511517b0() 	
 	mozjs185-ps-debug-1.0.dll!51152b87() 	
 	mozjs185-ps-debug-1.0.dll!5120f58e() 	
 	mozjs185-ps-debug-1.0.dll!510fc565() 	
 	mozjs185-ps-debug-1.0.dll!510fc6f1() 	
 	mozjs185-ps-debug-1.0.dll!510fc736() 	
 	pyrogenesis_dbg.exe!ScriptingHost::RunScript(const Path & pathname={...}, JSObject * globalObject=0x1cb114b0)  Line 97 + 0x6f bytes	C++
 	pyrogenesis_dbg.exe!CGUI::Xeromyces_ReadScript(XMBElement Element={...}, CXeromyces * pFile=0x0101d670, boost::unordered::unordered_set<Path,boost::hash<Path>,std::equal_to<Path>,std::allocator<Path> > & Paths={...})  Line 1427 + 0x3c bytes	C++
 	pyrogenesis_dbg.exe!CGUI::Xeromyces_ReadRootObjects(XMBElement Element={...}, CXeromyces * pFile=0x0101d670, boost::unordered::unordered_set<Path,boost::hash<Path>,std::equal_to<Path>,std::allocator<Path> > & Paths={...})  Line 1077	C++
 	pyrogenesis_dbg.exe!CGUI::LoadXmlFile(const Path & Filename={...}, boost::unordered::unordered_set<Path,boost::hash<Path>,std::equal_to<Path>,std::allocator<Path> > & Paths={...})  Line 1028	C++
 	pyrogenesis_dbg.exe!CGUIManager::LoadPage(CGUIManager::SGUIPage & page={...})  Line 155	C++
 	pyrogenesis_dbg.exe!CGUIManager::PushPage(const CStrW & pageName={...}, CScriptVal initData={...})  Line 81	C++
 	pyrogenesis_dbg.exe!`anonymous namespace'::PushGuiPage(void * __formal=0x02d5c320, std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > name="page_gamesetup.xml", CScriptVal initData={...})  Line 78 + 0x27 bytes	C++
 	pyrogenesis_dbg.exe!ScriptInterface_NativeWrapper<void>::call<std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >,CScriptVal,void (__cdecl*)(void *,std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >,CScriptVal)>(JSContext * cx=0x04f85d28, unsigned __int64 & __formal=18446462607322775552, void (void *, std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >, CScriptVal)* fptr=0x00586200, std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > a0="page_gamesetup.xml", CScriptVal a1={...})  Line 45 + 0x7d bytes	C++
 	pyrogenesis_dbg.exe!ScriptInterface::call<void,std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >,CScriptVal,&`anonymous namespace'::PushGuiPage>(JSContext * cx=0x04f85d28, unsigned int argc=2, unsigned __int64 * vp=0x05150068)  Line 104 + 0x282 bytes	C++
 	mozjs185-ps-debug-1.0.dll!5119e069() 	
 	mozjs185-ps-debug-1.0.dll!511b29bd() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!51176958() 	
 	mozjs185-ps-debug-1.0.dll!5110a4c2() 	
 	mozjs185-ps-debug-1.0.dll!51106f3d() 	
 	mozjs185-ps-debug-1.0.dll!511c4834() 	
 	mozjs185-ps-debug-1.0.dll!51259276() 	
 	mozjs185-ps-debug-1.0.dll!5125a3a0() 	
 	mozjs185-ps-debug-1.0.dll!5125a64e() 	
 	mozjs185-ps-debug-1.0.dll!5125a66f() 	
 	mozjs185-ps-debug-1.0.dll!510f16f3() 	
 	mozjs185-ps-debug-1.0.dll!510f1693() 	
 	pyrogenesis_dbg.exe!CButton::HandleMessage(SGUIMessage & Message={...})  Line 86	C++
 	pyrogenesis_dbg.exe!IGUIObject::SendEvent(EGUIMessageType type=GUIM_MOUSE_RELEASE_LEFT, const CStr8 & EventName={...})  Line 473 + 0x13 bytes	C++
 	pyrogenesis_dbg.exe!CGUI::HandleEvent(const SDL_Event_ * ev=0x0101f628)  Line 214 + 0x2e bytes	C++
 	pyrogenesis_dbg.exe!CGUIManager::HandleEvent(const SDL_Event_ * ev=0x0101f628)  Line 214 + 0x38 bytes	C++
 	pyrogenesis_dbg.exe!gui_handler(const SDL_Event_ * ev=0x0101f628)  Line 48 + 0xf bytes	C++
 	pyrogenesis_dbg.exe!in_dispatch_event(const SDL_Event_ * ev=0x0101f628)  Line 60 + 0x12 bytes	C++
 	pyrogenesis_dbg.exe!PumpEvents()  Line 192 + 0x9 bytes	C++
 	pyrogenesis_dbg.exe!Frame()  Line 356	C++
 	pyrogenesis_dbg.exe!RunGameOrAtlas(int argc=1, const char * * argv=0x02ca80d8)  Line 507 + 0x5 bytes	C++
 	pyrogenesis_dbg.exe!main(int argc=1, char * * argv=0x02ca80d8)  Line 550 + 0xd bytes	C++
 	pyrogenesis_dbg.exe!wmain(int argc=1, wchar_t * * argv=0x02ca8b88)  Line 380 + 0x14 bytes	C++
 	pyrogenesis_dbg.exe!__tmainCRTStartup()  Line 552 + 0x19 bytes	C
 	pyrogenesis_dbg.exe!wmainCRTStartup()  Line 371	C
 	pyrogenesis_dbg.exe!CallStartupWithinTryBlock()  Line 396 + 0x5 bytes	C++
 	pyrogenesis_dbg.exe!wseh_EntryPoint()  Line 424	C++
 	kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes	
 	ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes	
 	ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes

**Yves** · January 7, 2013

I compiled the patch and hit a breakpoint and stepped through successfully :-) So nice work.
For the UI, for speed, a single data endpoint would be good. [...] Possible?

Yes, that should be possible, but I'll concentrate on the more critical issues (like crashes) first, if that's OK.

Edit: Can you also please make sure you keep the first post up to date with callable end points and some basic [clipped] result examples.

I wanted to put everything into a wiki article yesterday, but had some track problems.

The problems are solved now, so I'll do it together with one of the next patches.

Finally, a way to send commands to the game. e.g. POST a JS command like "unit.state" which evaluates in the context of the current breakpoint, so we can change something, run continue and see if it worked. It would be a nicer way of checking local vars than printing them all.

I'll see what I can do. I thought with the current system the client code could search through the variables to display them in a userfriedly way... but probably it's good to have both options available.

Perhaps I'm not doing it right. I do get a success message after patch; however after I compile I don't have debug mode. Can someone post a screenshot of the generated HTML or save a copy of the webpage?

I think it would be quite difficult to develop the UI without having access to the debugging server.

I'm sure we can figure out where the problem is quite quickly if we do it step by step on IRC.

Ok, I have a very crude web UI here: https://github.com/0...script_debugger

Looking forward to checking it out... but I'll try to get these crashes fixed first.

More feedback in the game would be good. When I set a breakpoint, there is no warning the game that it got applied. And it should print before it hits the breakpoint to say "Breakpoint reached", else the game just freezes with no apparent reason (not obvious that it hit a breakpoint).

The problem is that triggering a breakpoint essentially means pausing execution of Javascript code and also pausing the c++ code that called the Javascript code. If it's the main thread you paused, there are no render()-calls anymore and whatever you pass to LOGERROR will only be rendered when you continue the main thread. Maybe we could call some rendering functions inside the breakpoint handler directly, but that's a lower priority at the moment.

Now, for bugs: I tried calling the /GetCallstack endpoint and I get a segfault. I tried setting multiple break points and I get a segfault.

... but there are some problems the first time it's hit. Like k776, I get a crash when attempting to get the call stack. Unfortunately I don't have debug symbols for the Spidermonkey lib so the crash call stack is somewhat useless
For some reason the breakpoint is only hit once the first time game setup loads. I played around with it some more and it seems if I continue, exit match setup, and return to hit the breakpoint a second time, then getting the call stack works. I can also continue or step through and the breakpoint is hit many times.
I got this crash once when exiting game setup and returning to hit the breakpoint a second time, it doesn't seem as easily reproducible as the first

I think that could be caused by some strictly forbidden things I do concerning multiple threads. :whistling:

I've thought about that this morning on the train and think It's probably not as complicated as I thought.

I'll try to fix that first.

zoot · January 8, 2013

The problem is that triggering a breakpoint essentially means pausing execution of Javascript code and also pausing the c++ code that called the Javascript code. If it's the main thread you paused, there are no render()-calls anymore and whatever you pass to LOGERROR will only be rendered when you continue the main thread. Maybe we could call some rendering functions inside the breakpoint handler directly, but that's a lower priority at the moment.

Any chance something like server-sent events could be implemented? Then the notification could be pushed to the browser (or whatever is accessing the debugger).

**Yves** · January 8, 2013

Any chance something like server-sent events could be implemented? Then the notification could be pushed to the browser (or whatever is accessing the debugger).

I know too little about that at the moment. I assume that the features of the webserver we are using (called mongoose) are quite limited.

Events triggered by the webserver would be handy for breakpoints to avoid polling from the browser, but I'll keep that for later because it's not essential at the moment.

It don't think it would help in the case you quoted. I was talking about ingame information.

zoot · January 8, 2013

Yeah, I understand it is lower priority, but it seems really simple. It's basically just using a special MIME type (text/event-stream).

**Yves** · January 10, 2013

The progress is quite good at the moment, but the change to a thread-safe design is quite a lot of work.

I hope I can get it done this weekend.

**Yves** · January 13, 2013

The bad news first: the weekend is over and the new version is not yet ready for release.

The good news is that the new threadsafe design is basically completed and I think It's a solid base to improve upon.

Many major bugs of the last version are fixed and all features are implemented again.

A new feature allows listing all registered threads and getting information about them. That way it should be possible to display the current position in a file.

The output looks like that:


{ "ThreadDebuggerID" : 1,"ScriptInterfaceName" : "GUI","ThreadInBreak" : true,"BreakFileName" : "gui/pregame/mainmenu.js","BreakLine" : 139 }

I would like to spend a few more hours to get this version more stable and write some documentation before I release it.

zoot · January 14, 2013

Looking forward to it

**MishFTW** · January 14, 2013

I did in fact get errors while using make config=debug. Here is the output.

**historic_bruno** · January 14, 2013

I did in fact get errors while using make config=debug. Here is the output.

Are you sure the patch applied correctly? It doesn't look like it, I suggest reverting your working copy, deleting the added files from the patch (source/scriptinterface/DebuggingServer.*), then reapplying it.

zoot · January 14, 2013

Alternatively, use this branch, which is confirmed to compile. Also remember to update your workspaces (perhaps even cleaning them first).

(Edit: I don't know if it builds on Mac, though.)

Edited January 14, 2013 by zoot

**Yves** · January 14, 2013

I did in fact get errors while using make config=debug. Here is the output.

Are you sure the patch applied correctly? It doesn't look like it, I suggest reverting your working copy, deleting the added files from the patch (source/scriptinterface/DebuggingServer.*), then reapplying it.


../../../source/scriptinterface/DebuggingServer.cpp:745: error: redefinition of ‘void CDebuggingServer::Step()’
../../../source/scriptinterface/DebuggingServer.cpp:269: error: ‘void CDebuggingServer::Step()’ previously defined here

DebuggingServer.cpp doesn't even have 745 lines. It looks like you applied the patch twice and the contents are twice in the file.

**Yves** · January 14, 2013

The new version is ready along with documentation on the Wiki.

http://trac.wildfiregames.com/wiki/JavascriptDebugging

It's still WIP but should work well enough to use it for developing the UI.

**k776** · January 15, 2013

Nicely done. I played around with it for a while and couldn't get it to segfault which is good. I've updated the JS debugger UI to work with the latest implementation, and I've added thread information and stack trace display to it. https://github.com/0ad/javascript_debugger

zoot · January 15, 2013

We should do an Eclipse client next

http://www.eclipse.org/articles/Article-Debugger/how-to.html

**historic_bruno** · January 15, 2013

The new version is ready along with documentation on the Wiki.
http://trac.wildfiregames.com/wiki/JavascriptDebugging
It's still WIP but should work well enough to use it for developing the UI.

Am I understanding correctly that it's possible/required to break into a single script runtime/thread while the others continue running? Don't debuggers typically work by breaking all threads simultaneously? Though perhaps that's the tricky part, knowing how to suspend the various runtimes so nothing bad happens?

**Yves** · January 15, 2013

Am I understanding correctly that it's possible/required to break into a single script runtime/thread while the others continue running?

Yes, that's correct.

Don't debuggers typically work by breaking all threads simultaneously? Though perhaps that's the tricky part, knowing how to suspend the various runtimes so nothing bad happens?

Actually I haven't yet understood how debugging multiple threads exactly works in Visual Studio or GDB.

Maybe I should try to figure that out as a next step.

At the moment I don't know a way to stop other threads instantly, but it should be quite easy to stop them when they call the next function.

**Yves** · January 15, 2013

I've updated the JS debugger UI to work with the latest implementation, and I've added thread information and stack trace display to it. https://github.com/0...script_debugger

Toggling breakpoints works, and I see the thread information, but the step and continue commands don't seem to work and I don't see the callstack.

We should do an Eclipse client next
http://www.eclipse.o...ger/how-to.html

That's probably too much work for me because I don't use eclipse .

I thought I'd make a codeblocks plugin first, but figured out that a webserver and a Web-GUI is probably easier and better.

**k776** · January 15, 2013

Toggling breakpoints works, and I see the thread information, but the step and continue commands don't seem to work and I don't see the callstack.

Fixed.

zoot · January 15, 2013

I thought I'd make a codeblocks plugin first, but figured out that a webserver and a Web-GUI is probably easier and better.

Do you use Codeblocks with JS? Does it do syntax highlighting etc.?

WIP: Javascript debugging - Web developer wanted

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

mmayfield45

Posted Images

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Join the conversation