Jump to content

Recommended Posts

In yesterdays IRC I mentioned about using jasmine tests for AI development and the topic was considered as maybe of interest to others. So I hereby provide a trimmed-down demonstrator on how a test suite can be set up.

The attached zip archive contains a copy of the API3 AI high-level interface with two test cases (one of them discovered the problem discussed here), plus the necessary infrastructure (jasmine 3.6.0 release and driver html page). To execute the test cases, just extract the zip content to some directory and load the common-api/jasmine-runner.html file into a scriptable webbrowser. Said html file is commented to show how the jasmine framework, the API3 under test and the test files interact.

Items to consider:

  • For the demo, I used the official jasmine standalone release which contains a version number in its path. In "production use" I rename the directories to get rid of the version number so no changes are needed when upgrading jasmine.
  • It is also possible to execute the unit tests via the SpiderMonkey js shell by loading all of the scripts and running the jasmine bootstrap code.
  • I also managed to connect the jasmine test suite to the JSCover code-coverage measurement tool, but this requires a more complicated setup, i.e. a list of all source and test case js files plus a platform-dependent batch job and a hacked-up JSCover driver js script. I can provide a demonstration if there is interest.

Any comments welcome.

jasmine-demo.zip

  • Like 2
Link to post
Share on other sites

Hi Teiresias, this is very interesting! Our use of C++ unit testing to run JS tests is really subpar. If we can use jasmine through spidermonkey it would allow us to have a better tool for testing JS code.

Would it be possible for you to demonstrate how to run, for instance, simulation component tests (in binaries/data/mods/public/simulation/components/tests/), using jasmine? Those tests rely on "Engine" methods which are defined in the 0ad engine, so I'd like to see how that works out.

Coverage information would be nice to have. I wouldn't say it is mandatory (we have engine coverage and we don't really look at it) but if we have it it's definitely a plus, and it would push us into analyzing coverage information more often.

Thanks a lot for your proposal!

Link to post
Share on other sites

Itms, thanks for the "padding on the shoulder". I admit I might be a bit extreme regarding testing, since I partially do this for a living.

Regarding your question

On 9/19/2020 at 10:30 AM, Itms said:

Would it be possible for you to demonstrate how to run, for instance, simulation component tests (in binaries/data/mods/public/simulation/components/tests/), using jasmine? Those tests rely on "Engine" methods which are defined in the 0ad engine, so I'd like to see how that works out.

I'm afraid, no. When writing jasmine tests I am in a JS-only world. Since a unit test is about testing the smallest isolatable parts of a software system - usually a single function or a class - this is not a problem for me. But if you intent to include the Pyrogenesis engine activities together with the JS code - that's actually an integration test, and usually harder to acchieve than unit testing. I don't know any off-the-shelf solution, in particular if multiple runtime environments are involved (native code vs. SpiderMonkey JS environment).

In my AI experiments I faced similar problems with the common AI and currently use two approaches:

  • Include the common AI code in the JS space where the jasmine tests execute. Since they are both JS this is possible. However, I still try to avoid this as much as possible since it introduces an external dependency.
  • Lift up to system test level, i.e. run the whole Pyrogenesis executable in autoplay mode with special script files to generate a scenario map tailored for testing and evaluating the AI behavior. This is a very cumbersome method and I try and avoid it whereever possible. No JSCover analysis done with this method.
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...