Linking the AI scripts (Javascript) To a Python work environment

[ramtzok1]

Hello again everyone!

Link to the last thread : 

This is a follow-up to the last thread I made; Our project supervisor wanted me and my partner to consolidate all the information from the beginning, so I will go again and explain our problems regarding the project we are developing.

Our main idea is to develop machine-learning capabilities within the AI in the game. To do so, we are planning to use a work environment (probably TensorFlow) that works with python. Therefore, we need to find a way to connect our ML code that will be written in python with the AI files that are written in JavaScript. So far, we have been offered to use interfaces that are already found in the game code in C++, write the data that we need to a file that we can use ML on, and calling the JavaScript functions using python server in a way that could make us access the valuable API. We have also been offered TensorFlow.js, but unfortunately it turns out that the mathematical libraries essential to ML are not included with it. At the end, we have chosen the third option, and started working on a server that will fulfill our objectives in creating the link. However¸ we have ran into some problems regarding the JavaScript connection. We tried to use jQuery to create requests with the server, but spiderMonkey does not come with it.

At the end of the day, we really want to use the server as a connection between JS and Python, and we would like to ask for any way for us to do so. Even so, if there are any other good ways that exist and can work with the game, we would be happy to hear.

Thank you to all for your help and patience with us so far! It is really not that obvious.

EDIT: Some more insight on what we have done so far: 
We want to extend the existing AI code, written in javascript, with machine learning code written in Python. To do so, we need to invoke our Python code from within the javascript code. We have been offered to make this connection via http (i.e. AJAX), but the game's spidermonkey engine doesn't seem to support AJAX. 

Edited December 17, 2018 by ramtzok1

[Stan`]

@mimo @(-_-) @elexis @Itms and @wraitii Can you help ?

• Did you have a look at the profiler ?
• Did you try to add a jQuery.js file in your AI files (Pretty sure that won't work but worth trying)
• Did you look at the feedback reporter that sends machine data to our servers ?

[Guest]

Regarding JQuery, well the game does use JS. But in a very different context. So I would recommend the server approach.

In my head, it was something like this.

AI <—> AIManager + server (running in a seperate thread)  <—> Python

Python would send and recieve data using a simple API.

Edited December 17, 2018 by Guest

[elexis]

I'm not sure if the performance requirements for such an endeavor are met. Machine learning simulates the game with semi-random inputs, so I assume you will have to run at the very least a couple of hundreds of matches, right? Try running one match - depending on the number of units on the map  and the number of Petra AI instances, you will get a lot of "lag", i.e. the engine simulating slower than it should. Say a 30 minute 1v1 game on a small map taking 5-10 minutes to simulate, a 60min 4v4 on a normal map with 150 pop 60+ min. Which means you can simulate somewhere between 20 and 50 games per day with a casual desktop computer and your CPU/cooler won't be happy. One can start new matches and replay previous matches without graphics from commandline, check the readme.

If performance isn't a dealbreaker, and one would require http communication, one would have to use libCURL to do http posts to the webserver. If doing so, sending JSON would be the data format that is easiest do digest by spidermonkey.

Assuming a python agent can be used with pyrogenesis/0ad, how should the simulation data be parsed? If it's a very simple learning algorithm, it might alternatively be considerable to implement that in JS than to implement the interface to a different program and teach that program how to parse/send simulation data.

[Imarok]

20 hours ago, elexis said:

Say a 30 minute 1v1 game on a small map taking 5-10 minutes to simulate, a 60min 4v4 on a normal map with 150 pop 60+ min. Which means you can simulate somewhere between 20 and 50 games per day with a casual desktop computer and your CPU/cooler won't be happy

Good analysis. But if you consider, that 0ad mostly runs single-threaded, you can multiply that number by 3

[Stan`]

@ramtzok1 Have you guys given up ?

[ramtzok1]

On 1/17/2019 at 10:36 PM, stanislas69 said:

@ramtzok1 Have you guys given up ?

Hey,

No, we haven't!

I managed to make the linking!

I dug down into the game engine code and figured out what code was related to Engine.{FunctionName} in JavaScript files.

I modified the code by adding 2 functions, AppendToBuffer and WriteToFile so we can send data the Python, I also added useful functions that cannot be accessed from the simulation folder.

Right now we are trying to understand Petra's attack and defense in order to develop the machine learning and we really need an active Petra maintainer so we can ask him about stuff we don't fully understand.

[Stan`]

1 hour ago, ramtzok1 said:

No, we haven't!

Great news

1 hour ago, ramtzok1 said:

I dug down into the game engine code and figured out what code was related to Engine.{FunctionName} in JavaScript files.

 I modified the code by adding 2 functions, AppendToBuffer and WriteToFile so we can send data the Python, I also added useful functions that cannot be accessed from the simulation folder.

Would be nice if you could fork the game on GitHub so we could see your changes and your code

1 hour ago, ramtzok1 said:

Right now we are trying to understand Petra's attack and defense in order to develop the machine learning and we really need an active Petra maintainer so we can ask him about stuff we don't fully understand.

Let me see if I can find a way to help you with that. @mimo

 

@ramtzok1 Our lead and only AI dev left the team to make a Fork of the game. Your best bet is to contact him here:

On 12/11/2018 at 10:09 PM, fatherbushido said:

As said before, the state of that fork is not yet publicly available, but all information will be given on our website when available.

Meanwhile, you can reach us at https://webchat.freenode.net/?channels=forkad

Kind regards,

[sarcoma]

Then the Petra maintainer  already works with your lead in that fork

My mistake, Stan said that, not ramzok1

Edited January 24, 2019 by sarcoma

[Guest]

This AI is now lightyears behind anyway.

[sarcoma]

If I'm not mistaken, many of the files in Petra deal with hard-coded decisions to evaluate feature X of the game state and take an action, but if you add machine learning you have to scratch most of that. It would be nice to have a clean base, richer than baseAI with the queues and serialization and whatnots, so people would have to worry only with stimuli and action to allow easy modding with different machine learning methods.

[gameboy]

@mimo Whete are you?

[Stan`]

3 hours ago, (-_-) said:

This AI is now lightyears behind anyway.

Really ? They made so many fixes during the freeze ?

I guess since they changed the simulation as well we won't be able to use it but it will be nice to check when they release the source once they make a release.

[Guest]

36 minutes ago, stanislas69 said:

Really ? They made so many fixes during the freeze ?

Yes. (And not just for the AI.)

("during the freeze is wrong" I guess. "Since mimo left" would be better as the freeze had ended for a while now. I dont see any AI commits after r21773 back in April 2018)

Don’t take what I say as facts. Who am I to say with so much certainty that there is currently no one here at WFG working on the AI?...but it is the logical conclusion I can arrive at considering all available information.

Edited January 24, 2019 by Guest

[Stan`]

8 minutes ago, (-_-) said:

Don’t take what I say as facts. Who am I to say with so much certainty that there is currently no one here at WFG working on the AI?...but it is the logical conclusion I can arrive at considering all available information.

I can say there is no one. In fact I even stated it above

9 minutes ago, (-_-) said:

Yes. (And not just for the AI.)

Ah nice for them. Can't wait to see their first release Do you also happen to know an ETA ?

 

10 minutes ago, (-_-) said:

I dont see any AI commits after r21773 back in April 2018)

Well save for Mimo who didn't really have to go through the review process even though he made some reviews there is no commit by the two other members in 2018. Leper's work was merged by Itms and I don't remember seeing patches from fatherbushido. only reviews and comments.

[Guest]

End of first quarter 2019 *IIRC*. But don’t consider that statement as fact. I just vaguely recalls something like that.

Edited January 24, 2019 by Guest

[Stan`]

1 hour ago, (-_-) said:

End of first quarter 2019 *IIRC*. But don’t consider that statement as fact. I just vaguely recalls something like that.

Thanks. They haven't really communicated anything so far, so I was wondering

[ramtzok1]

On 1/24/2019 at 12:14 AM, stanislas69 said:

Would be nice if you could fork the game on GitHub so we could see your changes and your code

No problem, I will upload to my fork, though we are using an old version of the game with no plans of updating to the new one.

 

On 1/24/2019 at 12:14 AM, stanislas69 said:

Our lead and only AI dev left the team to make a Fork of the game. Your best bet is to contact him here:

Ok, will do, thank you for that!

On 1/24/2019 at 7:04 AM, sarcoma said:

If I'm not mistaken, many of the files in Petra deal with hard-coded decisions to evaluate feature X of the game state and take an action, but if you add machine learning you have to scratch most of that. It would be nice to have a clean base, richer than baseAI with the queues and serialization and whatnots, so people would have to worry only with stimuli and action to allow easy modding with different machine learning methods. 

1

We won't build the AI from scratch in our case instead the idea is to use some of Petra's parts (our project is very limited on time) for an example gathering. It will basically be bunch of if-else statements reading the output of the Python ML module.

Our machine learning will be focusing mostly on defending properly since we found it to be a lot easier to change and improve than attacking (because of deferent attacking and execution plans).

[Stan`]

15 minutes ago, ramtzok1 said:

Ok, will do, thank you for that!

Always happy to help. Always happy to see interesting initiatives so if you could keep updated that'd be awesome

[ramtzok1]

On 1/25/2019 at 11:56 PM, stanislas69 said:

Always happy to help. Always happy to see interesting initiatives so if you could keep updated that'd be awesome

There you go.

https://github.com/ram1660/0ad

Go to the AI folder to find our AI badbot.

[Stan`]

1 hour ago, ramtzok1 said:

There you go.

https://github.com/ram1660/0ad

Go to the AI folder to find our AI badbot.

Well done, keep it up.

[abhibp1993]

On 12/16/2018 at 3:25 PM, ramtzok1 said:

Our main idea is to develop machine-learning capabilities within the AI in the game. To do so, we are planning to use a work environment (probably TensorFlow) that works with python. Therefore, we need to find a way to connect our ML code that will be written in python with the AI files that are written in JavaScript. So far, we have been offered to use interfaces that are already found in the game code in C++, write the data that we need to a file that we can use ML on, and calling the JavaScript functions using python server in a way that could make us access the valuable API. We have also been offered TensorFlow.js, but unfortunately it turns out that the mathematical libraries essential to ML are not included with it. 

1

1

How about binding some necessary  C++ functions in python using Boost.Python? For instance, tensorFlow has C++ API. Hence, the key functionality can be written in C++, and the user functions for extending/writing AI can be exposed in Python.   

 

(It's only been 2 days since I started looking at the code, so I cannot comment on the feasibility of this, but I didn't see this option mentioned above.)

[irishninja]

I realize I am pretty late to comment on this but if anyone is still interested in this, this revision adds support for using 0 AD like an OpenAI gym environment. There are a couple simple examples using this API at https://github.com/brollb/simple-0ad-example where the agent learns how to kite cavalry archers when fighting infantry!