Documentation for the engine and code

[AInur]

Hello, I am enjoying this game very much and as a C++ developer, I would like learn more about how the game is programmed, especially the engine. Hopefully, I can contribute to your development efforts!

Unfortunately, most documentation websites related to the engine are down with 404 error. It would be very helpful if somebody could point me to the documentations of the Pyrogenesis engine or some design document which explains how the C++ is structured. Additionally, I would like to understand how the Javascript in the mods is integrated into the main engine and how they are called. 

So far, I have found the list of engine commands and Wiki on modifying the game, which are indeed helpful. It would be even better if some documentation can point me to exactly where in the source code these methods are defined and the logic behind it. I would also like to use this topic as a wiki for programmers where they can post documentations.

 

Engine functions: https://gitea.wildfiregames.com/0ad/0ad/wiki/EngineFunctions

Overall Wiki: https://gitea.wildfiregames.com/0ad/0ad/wiki

 

 

 

[Stan`]

Hey, and welcome.

  

51 minutes ago, AInur said:

Unfortunately, most documentation websites related to the engine are down with 404 error.

Can you give us the links? https://docs.wildfiregames.com/  should work.

You can also ask your questions directly on IRC (Please don't wait for an answer in case no one answers directly, most of the devs read the logs, and might answer asynchronously there or here)

[AInur]

20 minutes ago, Stan` said:

https://docs.wildfiregames.com/  should work

Thank you for the reply, this link contains everything that I need for now.

 

To be specific, I would like to understand the game state object first. How is it synchronised between players and how can we prevent/catch OOS errors? Where can I find the list of attributes associated with this object? For example, how can I query the current turn, the current number of units in the map, the positions of each entity?

 

Secondly, I have opinionated a list of possible improvements that I could potentially code up, but would need some assistance from you:

1. The game and its freezes for a few seconds when there is a large battle. This could be caused by serial processing of too many objects at once. Perhaps we can offload some of the computation into async threads or parallel threads? Parallel processing would be a great improvement.

2. The 'AI' algorithm needs much improvement. I have seen past forum posts discussing solutions to this, but they are limited to improving the economy of the AI instead of its strategy. Ideally, the AI could interact with the player a bit more and respond to the strategies of the player. For example, if the AI sees the player making a lot of cavalry, then it should respond by making spearman. Currently, it is unable to adapt its strategy nor scout.

3. Occasionally, the game crashes when we are changing mods. There should be a more robust way to handle exceptions; if the mod is corrupt, then the engine should forbid this operation temporarily instead of crashing.

4. For Windows clients, due to the 32bit 4GB RAM usage limit, we can create a temporary VRAM and page data to that temporary file instead of crashing.

5. I would like an API interface where I can read out game data and inject commands into the game session without using the default game GUI. This would facilitate the training of better AI models powered by TensorFlow neural networks, as well as help with debugging.

[Stan`]

6 minutes ago, AInur said:

1. The game and its freezes for a few seconds when there is a large battle. This could be caused by serial processing of too many objects at once. Perhaps we can offload some of the computation into async threads or parallel threads? Parallel processing would be a great improvement.

There is already a fair bit of threading going on, there is still work to be done in that area though. The problem is the computations for the JS part can only be done in the same thread for now.  https://gitea.wildfiregames.com/0ad/0ad/issues/5874

8 minutes ago, AInur said:

 

2. The 'AI' algorithm needs much improvement. I have seen past forum posts discussing solutions to this, but they are limited to improving the economy of the AI instead of its strategy. Ideally, the AI could interact with the player a bit more and respond to the strategies of the player. For example, if the AI sees the player making a lot of cavalry, then it should respond by making spearman. Currently, it is unable to adapt its strategy nor scout.

Currently the AI does not have to as it is omniscient, and plays with the map revealed.

8 minutes ago, AInur said:

. Occasionally, the game crashes when we are changing mods. There should be a more robust way to handle exceptions; if the mod is corrupt, then the engine should forbid this operation temporarily instead of crashing.

No objections there.

9 minutes ago, AInur said:

r Windows clients, due to the 32bit 4GB RAM usage limit, we can create a temporary VRAM and page data to that temporary file instead of crashing.

There is work being done on a 64bit build here: https://gitea.wildfiregames.com/0ad/0ad/pulls/7121 The binaries are not uploaded yet but hopefully will be soonish for testing.
 

10 minutes ago, AInur said:

I would like an API interface where I can read out game data and inject commands into the game session without using the default game GUI. This would facilitate the training of better AI models powered by TensorFlow neural networks, as well as help with debugging.

Have you seen the rl interface?
https://gitea.wildfiregames.com/0ad/0ad/src/branch/main/source/rlinterface

And the tooling required to use it (it's a http interface)

https://gitea.wildfiregames.com/0ad/0ad/src/branch/main/source/tools/rlclient/python

https://gitea.wildfiregames.com/0ad/0ad/wiki/GettingStartedReinforcementLearning
 

11 minutes ago, AInur said:

o be specific, I would like to understand the game state object first. How is it synchronised between players and how can we prevent/catch OOS errors? Where can I find the list of attributes associated with this object? For example, how can I query the current turn, the current number of units in the map, the positions of each entity?

Each client compute the state and send hashes to other players, which require a very low bandwidth. Turnmanager.cpp should have the turn number  See also
https://gitea.wildfiregames.com/0ad/0ad/wiki/SimulationRequirements and https://gitea.wildfiregames.com/0ad/0ad/wiki/WfgAcademiaInto

 

[AInur]

Hi @Stan`, thank you very much for the replies above, they were very helpful. I have some additional questions:

• How can I log information via javascript files in the public mod?
• How can I print out a string into the command line? For example, I want to print out the number of my infantry units every 10 seconds.
• How can I write a string into a file at a specified location?
• How can I append to a json file without overwriting it?

I have found the engine function of WriteToJson, which is helpful but it overwrites the output file on each call. I would like it to append instead of overwrite.

[AInur]

23 minutes ago, AInur said:

How can I print out a string into the command line?

I have tried adding Engine.PostMessage("text") to one of the .js files.

This failed with an error of PostMessage method is not defined.

Then I tried to add console.log into the js files, and peculiarly, javascript cannot find the command console.log()

 

[Stan`]

38 minutes ago, AInur said:

How can I log information via javascript files in the public mod?

Use

warn("")  // Prints Warning : your message in yellow
error("") // Prints Error : your message in red
print("") // Prints the message. It's not displayed by the interface. Only available in the mainlog and the terminal (if you're on macOS or Linux)

To show objects use

warn/print/error(uneval(object)) // NOTE big objects will crash the game.

https://gitea.wildfiregames.com/0ad/0ad/wiki/Logging Might be outdated, haven't checked.
 

38 minutes ago, AInur said:

• How can I write a string into a file at a specified location?
• How can I append to a json file without overwriting it?

• I have found the engine function of WriteToJson, which is helpful but it overwrites the output file on each call. I would like it to append instead of overwrite.

We don't really expose many functions for writing files, as it will start doing weird things with packaged mods.

See https://gitea.wildfiregames.com/0ad/0ad/src/branch/main/source/ps/scripting/JSInterface_VFS.cpp for exposed functions or
See https://gitea.wildfiregames.com/0ad/0ad/src/tag/a26/source/ps/scripting/JSInterface_VFS.cpp#L268  for A26