Engine enhancements for rebuilding A27

[Seleucids]

I would like to share some engine changes that I made locally to A27-0 release. Feel free to apply my patch and rebuild for better performance / more features. 

Core changes:

1. Turn off Nursery Heuristic (Dunedan's performance patch) which greatly increases fps in fights. 

2. Does not write config strings to mainlog.html but prints out in console instead. This shortens the mainlog by a lot of spam, especially if you use Autociv and / or ModernGUI

3. Prettier console with less transparent background, better cursor, larger fonts etc

4. Prints out every step of the handshake process when joining / hosting a game. This allows you to see why you can't join someone or who is the "unknown player" that failed to join you. The "unknown player" often freezes the host and ruins the game as the host doesn't know which username to kick. 

5. Network warnings if the ping is more than 200ms. The default warning time is scaled with turn time, so sometimes the game might freeze due to bad conn but still you couldn't tell who was the cause. 

sevda.diff

[Deicide4u]

8 minutes ago, Seleucids said:

1. Turn off Nursery Heuristic (Dunedan's performance patch) which greatly increases fps in fights. 

Is this for multiplayer only?

[Atrik]

2 minutes ago, Deicide4u said:

Is this for multiplayer only?

No

[Seleucids]

I am looking at other possible changes to the engine that can improve its performance while staying compatible with the public lobby. Intel has provided a nice investigation and useful advice: https://www.intel.com/content/www/us/en/developer/articles/technical/identifying-the-frame-rate-bottleneck-in-0-ad.html

In light of this article, I think there are 2 potential improvements at hand:

1. Remove garbage collection where possible. 

2. Implement the mathematical calculations in C++ in the engine, then expose them as functions to the Javascript mods. 

 

This morning, I implemented point one by removing the GC steps from simulation2.cpp. Normally, at some point in late game, you stutter when selecting barracks or units, or there is a small sudden freeze caused by GC. Now with the GC removed, the stutters and freezes are even less (ofc, with all previous patches included). There was no OOS with anybody. 

 

Regarding point 2, I will need to conduct some experiments:

1. Dry run a Javascript and a C++ maths script that do identical calculations (for example, integrate arctan(x) dx over some finite domain). Compare the speed of the compilers. Some users claimed that C++ performs 250 times faster than Javascript at integration, but that remains to be tested. In an integration race against Python, C++ performed ~100 times faster. 

2. Implement all functions in globalscripts/math.js into C++ engine functions and just call them whenever needed. 

• I am not sure how slow the interfacing is, so the final products need to be tested. 

Pathfinder is also a big source of lag so maybe i can migrate that to C++ instead. 

 

[wraitii]

21 minutes ago, Seleucids said:

This morning, I implemented point one by removing the GC steps from simulation2.cpp. Normally, at some point in late game, you stutter when selecting barracks or units, or there is a small sudden freeze caused by GC. Now with the GC removed, the stutters and freezes are even less (ofc, with all previous patches included). There was no OOS with anybody. 

This will lead to GCs when the game runs out of memory. I think you're better off implementing my Incremental GC fix  (which I merged since A27).

21 minutes ago, Seleucids said:

Regarding point 2, I will need to conduct some experiments:

1. Dry run a Javascript and a C++ maths script that do identical calculations (for example, integrate arctan(x) dx over some finite domain). Compare the speed of the compilers. Some users claimed that C++ performs 250 times faster than Javascript at integration, but that remains to be tested. In an integration race against Python, C++ performed ~100 times faster. 

2. Implement all functions in globalscripts/math.js into C++ engine functions and just call them whenever needed. 

• I am not sure how slow the interfacing is, so the final products need to be tested. 

Pathfinder is also a big source of lag so maybe i can migrate that to C++ instead. 

Pathfinder is already in C++. I don't think you'll win much by swapping JS math functions to C++ unfortunately, as that's not really the bottleneck.

[alre]

some time ago, there was consensus that major reasons for lag - especially during big fights - were javascript attack/die routines and c++ proximity search. In particular, I remember the latter was a very simple function and a relatively low hanging optimization opportunity.

[Stan`]

Decaying causes some trouble apparently, https://gitea.wildfiregames.com/0ad/0ad/pulls/7616 that's a bit fixed by @nani's autociv since it limits the amount of corpses.

There is also https://code.wildfiregames.com/D1618 which would help when many units are selected.

EDIT1: A27 also supports the --with-lto flag iirc which might slightly improve performance you can also force the cxxflags with march=native and mtune=native probably O3 as well.