Jump to content

lag investigation thread


Recommended Posts

Hey all,

mostly out of curiosity I tried to simulate a few different scenarios to find out what parts of the code contribute the most to the lag and should be considered first for future optimizations.

All this was tested with svn rev 25963 and about 500 to 600 units for each test.

TLDR: Most of the things I tested worked better than I thought, but the main problem on my system seems to be unit death (and potentially overkill) and range queries of attacks and especially of missed projectiles. Also: the current AI implementation: See https://code.wildfiregames.com/D3769

_____________________________________________________________________________________________

1st test: long range pathfinding with one type of unit

result for me: not that bad actually very little lag 1/10

----------------------------------------------------

2nd test: long range pathfinding with multiple unit types (graphic influence)

result for me: also good, very little lag 1/10

----------------------------------------------------

test 3: melee combat with 600 units |

result for me: noticeable lag 7/10

----------------------------------------------------

test 4: ranged combat with 600 units

result for me: without having a good measurement I would say the most lag so far 10/10

----------------------------------------------------

test 5: ranged combat with 600 units but emphasis on the projectile calculation

result for me: nearly no lag 1/10

----------------------------------------------------

test 6: ranged combat with 600 units but emphasis on the overkill factor

result for me: huge lag 10/10

----------------------------------------------------

test 7: gathering & short pathfinding

result for me: nearly no lag 1/10

----------------------------------------------------

test 8: BuildingAI

result for me: nothing 0/10

To gather all the information of this thread in one place:

see explanation by @wraitii to why this may be:

18 hours ago, wraitii said:

For what it's worth, I'm pretty sure that the speed difference is because missing arrows query units around them to try and find alternative targets, whereas successful arrows don't. The former is obviously much slower. Thus why it seems like 'overkill' is the problem, but it's in fact just that missing arrows are computationally expensive (and every arrow after the target has been killed misses).

You'd probably notice different results if you used an attack with splash, since splash does range queries anyways (as in, just much more lag all around lol).
See https://code.wildfiregames.com/source/0ad/browse/ps/trunk/binaries/data/mods/public/simulation/components/DelayedDamage.js$71

I think you're also compounding the results by having a massive blob of units nearby, since that probably slows down the C++ range query.

Overall, there is little easy gain to make here unless we got much smarter about immediate range queries.

 

Edited by maroder
  • Like 4
Link to comment
Share on other sites

I updated the post

What I would conclude from all that (no guarantee on correctness) is that the main reason for lag (at least for me) is the overkill factor.

So the engine seems to dislike it if the health of a unit is so much reduced that it dies, but there are still other attacks incoming, which are now targeting an entity that is already dead. And this happens much more often when ranged units attack vs when melee units attack.

All other parts, be it graphics, short and long range pathfinder ect, seem not too bad from these tests.

Edited by maroder
  • Like 1
  • Thanks 1
Link to comment
Share on other sites

Yeah, rewriting all that seems like a huge project, so I get that part of the argument, but personally I don't think the modability factor should play a role here (and I like to make mods).

The question is who wants to mod a game that is not functioning correctly? not doing optimization only because someday someone might want to change it seems to me like a bad tradeoff. If there are ways to optimize the javascript part that should of course be preferred, but if the only way to fix that would be to reduce some of the modability, then that is the sacrifice that has to be done (imo).

Edited by maroder
  • Like 2
  • Thanks 1
Link to comment
Share on other sites

And the other question is: would it really need a complete rewrite? From what I saw I would say that that the problematic part is the unit death / attacking combination and not all of the attacking routine. But I also would really like to hear what conclusions other people draw from the videos :)

Link to comment
Share on other sites

Aside from re-writing code, one could still implement changes to reduce overkill (and therefore, lag) such as attack-ground. It would still be the player's decision whether or not to use attack-ground, but it would also be more effective in large battles due to less overkill.

However, this does looks laggy on its own right, maybe it just appears bad since its shown in 0.5x speed.

Attack-ground:

 

Link to comment
Share on other sites

35 minutes ago, maroder said:

Yeah, rewriting all that seems like a huge project, so I get that part of the argument, but personally I don't think the modability factor should play a role here (and I like to make mods).

The question is who wants to mod a game that is not functioning correctly? not doing optimization only because someday someone might want to change it seems to me like a bad tradeoff. If there are ways to optimize the javascript part that should of course be preferred, but if the only way to fix that would be to reduce some of the modability, then that is the sacrifice that has to be done (imo).

The problem is that attack.js is linked to so many components that you would have to freeze them too (eg, health resistance unitai and possibly others) rendering the game a lot less moddable not just a bit. I'm hoping there is a better alternative but mayber there isn't.

 

What's slow when Units are dying are the onownership messages and another one. You can see it pretty clearly using profiler2. Making the messages faster might help a lot. Could even be threaded maybe.

 

  • Like 1
Link to comment
Share on other sites

  • 1 month later...
On 13/10/2021 at 10:03 AM, maroder said:

Yeah, rewriting all that seems like a huge project, so I get that part of the argument, but personally I don't think the modability factor should play a role here (and I like to make mods).

The question is who wants to mod a game that is not functioning correctly? not doing optimization only because someday someone might want to change it seems to me like a bad tradeoff. If there are ways to optimize the javascript part that should of course be preferred, but if the only way to fix that would be to reduce some of the modability, then that is the sacrifice that has to be done (imo).

I agree

Link to comment
Share on other sites

On 13/10/2021 at 5:30 PM, maroder said:

So the engine seems to dislike it if the health of a unit is so much reduced that it dies, but there are still other attacks incoming, which are now targeting an entity that is already dead. And this happens much more often when ranged units attack vs when melee units attack.

For what it's worth, I'm pretty sure that the speed difference is because missing arrows query units around them to try and find alternative targets, whereas successful arrows don't. The former is obviously much slower. Thus why it seems like 'overkill' is the problem, but it's in fact just that missing arrows are computationally expensive (and every arrow after the target has been killed misses).

You'd probably notice different results if you used an attack with splash, since splash does range queries anyways (as in, just much more lag all around lol).
See https://code.wildfiregames.com/source/0ad/browse/ps/trunk/binaries/data/mods/public/simulation/components/DelayedDamage.js$71

I think you're also compounding the results by having a massive blob of units nearby, since that probably slows down the C++ range query.

Overall, there is little easy gain to make here unless we got much smarter about immediate range queries.

  • Thanks 2
Link to comment
Share on other sites

Thanks for the insights! That explains a lot.

38 minutes ago, wraitii said:

Overall, there is little easy gain to make here unless we got much smarter about immediate range queries.

Sad. Cause as the videos show, at least on my system it feels like this is the main reason for the lag.

41 minutes ago, wraitii said:

I think you're also compounding the results by having a massive blob of units nearby

:P you mean like in every fight situation in every game? I would say that's gold standard.

Link to comment
Share on other sites

Do individuals specifically get hit and die by arrows in total war? Never looked that closely to it. Otherwise it's probably just calculating the entire battalion as 1 hitbox and lets random soldiers die when the health drops of the battalion entity.

  • Like 1
Link to comment
Share on other sites

17 hours ago, wraitii said:

Overall, there is little easy gain to make here unless we got much smarter about immediate range queries

After having thought about it a bit more: You're insights explain why ranged attack causes more lag then melee and why this is hard to improve, but even the melee attacking (and especially the unit death part) causes much lag. Do you think this could be a better area for improvements?

Link to comment
Share on other sites

12 hours ago, Grapjas said:

Do individuals specifically get hit and die by arrows in total war? Never looked that closely to it. Otherwise it's probably just calculating the entire battalion as 1 hitbox and lets random soldiers die when the health drops of the battalion entity.

Each soldier has his own hitbox in TW. Deaths occur per soldier with the soldier's stats (armor and life points) vs. the enemy's attack (attack value plus a dice roll). The battalion as a group only has high level things like morale, coherence, direction (affects armor), range, and stamina. 

Link to comment
Share on other sites

14 hours ago, wowgetoffyourcellphone said:

Question. So, in a Total War battle, you can also have 100s of arrows in a volley, but have less lag. What are they doing differently? 

I would assume they are much smarter about collision detection :P 

They probably have much more persistent data structures for collision detection in such a manner, because otherwise the game just wouldn't work. For example, they can probably first consider all formations (of which there are very few) to decide a new target to pick, where we must consider all individual units.

Or another example: for the most part, they don't have to consider new units joining the battle (I think this may be somewhat different in TW WarHammer but it remains limited). That probably enables them to use some clever hacks that we can't so easily use.

2 hours ago, maroder said:

After having thought about it a bit more: You're insights explain why ranged attack causes more lag then melee and why this is hard to improve, but even the melee attacking (and especially the unit death part) causes much lag. Do you think this could be a better area for improvements?

There is a lot of stuff happening in combat. If the target dies, UnitAI needs to choose a new target, which is by itself rather slow -> range query + filtering for preferred targets, sorting, etc. That's not even mentioning the fact that yes, dying by itself could be a bit slow.
This is indeed an area of possible improvements, and some work has been done (e.g. rP25102)

To be honest, the arrows range query could perhaps be improved by having some in-turn caching or some other logic to batch missile hits (so that multiple missiles landing in the same area don't do multiple range queries or something), but that doesn't sound entirely trivial to do without breaking in some edge-cases. It's always a question of code complexity, accuracy, edge-cases with these kind of optimisations.

---

Edit: Also, while such "in a vacuum" performance profiling is useful when you're working to optimise a specific function, be careful about generalising. In an actual MP replay, you'll most likely see a much more nuanced picture.

  • Like 3
Link to comment
Share on other sites

On 12/01/2022 at 5:09 AM, maroder said:

might also be worth asking if @badosu has any knowledge or insight how the range queries/ attacking is handled over in BAR (and if it is also performance critical there)

Am I correct in thinking that 0AD performs these calculations in JS?

I'm looking at how BAR handles it and asking engine people. But I think in BAR its C engine code, with hooks/callins to allow for customization.

  • Thanks 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...