Message System / Simulation2

[phosit]

The simulation requires much time in the late game. While this is expected we still should ompimize it.
For reference: https://trac.wildfiregames.com/wiki/SimulationArchitecture

There are many ideas:

• @Freagarach tried to send messages only to some entities. https://code.wildfiregames.com/D2704 [[Abandoned]]
• @Mercury and @wraitii think about multithreading. i.a: https://wildfiregames.com/forum/topic/90006-mutex/
• Mercury also made an allocator. https://code.wildfiregames.com/D4718
• There is a new datastructure from wraitii. https://code.wildfiregames.com/D1739 

Meta:

Isn't that (many improvement ideas) a sign of a deeper problem. Should we change the message system or move away from it altogether? Do you have other idears?
(Did we change the system already, why is there a 2 after Simulation?)

Edited November 17, 2022 by phosit

[phosit]

Oh, yes I have an idea. Responding to my own call ... and laughing about my own joke

Currently the components are (among others) accessed throu

std::map<ComponentTypeId, std::map<entity_id_t, IComponent*> > m_ComponentsByTypeId;

I would make a container for each ComponentType

std::map<entity_id_t, CCmpAIInterfaceScripted> AIScripted;
std::map<entity_id_t, CCmpAIManager> AIManager;
// ... all components known to C++

// just as bevore
std::map<ComponentTypeId, std::map<entity_id_t, CCmpScript>> ScriptedComponents;

There are no interface-classes; no virtual functions; the components could be stored inside the container. In short much less redirections and propably more inlining.

[hyperion]

On 12/11/2022 at 8:03 PM, phosit said:

@Freagarach tried to send messages only to some entities. https://code.wildfiregames.com/D2704 [[Abandoned]]

Many messages probably originate in javascript and are consumed in javascript and so wouldn't have to leave javascript space at all. Maybe moving the dispatcher to javascript might be an option if expensive components can't be moved to c++. Just a guess but the cost of communication between c++ and javascript might outweigh the extra cost of using javascript.

Maybe something like nodjs event emitter and libuv could be used as inspiration.

 

On 12/11/2022 at 8:03 PM, phosit said:

@Mercury and @wraitii think about multithreading. https://wildfiregames.com/forum/topic/90006-mutex/

This thread is a bit of a headache, first it looks like "let's sprinkle mutex in case we need it some day".

Then it shifts to headless, which at it's easiest could be a null render backend and a driver (possibly some tcp socket protocol) and has nothing to do with threading per se.

Then I see some hint of a server client setup where you don't send commands but state transitions or error corrections, which would possibly allow for much more aggressive parallelization.

I'm lost at that point.

 

[phosit]

On 16/11/2022 at 1:13 PM, hyperion said:

Just a guess but the cost of communication between c++ and javascript might outweigh the extra cost of using javascript.

Regarding the function call cost I can only guess too.

The messages(arguments) are typically smal / easy to convert. That should not be a problem

On 16/11/2022 at 1:13 PM, hyperion said:

Maybe something like nodjs event emitter and libuv could be used as inspiration.

Thouse use the Reactor-/ PubSub-/ Command-pattern (which is it?)for async stuff. We nead it for consomization. More indication that we use the wrong pattern.

[hyperion]

14 hours ago, phosit said:

Reactor

This one for nodjs

15 hours ago, phosit said:

More indication that we use the wrong pattern

Well, some 15 years ago javascript was slow as hell ... So no surprise we have different choices this days. As the simulation under current constraints needs to be single threaded javascript is an option. Again, I can only make an educated guess and you will have to consult specialist in this field but I think it might be well worth it.

[fabio]

Some other ideas are here (a bit old page, but being a wiki you can update it): https://trac.wildfiregames.com/wiki/GamePerformance

[alre]

I believe range queries used by fighitng routines and auras could be optimized wildly by implementing a matrix of moving units such as that used by the pushing component - in fact, no reason to use another one, that one would do just right. I don't know c++ enough to try this  change myself though, and it sounds absurd to remake it in javascript (although that would probably be an improvement already).

[phosit]

2 hours ago, fabio said:

Some other ideas are here (a bit old page, but being a wiki you can update it): https://trac.wildfiregames.com/wiki/GamePerformance

I saw that page already and ignored it thaught it is outdated . I'll see what i can update.

1 hour ago, alre said:

I believe range queries used by fighitng routines and auras could be optimized wildly by implementing a matrix of moving units such as that used by the pushing component - in fact, no reason to use another one, that one would do just right.

I don't know the units pushing code. It would be great if we can generalize it to optimize range queries.

[wraitii]

16 hours ago, alre said:

I believe range queries used by fighitng routines and auras could be optimized wildly by implementing a matrix of moving units such as that used by the pushing component - in fact, no reason to use another one, that one would do just right. I don't know c++ enough to try this  change myself though, and it sounds absurd to remake it in javascript (although that would probably be an improvement already).

I'm fairly certain the range manager already does this in a way. The problem is that you do want to send _only_ the right units to JS, because doing further filtering in JS is super slow.

[alre]

40 minutes ago, wraitii said:

I'm fairly certain the range manager already does this in a way. The problem is that you do want to send _only_ the right units to JS, because doing further filtering in JS is super slow.

when I watched into it (a couple of months ago), the c++ function that is always called from javascript just run a query over all units.

[hyperion]

3 hours ago, alre said:

when I watched into it (a couple of months ago), the c++ function that is always called from javascript just run a query over all units.

The in range depends also on height, changing that would make it cheaper and allow to use let's say l1-norm to filter possible targets first

[alre]

2 hours ago, hyperion said:

The in range depends also on height, changing that would make it cheaper and allow to use let's say l1-norm to filter possible targets first

true that. btw, non-parabolic range query do a L1 selection first, and then compute euclidean distances.

Anyway, one could also switch off height sensitive queries based on maps, bc most maps are flat, but that's not a great solution if you want the game to play well on 3d maps too.

[phosit]

On 16/11/2022 at 1:13 PM, hyperion said:

Maybe moving the dispatcher to javascript might be an option if expensive components can't be moved to c++. Just a guess but the cost of communication between c++ and javascript might outweigh the extra cost of using javascript.

How about sending a message to a entity instead of a component and the JS-entity dispatches to its component? then there would only be one c++ -> JS comunication per entity.

[hyperion]

4 hours ago, phosit said:

How about sending a message to a entity instead of a component and the JS-entity dispatches to its component? then there would only be one c++ -> JS comunication per entity.

Do you have an example of a component messaging another within the same entity? Also do the components know which other components belong to the same entity without querying the engine?

[phosit]

2 hours ago, hyperion said:

Do you have an example of a component messaging another within the same entity? Also do the components know which other components belong to the same entity without querying the engine?

That's not what I ment.

The messages are still sent via C++ to some JS-entity class that class then sends the message to its component.

 

[phosit]

What do you (all) think about adding a layer between the simulation and the renderer. That layer would use the data from the simulation to calculates the data which are not syncronized (corpses, interpolation, sound...). The renderer and the audio-system den does use thous data.

In other words: Splitting the simulation in a global and a local part.

That will be simpler to implement when there is a clear boundrary between simulation and renderer.