Kuba386

Hello, I've just tried to create an account after a long break from the game and I got the same error. 'Authentication Failed. Incorrect password or account does not exist' Well, account exists since I just created it and password is certainly correct. Username: 11JK Game version: Alpha 24 Build May 18, 2021 (24937-release) @user1

I've updated my patch, fixed issues with whitespace differences and some other small changes. Now it builds successfully and passes tests.

I've uploaded new patch to D14 that should fix memory leak issues. My code editor (CLion) messed the code by automatically reformatting source files... That's why there are whitespace differences. How can you tell arcanist to ignore them?

I don't have any problems with 0-1 threads... That's actually bad because it makes issue harder to find for me. Is it OS-specific?

Yeah, sure. I'm working on update but recently (because of christmas) I had no time, and development was frozen. Today I will probably start actual work again. Yeah, sure. I hope it will . But first we need to analyze and solve bugs reported here, and maybe do what I was talking about on phabricator. Memory leak is kind of a release blocker.

fabryka

Windows linux or mac? for linux it's ~/.local/share/0ad/replays/0.0.23

I've set up a game with 6 AIs, one hard defensive and 5 very hard aggressive. I went to kitchen to eat something and after 30-40 minutes when I went back game was already using all of my RAM (8GB, game was using like 7.3) and 4 GB of swap. Hard defensive AI has crushed opponents and number of units on map was ~700. When I exited to main menu game was still using all that RAM. This memory is not used by pathfinder components. I don't know what is allocating it, maybe JS.

Yeah, I forgot. If you din't tick the checkbox default number of threads will be used, so by default default number of threads is used Thank you for packaging this patch I will take a look at this too. Actually my memory usage is 3.5 GB for 7 AIs on linux. My memory usage exceeds 4GB (5.5GB at the moment), but I get no crash. I've heard that windows version is 32-bit so cannot allocate more than 4GB.

What if you use one thread or disable threading? (0 threads)

Since you have 4 logical processors I would try using 4-6 threads. Would be nice if you could try using 1,2,4,6 and post results here.

Sad thing to hear. I will look at this issue. Thank you very much for reporting. At least one to see a difference. Using single thread gives big difference from not using threading at all. Usually optimal value is around 2*(number_of_cores-1). At least from my experience.

@(-_-) thank you for testing. I've also done some testing with @nani This pathfinder patch does not make game lagless, but it stops pathfinder from being a bottleneck. Next bottlenecks are network latency for MP games and AI for SP games. Lags you have seen were most probably caused by network latency and not by pathfinder. Yes, that means MP game on local network where all players have multi-core CPUs(no network latency, no AI, high number of threads for all players) is almost lagless.

I've attached it to my post I only have windows and linux on my computer so I probably can't help you much with compilation on mac . If people will have problems compiling this patch I will just host binaries somewhere. I can only do it for windows and linux tough. What does tonight mean for you? Could you give UTC time?

I've merged D14 and D53 patches and it has improved pathfinder performance a lot (see this topic). I've already made a few tests myself, however a real multiplayer game is needed. The purpose of this test is to check: - if there aren't any errors - how does the patch improve performance Common issue of multiplayer games with large number of players are lags, appearing when there are lots of units. In some cases it makes game unplayable. Goal of this patch is to fix these lags or at least decrease them. We need to play such multiplayer game with lots of units and unlimited population. We need at least 6 players. We already have two: me and @Feldfeld Download latest version of code, apply attached patch and compile. (See build instructions for more info) EDIT: attached patch contains memory leak, newer version is available to download on D14. pathfinderupdate.patch

I've just confirmed that MD5 hashes of final state of replays match when using different number of threads and different platforms (tested windows and linux). Tested with 10 different replays. This is already safe enough to play a multiplayer game. Thank you Feldfeld Anyone else would like to play? EDIT: I've created a topic for this

This in not off-topic (well, actually second page of this topic isn't about GA* at all... but I mean it isn't off topic in comparison with other posts on second page). Actually default number of threads that wraitii implemented in his code was 2*(number_of_cores-1). And experiments shown that it's true. Actually I've achieved best results when using 6 and 7 threads (I'm on 4-core CPU). Well, there are also other threads, not only main and pathfinding.

I've just combined D14 with D53 and result is even better. I've attached the patch and a quite heavy replay so you can test it yourself. Try running it with multithreading disabled (set number of threads to 0), and enabled (experiment with number of threads, typically optimal value is 2*(number_of_cores - 1)). You will get out of sync error if you run replay with original version of the game so you have to run it with this patch. Notify me if you have any problems with compiling or running the patch. Also would be nice if anyone would like to play a multiplayer game with me using this patch. pathfinderupdate.patch 2018-12-19_0007.zip

Thanks

Sure.

No, thanks. I can test it on Windows myself. Only reason I haven't already done so is because I haven't compiled it for Windows yet. No problem Sorry for that It has been very pleasant for me so far.

@wraitii "doing it kinda dangerously is very easy and I have a patch for it on Phabricator somewhere". Well, I've updated* your patch so it compiles on latest version. And I must admit I was quite surprised with the result. It really significantly decreased amount of lags for single player game. However when I tried to open replay took with original version, I got out of sync error. (actually not a suprise) However there were no problems when I've played a multiplayer game over LAN. Even when using different amount of worker threads for each game instance. Not only that there were no problems, while there were still some lags in single player game, in multiplayer game lags actually disappeared. 3 players, 1500 units fighting in the center of map, 46 FPS and almost no lags (almost means that there still were some lags, but they were almost unnoticeable). 2 instances of the game running at computer with intel i5-7300 and NVIDIA GTX 1050 8GB RAM, and 1 instance running at computer with intel i5-5th gen and NVIDIA GTS 450 8GB RAM. When I recreated this situation with original version of the game, lags were so huge that game was almost unplayable. I've added option in settings, so players can change amount of worker threads (I actually used it just for debugging purposes, probably not needed for end users). I haven't noticed any problems with determinism in this one game. I've uploaded it back to D14. Oh, and I only tested it on linux, but I think it should work on other platforms too. I'm now thinking about more safe version, since it would be very nice to save such speedup. * 'updated'... hmm... actually changed a few lines....

I have already done that, but it didn't help. The problem is however solved. There was a function declared, but not defined, and incorrectly used extern variable. Next problem is that game deadlocks when map is loaded. I have some ideas how to solve it tough.

No, I haven't seen it before. Will read it soon. But now I have other problem. I've added some code, it passes compilation phase, but there are errors when linking pyrogenesis (undefined references to things I have added to ThreadUtil.h) ==== Building pyrogenesis (release) ==== main.cpp Linking pyrogenesis /usr/bin/ld: ../../../binaries/system/libsimulation2.a(CCmpPathfinder.o):(.data.rel.ro._ZTV14CCmpPathfinder[_ZTV14CCmpPathfinder]+0xc0): undefined reference to `CCmpPathfinder::ComputeShortPath(IObstructionTestFilter const&, CFixed<int, 2147483647, 32, 15, 16, 65536>, CFixed<int, 2147483647, 32, 15, 16, 65536>, CFixed<int, 2147483647, 32, 15, 16, 65536>, CFixed<int, 2147483647, 32, 15, 16, 65536>, PathGoal const&, unsigned short, WaypointPath&)' /usr/bin/ld: ../../../binaries/system/libsimulation2.a(CCmpPathfinder_Worker.o): in function `CCmpPathfinderWorker::ProcessLongRequests(std::vector<AsyncLongPathRequest, std::allocator<AsyncLongPathRequest> > const&)': /home/kuba/0ad-source/0ad/build/workspaces/gcc/../../../source/simulation2/components/CCmpPathfinder_Worker.cpp:676: undefined reference to `ThreadUtil::g_MainThreadMutex' /usr/bin/ld: /home/kuba/0ad-source/0ad/build/workspaces/gcc/../../../source/simulation2/components/CCmpPathfinder_Worker.cpp:678: undefined reference to `ThreadUtil::g_MainThreadMutex' /usr/bin/ld: ../../../binaries/system/libsimulation2.a(CCmpPathfinder_Worker.o): in function `CCmpPathfinderWorker::ProcessShortRequests(std::vector<AsyncShortPathRequest, std::allocator<AsyncShortPathRequest> > const&)': /home/kuba/0ad-source/0ad/build/workspaces/gcc/../../../source/simulation2/components/CCmpPathfinder_Worker.cpp:695: undefined reference to `ThreadUtil::g_MainThreadMutex' /usr/bin/ld: /home/kuba/0ad-source/0ad/build/workspaces/gcc/../../../source/simulation2/components/CCmpPathfinder_Worker.cpp:697: undefined reference to `ThreadUtil::g_MainThreadMutex' /usr/bin/ld: ../../../binaries/system/libsimulation2.a(CCmpPathfinder_Vertex.o): in function `CCmpPathfinderWorker::ComputeShortPath(IObstructionTestFilter const&, CFixed<int, 2147483647, 32, 15, 16, 65536>, CFixed<int, 2147483647, 32, 15, 16, 65536>, CFixed<int, 2147483647, 32, 15, 16, 65536>, CFixed<int, 2147483647, 32, 15, 16, 65536>, PathGoal const&, unsigned short, WaypointPath&)': /home/kuba/0ad-source/0ad/build/workspaces/gcc/../../../source/simulation2/components/CCmpPathfinder_Vertex.cpp:582: undefined reference to `ThreadUtil::g_MainThreadMutex' /usr/bin/ld: ../../../binaries/system/libengine.a(GameSetup.o):/home/kuba/0ad-source/0ad/build/workspaces/gcc/../../../source/ps/GameSetup/GameSetup.cpp:873: more undefined references to `ThreadUtil::g_MainThreadMutex' follow collect2: error: ld returned 1 exit status make[1]: *** [pyrogenesis.make:85: ../../../binaries/system/pyrogenesis] Error 1 make: *** [Makefile:69: pyrogenesis] Error 2 Any idea what can I do with it?

Well, 'passing events one-by-one' is exactly what is 0ad code actually doing. void CComponentManager::PostMessage(entity_id_t ent, const CMessage& msg) { PROFILE2_IFSPIKE("Post Message", 0.0005); PROFILE2_ATTR("%s", msg.GetScriptHandlerName()); // Send the message to components of ent, that subscribed locally to this message std::map<MessageTypeId, std::vector<ComponentTypeId> >::const_iterator it; it = m_LocalMessageSubscriptions.find(msg.GetType()); if (it != m_LocalMessageSubscriptions.end()) { std::vector<ComponentTypeId>::const_iterator ctit = it->second.begin(); for (; ctit != it->second.end(); ++ctit) { // Find the component instances of this type (if any) std::map<ComponentTypeId, std::map<entity_id_t, IComponent*> >::const_iterator emap = m_ComponentsByTypeId.find(*ctit); if (emap == m_ComponentsByTypeId.end()) continue; // Send the message to all of them std::map<entity_id_t, IComponent*>::const_iterator eit = emap->second.find(ent); if (eit != emap->second.end()) eit->second->HandleMessage(msg, false); // look, just calling HandleMessage() function. // Main threads is actually calling HandleMessage() of all receivers. One-by-one. // When you send a message you have to wait for all of your receivers to process it. Correct me if I'm wrong. } } SendGlobalMessage(ent, msg); } How would you notice CCmpUnitMotion class that async pathfinding is done and pass result? Actually all these messages are interface, because HandleMessage() function is interface function. But if you just call interface function from worker thread it will execute in worker thread(If it won't, if it will somehow execute in main thread, then this topic is quite pointless). Interface of pathfinder worker thread in main thread. This way pathfinder thread couldn't tell main thread about finished pathfinging. Main thread would have to check if worker thread already done it. This means it would have to check if pathfinder already finished computing path. This actually seems to be not that bad solution. CCmpPathfinderWorker would have it's local 'result queue' and main thread would lock pathfinder worker thread and check if there is something in that queue. But when would it check that?(where in code?) Main advantage of PostMessage() function is that receiver is getting your message instantly. With that queue it wouldn't be instantly. Then maybe it's not you who are all these questions to. Anyway thanks for your answer. EDIT: Another thing just came to my mind. When we send message about path result to CCmpUnitMotion, then it only does some updates. Whet would happen if we just lock main thread, send message, have it processed in worker thread, do all updates to CCmpUnitMotion in worker thread, and then unlock main thread and continue work? Only thing that I'm not sure of is what would happen if we lock main thread, and can we even do that? Won't it result in a deadlock or something?