Kuba386 Posted December 21, 2018 Report Share Posted December 21, 2018 (edited) I've merged D14 and D53 patches and it has improved pathfinder performance a lot (see this topic). I've already made a few tests myself, however a real multiplayer game is needed. The purpose of this test is to check: - if there aren't any errors - how does the patch improve performance Common issue of multiplayer games with large number of players are lags, appearing when there are lots of units. In some cases it makes game unplayable. Goal of this patch is to fix these lags or at least decrease them. We need to play such multiplayer game with lots of units and unlimited population. We need at least 6 players. We already have two: me and @Feldfeld Download latest version of code, apply attached patch and compile. (See build instructions for more info) EDIT: attached patch contains memory leak, newer version is available to download on D14. pathfinderupdate.patch Edited December 29, 2018 by Kuba386 4 Quote Link to comment Share on other sites More sharing options...
Servo Posted December 21, 2018 Report Share Posted December 21, 2018 (edited) How we get the patch? How to incorporate to our game? I’m using the Mac OS rebundle these days. Tonight im available if it works on me. Humnn that’s hard to build on Mac for me (svn). Edited December 21, 2018 by Servo 1 Quote Link to comment Share on other sites More sharing options...
Kuba386 Posted December 21, 2018 Author Report Share Posted December 21, 2018 (edited) 18 minutes ago, Servo said: How we get the patch? I've attached it to my post 18 minutes ago, Servo said: How to incorporate to our game? 23 minutes ago, Kuba386 said: Download latest version of code, apply attached patch and compile. (See build instructions for more info) 18 minutes ago, Servo said: Humnn that’s hard to build on Mac for me (svn). I only have windows and linux on my computer so I probably can't help you much with compilation on mac . If people will have problems compiling this patch I will just host binaries somewhere. I can only do it for windows and linux tough. 18 minutes ago, Servo said: Tonight im available if it works on me. What does tonight mean for you? Could you give UTC time? Edited December 21, 2018 by Kuba386 Quote Link to comment Share on other sites More sharing options...
Lion.Kanzen Posted December 22, 2018 Report Share Posted December 22, 2018 14 hours ago, Kuba386 said: The purpose of this test is to check: - if there aren't any errors - how does the patch improve performance You are posting in wrong forum. I guess. Quote This forum is primarily for discussion for gameplay balance in format of: Topic: Subject to be tested (e.g. Early game Cavalry Skirmisher rush) and description. Replies to Topic: Replays of games (1v1s are best) that involve the subject of testing. Analyze the game (mistakes, well-executed aspects, etc.) and write a summary. Commentary of balance is encouraged, but must have a replay reference. Also, please include the revision of the game that was tested! Proposed balance changes will be written by team members for visibility and comments in this forums as well. When a new balance-changing change occurs, topics of outdated revisions will gain a new post with the changes. Do note that in order to keep this efficient, topics created may be edited/deleted/moved/merged to make the format of this subforum concise and consistent. Quote Link to comment Share on other sites More sharing options...
Guest Posted December 22, 2018 Report Share Posted December 22, 2018 I have tested this on multiplayer. With both having applied the patch and on different OSes. Linux and windows. The game was in sync, so I suppose there arent issues with determinism. At least, the kind that’s easy to spot and fix. However, there were freezes. Which I haven’t investigated yet. May or may not be related. There were around 200 units moving. Could be the thing elexis posted on phab. Quote Link to comment Share on other sites More sharing options...
Stan` Posted December 22, 2018 Report Share Posted December 22, 2018 @Servo I could distribute a modified version of the game with that patch applied but you'd have to install it manually. would that be okay ? (Could make a new bundle but it takes around 6h so meh) 1 Quote Link to comment Share on other sites More sharing options...
Kuba386 Posted December 22, 2018 Author Report Share Posted December 22, 2018 @(-_-) thank you for testing. I've also done some testing with @nani This pathfinder patch does not make game lagless, but it stops pathfinder from being a bottleneck. Next bottlenecks are network latency for MP games and AI for SP games. Lags you have seen were most probably caused by network latency and not by pathfinder. Yes, that means MP game on local network where all players have multi-core CPUs(no network latency, no AI, high number of threads for all players) is almost lagless. Quote Link to comment Share on other sites More sharing options...
vladislavbelov Posted December 22, 2018 Report Share Posted December 22, 2018 6 minutes ago, Kuba386 said: This pathfinder patch does not make game lagless, but it stops pathfinder from being a bottleneck. Did you run the profiler in these matches? It may tell us, what time was spending for. Quote Link to comment Share on other sites More sharing options...
elexis Posted December 22, 2018 Report Share Posted December 22, 2018 1 hour ago, Kuba386 said: Lags you have seen were most probably caused by network latency and not by pathfinder. Network latency should not be an issue, the turn lengths are 500ms in multiplayer and you can test by simulating lag that if the meanRTT <= turnLength, there is no network lag perceivable. On "Simulation interpolation lag": However there is a different aspect that can make the simulation consume 550-600ms, even when there is no performance bottleneck: it's the approximation / interpolation of the turn events to the intended turn length. The computation may be done in 50ms, but the events should unfold in 500ms on screen. I think the approximation depends on the performance of the recent frames, is inaccurate and can thus overshoot often. At least that's the only theory that I can come up with that explains when one measures and displays the time between the "TurnStart" and "TurnEnd" network packets (#5323). If I recall correctly, even on a new_rms_test map with about 20 entities on the entire map and only the localclient, < 10ms ping and no units moving there are often 550ms between the two packets. Other things to thread: On 12/13/2018 at 10:40 PM, Kuba386 said: There are many other things to multithread. Like NetClient or map generation. I can try multithread them too, but I'm really not sure how much can I do here. There are some edge cases that need async calls or threading: The NetClient threading doesn't really remove performance bottlenecks, but it is absolutely necessary to split it so as not to get disconnected when some C++ or JS code blocks the main thread for some seconds, like we saw in every single a23 4v4 if one didn't simulate lag to increase the timeout tolerance artifically. (ticket and patch somewhere) Map generation is threaded already (one couldn't see the loading screen progress). But not all parts of the loading screen are threaded. Running the renderer and simulation in different threads might or might not have potential (for both loading screen and ingame). Rejoining clients that finished the loading screen simulate the missed turns - that can freeze the application for many seconds. This is the most important issue to be fixed. It would be probably better if the rejoining client simulates 3 turns, the other players one turn, rather than the others simulating 0 turns while the rejoining client simulates all missing turns. XMPP connection startup freezes the GUI until the server responds or timeout (ticket somewhere) STUN connection buildup freezes the GUI for some hardcoded seconds replay menu loading thousands of new replays will freeze for that period userreporter libCURL interaction was threaded, but if the server times out, then the main thread exists while the userreporter thread still waits (ticket somewhere) 1 Quote Link to comment Share on other sites More sharing options...
Stan` Posted December 23, 2018 Report Share Posted December 23, 2018 Okay, so first I must say how astounding the performance is with that patch (I'm using 12 threads on an hexacore CPU I7-8750H) I can play a game at 20x with 4AIs at around 10-20 fps in mid game. (It went so well that I ran out of memory maybe the Leak @elexis is tracking down) I think you forgot to edit the default.cfg to set the default number of threads (By default no number is set) Every time I open the option menu I get a warning, likely related to 1. Here is the log. WARNING: JavaScript warning: gui/options/options.js line 228 reference to undefined property option.tooltip WARNING: JavaScript warning: gui/options/options.js line 263 Script value conversion check failed: v.isString() || v.isNumber() (got type undefined) WARNING: JavaScript warning: gui/options/options.js line 228 reference to undefined property option.tooltip WARNING: JavaScript warning: gui/options/options.js line 263 Script value conversion check failed: v.isString() || v.isNumber() (got type undefined) WARNING: JavaScript warning: gui/options/options.js line 263 Script value conversion check failed: v.isString() || v.isNumber() (got type undefined) @Servo Here is a Mac version of the game with the patch: system.zip To install: Download the file Extract it (clicking on it should do the trick) Open Finder Go the Application folder Right click on 0 A.D. Click on show contents Open Contents Copy LibAtlasUI.dylib in Frameworks Copy LibAtlasUI.dylib in Frameworks Copy Pyrogenesis in MacOs Install pathfinder_mod.zip Launch the game Enable and Start the mod In the game options set the number of threads you want For those on windows that don't want to compile here is the modified version on Windows (Sorry linux guys but you can compile yourselves ) Here is the windows version patched exectuables: system.zip Go in your 0AD installation folder Go to binaries go to system Overwrite the files Install pathfinder_mod.zip Launch the game Enable and Start the mod In the game options set the number of threads you want 1 Quote Link to comment Share on other sites More sharing options...
wowgetoffyourcellphone Posted December 23, 2018 Report Share Posted December 23, 2018 (edited) Should "pyrogenesis" file in system.zip be the exe or pdb or? It currently has no file type. Edited December 23, 2018 by wowgetoffyourcellphone Quote Link to comment Share on other sites More sharing options...
Stan` Posted December 23, 2018 Report Share Posted December 23, 2018 34 minutes ago, wowgetoffyourcellphone said: Should "pyrogenesis" file in system.zip be the exe or pdb or? It currently has no file type. Try the other zip EDIT: Made the link not bug. 1 Quote Link to comment Share on other sites More sharing options...
wowgetoffyourcellphone Posted December 23, 2018 Report Share Posted December 23, 2018 (edited) How many "threads" should I set? Edited December 23, 2018 by wowgetoffyourcellphone Quote Link to comment Share on other sites More sharing options...
Stan` Posted December 23, 2018 Report Share Posted December 23, 2018 You can try one per core. Intel I3 usually have 2 to 4 Intel I5 4 Intel I7 8 to 12 Intel I9 can go up to 32 I think At least two to see a difference Quote Link to comment Share on other sites More sharing options...
Guest Posted December 23, 2018 Report Share Posted December 23, 2018 Regarding the OOM, someone over at HC also had encountered an OOM. The common thing being multiple AIs. Quote Link to comment Share on other sites More sharing options...
OptimusShepard Posted December 23, 2018 Report Share Posted December 23, 2018 I tested it a few times. One time it crashes, the other times the game freezes. Without the patch I didn't have any "problems" (besides lagging). Til the game crashes it runs much smoother. My Settings: - Intel i5 8250u, 8GB RAM, UHD 620 - 500 vs 500 (AI) vs 500 (AI) vs 500 (AI) Units - Settings to 6 Threads and higher crashlog.dmp 1 Quote Link to comment Share on other sites More sharing options...
OptimusShepard Posted December 23, 2018 Report Share Posted December 23, 2018 16 minutes ago, OptimusShepard said: My Settings: - Intel i5 8250u, 8GB RAM, UHD 620 - 500 vs 500 (AI) vs 500 (AI) vs 500 (AI) Units Same settings, 4 Threads it freezes because of error "out of memory". Quote Link to comment Share on other sites More sharing options...
Stan` Posted December 23, 2018 Report Share Posted December 23, 2018 32 minutes ago, (-_-) said: Regarding the OOM, someone over at HC also had encountered an OOM. The common thing being multiple AIs. Yes that's AIs. However the fact that the game crashes with out of memory makes me think @Kuba386's patch triggers something faster. I compiled the exe with the large address aware patch and it crashed at around 3.5 GB of RAM usage. 4 AIs hardest difficulty. No lag whatsoever though. Quote Link to comment Share on other sites More sharing options...
Kuba386 Posted December 23, 2018 Author Report Share Posted December 23, 2018 1 hour ago, OptimusShepard said: Same settings, 4 Threads it freezes because of error "out of memory". Sad thing to hear. I will look at this issue. Thank you very much for reporting. 2 hours ago, stanislas69 said: At least two to see a difference At least one to see a difference. Using single thread gives big difference from not using threading at all. 2 hours ago, wowgetoffyourcellphone said: How many "threads" should I set? Usually optimal value is around 2*(number_of_cores-1). At least from my experience. Quote Link to comment Share on other sites More sharing options...
wowgetoffyourcellphone Posted December 23, 2018 Report Share Posted December 23, 2018 Spoiler OS Name Microsoft Windows 10 Home Version 10.0.17134 Build 17134 Other OS Description Not Available OS Manufacturer Microsoft Corporation System Name JUSTUSLAPTOP System Manufacturer Acer System Model Aspire V3-771 System Type x64-based PC System SKU Aspire V3-771_0686_2.14 Processor Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz, 2601 Mhz, 2 Core(s), 4 Logical Processor(s) BIOS Version/Date Insyde Corp. V2.14, 12/7/2012 SMBIOS Version 2.7 Embedded Controller Version 0.00 BIOS Mode UEFI BaseBoard Manufacturer Acer BaseBoard Model Not Available BaseBoard Name Base Board Platform Role Mobile Secure Boot State On PCR7 Configuration Binding Not Possible Windows Directory C:\WINDOWS System Directory C:\WINDOWS\system32 Boot Device \Device\HarddiskVolume2 Locale Earth, Sol system Hardware Abstraction Layer Version = "10.0.17134.471" User Name JustusLaptop\JustusAvramenko Time Zone Eastern European Standard Time Installed Physical Memory (RAM) 6.00 GB Total Physical Memory 5.82 GB Available Physical Memory 1.49 GB Total Virtual Memory 11.4 GB Available Virtual Memory 4.79 GB Page File Space 5.54 GB Page File C:\pagefile.sys Kernel DMA Protection Off Virtualization-based security Not enabled Device Encryption Support Reasons for failed automatic device encryption: TPM is not usable, PCR7 binding is not supported, Hardware Security Test Interface failed and device is not InstantGo, Un-allowed DMA capable bus/device(s) detected, Disabled by policy, TPM is not usable Hyper-V - VM Monitor Mode Extensions Yes Hyper-V - Second Level Address Translation Extensions Yes Hyper-V - Virtualization Enabled in Firmware Yes Hyper-V - Data Execution Protection Yes So, threads = 2 for the system above? Quote Link to comment Share on other sites More sharing options...
Kuba386 Posted December 23, 2018 Author Report Share Posted December 23, 2018 (edited) Since you have 4 logical processors I would try using 4-6 threads. Would be nice if you could try using 1,2,4,6 and post results here. Edited December 23, 2018 by Kuba386 Quote Link to comment Share on other sites More sharing options...
Kuba386 Posted December 23, 2018 Author Report Share Posted December 23, 2018 2 hours ago, OptimusShepard said: Same settings, 4 Threads it freezes because of error "out of memory". What if you use one thread or disable threading? (0 threads) Quote Link to comment Share on other sites More sharing options...
Stan` Posted December 23, 2018 Report Share Posted December 23, 2018 Do you get the out of memory error when pitting 4 AIs against each other @Kuba386 ? Might be a windows only bug. Quote Link to comment Share on other sites More sharing options...
Kuba386 Posted December 23, 2018 Author Report Share Posted December 23, 2018 (edited) 6 hours ago, stanislas69 said: I think you forgot to edit the default.cfg to set the default number of threads (By default no number is set) Yeah, I forgot. If you din't tick the checkbox default number of threads will be used, so by default default number of threads is used Thank you for packaging this patch 6 hours ago, stanislas69 said: It went so well that I ran out of memory maybe the Leak @elexis is tracking down I will take a look at this too. 8 minutes ago, stanislas69 said: Do you get the out of memory error when pitting 4 AIs against each other @Kuba386 ? Might be a windows only bug. Actually my memory usage is 3.5 GB for 7 AIs on linux. My memory usage exceeds 4GB (5.5GB at the moment), but I get no crash. I've heard that windows version is 32-bit so cannot allocate more than 4GB. Edited December 23, 2018 by Kuba386 1 Quote Link to comment Share on other sites More sharing options...
Guest Posted December 23, 2018 Report Share Posted December 23, 2018 (edited) The only leak fix I have seen is the one on phab. Which would be irrelevant for a standard game (see the rev proposal). If something else, I don’t know. But most likely not windows specific. Edited December 23, 2018 by Guest Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.