wraitii

Tuan Kuranes: I meant try to avoid as many temporary allocations as we can, and use the existing ones as efficiently as possible. Beyond that, it's probably the only way to go to get a pool. Now, on the other hand, it seems to me this whole debate is quickly getting into the "unconstructive" area. There's a little too much being thrown in the air with numbers flying all over the place. I would suggest a little recap of the points debated here, because based on what is being discussed, it sounds a little too much like the whole game code needs to be rewrote absolutely-or-it's-never-gonna-work, and that's not going to be very constructive.

Looked at your code, Tuan Kuranes. Beyond the ton of const changes (you could have put those in another patch for readability ), it seems mostly sensible. I'd recommend you take a look at the existing patches for the rangeManager, there are some more things there. The fixed issue is not an issue as long as the network cannot deal with smaaaall precision error: we can't afford the change. I would rather optimize the current code to avoid too many memory allocations/deallocations than switch to a pool system though. We need to know how much faster that would be than better use of memory.

I like the third the most, aesthetically. Fortress feels AoM like but that's not a bad thing. Great concepts nonetheless.

That was later in the game. I'm not sure what calls this function, or for the matter how accurate that number is (I'm fairly confident for the main thread, but this was apparently another thread. The JS stuff is kind of exploded in the profiler). Frankly, I'm really not sure what that particular function is. If I investigate it further, it tales me to assembly code which reads "Jaeger trampoline", which is apparently a thing from MethodJIT.

Basically it's Xcode's profiling tool, and that's a % of the time taken by the game to run over about 30 seconds. I'm not sure if that's clear? edit: Apple's guide to the Xcode instruments. it's not really that useful though ^^

Historic_Bruno has a point with the fact that late-game is where the real lag happens. But there are some interesting things to get from early-game extreme situations, because that's when the non-obvious slowdowns are noticeable. Some more random profiling info from starting a game on Peloponnese (according to XCode's profiler). Time spent in "QueryInterface": 2.5% Time spent in "InstancingModelRenderer:RenderModel()": 18.7%. 13.7% of that was system command "glUpdateDispatch" (not sure if that means anything, just shows we can get interesting tidbits of data). To contrast, some info On Median Oasis with 3 very hard Aegis bots at 19 minutes. They had expanded greatly. CheckStackandEntherMethodJIT: 16.9% CGame and CRender took basically the same amount of time, respectively 38 and 28% of the time (at game start it's much more contrasted). Pathfinder::ProcessShortRequest was 7.3% of the time (total pathfinding was 7.9). "BroadcastMessage" took 15% (23% total, but the direct calls were 15%), which went (absolute numbers): -2.3% to VisualActor update (0.9 from calls to CUnit::ReloadObject in CUnit:SetEntitySelection, 1.0% from UpdateVisibility) -2.9% to UnitMotion (1.1% for "TurnTo, 0.7% for "MoveTo", 0.6% for calls to Pathfinder::CheckMovement) -8.2% to the rangeManager, with expectedly 7.8% for ExecuteActiveQueries (about 30% of the time there was SpatialSubdivision::GetInRange.) We have patches that speed this up considerably, I'm thinking it would take less than 2/3% with those, which is non-neglectible. Rendering took 38.4%, 2.5% for the GUI and 4.4% to enumerate objects (which is basically caused by GetWorldBoundsRec and "CModel:ValidatePosition" Some interesting tidbits for the renderer (numbers are now relative to the time it took to complete RenderSubmissions) There was basically no water in my shot, yet waterreflection takes 6.8% of the time, mainly because of RenderPatches. TerrainRenderer::renderpatches takes 11% of the time, mostly for renderblends and renderdecals (5.5 and 3.7%). The shadowmap took basically the same time as rendersilhouette, 17%. Overall InstancingModelRenderer::Rendermodel takes 36.9% of the time spent in RenderSubmission. Now all this doesn't mean much, obviously, but it does show that rendering takes time. But it also shows that the rangeManager is right now one of the biggest slowdowns. BroadcastMessage's overhad was about 3% of the time, which isn't really too much.

About the AI. We should be able to move pathfinding to C++ (perhaps using the main pathfinder with that. The issue is that if it's threaded, it needs to be thread-safe. I'm thinking AIs wouldn't really need much more beyond knowing if something is accessible, a "real distance from there to there" feature and some basic long-range pathfinder (which I have implemented in JS as an oversampled A* (basically I look over every 3 tiles and note very tile, which makes it acceptably fast on my computer), so basically hooking some of the long-range pathfinder features could be enough). The AI also deals a lot with maps, when it really shouldn't. In particular for dropsite placements, it does fairly expensive stuffs. I'm not sure how possible it is to switch this to C++, but there ought to be some way to gain performance there. Finally the entity "collection" system for AIs is still fairly dumb and could be optimized a lot. And then there's the rest. In particular, the GC issue is tougher to deal with. Fixing all AI leaks, if any, wouldn't fix it completely since given the architecture (it gets the simulation state from the JS (maintained by AIProxy), and that's about 40kb each turn) it will OutOfMemory someday.

This is one of those things that seem like they'd improve performance slightly and be generally good, I support it. FYI, XCode has a really interesting profiler too in the instruments, so I could perhaps also get some data from that. (edit: that code does seem crazily inefficient, though) Edit2: ran the game a little in the main menu, Xcode's profiler tells me about 40% of the time was spent walling CGUI: textDraw if I read things correctly.

Excellent post, Yves, I completely agree. I'd add that there are probably a lot of slightly inefficient code bits in quite a few areas of the game, which overall would probably give semi-substantial improvements, but would require probably not too obvious changes (and possibly tedious ones, like caching things, or reinventing slightly better algorithms). I would like to make a case for proper model culling, we currently cull models once per frame, in a not really efficient way (we have to check the world bounds against the frustrum, which is fairly slow, and sometimes recalculate the bounds, which is slower). We would need to do it faster, so we can do it more often (the code for water reflection and refraction fakes culling by flagging unwanted models. But since the rendered recreates the bucket and all, this is not really efficient (and again, the method for culling is slow). This is bad because we can't really do this efficiently, it introduces bugs, and particularly as I'm working on multiple water level (infinite, really), we would really need to be able to better cull models. This is sort of linked to the Ogre and the Octree discussion, but I think it's one of the more important point.

I'd be against a pop-up reminder, I usually find those things more annoying than anything else. I would be all-right with an in-game button though (to trick the player: in the "you won" screen )

I've just committed some fairly extensive behind the door changes to the water code: it no longer uses textures but rather vertex data for rendering the foam. Please report any issue that could arise. It should be faster for rendering but it's most likely unnoticeable.

Similar results in both my Macbook air with GAM 950 and my ≈4 year old iMac: On Gambia river, after lowering the camera as much as possible (using the start camera) and revealing map: Without the patch, full rendering of a frame is according to the profiler 170ms. With it, it's 140 ms. I'm fairly sure we could win another 40 ms if bushes used the no-sort material too. Doesn't seem to be any artifacts but I invite you to test it out. Obviously this only speeds up tree rendering when there are lots of them. For the record, I think we could also use alpha test on a per-material basis, and it could be another speed up for trees without many changes either.

Sure. First screenshot is with sorting, second screenshot is all sorting disabled (couldn't bother to reapply the patch). (fairly big files, didn't want to compress).

Original change talked about in the topic is now in a patch at #1920.

There are no such mods now. It can definitely be modded in using the experience though. This reminds me of "rpg" maps in AoE 2 multiplayer.

That along with different water levels would be nice. it would require changes to atlas tough for support, I'm not too sure how (i was thinking keeping a 2d map of water height, some basic info for running water. I think changing the texture could then be handled automatically: if the water plane is flat/enclosed, it's a lake, I it's big enough, an ocean, and of it's of different heights, it's a river (possibly switching in the middle). The rest of what you describe ought to be doable. Biggest slowdown right now is refraction/reflection rendering tough and there's not much currently to do to speed that up (need to speed the rznderer up...)

Current water rendering is not exceedingly satisfactory, so some changes are definitely in order. I'm not sure how doable a system like red alert would be, it's obviously fairly advanced, but we should be able to do better than what we have.

From what I know of the code, implementing ogre fully means changing the entirety o the GUI folder and perhaps also converting GUI files(to use ogre. If we actually switch the GuI its a full change) It means a complete change of the renderer folder. It means tons of updates in the graphics folder, but that actually shouldnt be too much of a problem. Might require some changes to collada(can't recall if ogre handles collada natively). Ps/ probably has files that would need to be changed as its the core of pyro genesis, but that also shouldn't be a problem with your experience The issue is maths/. If you use the ogre provided versions of matrices, vectors, it Ould require changing things everywhere in the code. Perhaps a simpler solution would be to typedefs those types (which would maintain the usage of CWhatever for class names) The rest should be mostly left as is. However ogre was not chosen before, and we need to know why to check if the reasoning still applies. I suggest you start a new topic explaining what we would gain (in terms of speed, flexibility, features) and how much time you think it'll take after assessing the necessary changes. Then well be able to take a sensible decision

It's definitely a very interesting long term perspective, particuarly as the renderer is quite limited. On the topic of floats vs fixed, unless serialization and OOS checks can be efficiently changed to disregard all roundin errors, this is something that needs to be kept. Perhaps however it can be optimized a little (20% seems fairly big). Anyway, there are other areas where optimizations are interesting so that's nothing too important. Edit: this is probably impossible, but another solution would be SP to have floats and Mp to have cFixed. Might require two apps though, à la CoD and perhaps some other games.

Even for fairly small areas it's very computationally intensive. There's a reason if hardcore realistic water is really rare in video games

I think that having the basics down is the necessity. If some stuffs require behind the door changes later, it's less of a problem. This is so the work done on components afte the ECS will be there for good, such as optimizations or new features. The renderer is a big Change that has the advantage of being very long term and really good FPS-wise, so that's why I think it's the second highest priority (particulalrly as theres really noone else to do it). Of course, that's only my opinion, and others should weigh in on this.

It's fairly simple, really. If its too long to refactor the ECS, then it'll stall development and while in alpha with not all features implemented, that's really undesirable. Unless it can be done fully and then one day committed in one go, it should not be done now. That's true of the renderer too, but to a lesser extent, and wouldn't prevt adding new things. It's however probably not quite as long (depends. You might well be fast if you know Ogre). The problem with focusing on the renderer is that changes to the components are "lost" in that many things will change, so it might also prevent work on thy, whoch we do not want. My vote would be for the ECS, but it needs to be in one go, with full support of existing functionality, which means it needs git. Then the focus should probably be the rznderer. Other things can likely be done by other people anyway if the architecture is there (slower, but that doesn't really matter)

From what I know of ogre, switching to I would almost allow to keep every file as is. Might require rewriting shaders to cg, but that's close enough from glsl. We wouldn't really lose much. I'd agree that it would be more urgent to, say, rewrite the component system: this would probably stall development for a while, but allow it to resume faster later (c++ side anyway). The renderer is "not quite slow enough", I find. It's an area where absurd optimizations could be done, but generally I think it'd be best to have the final component/simulation architecture down first.

Basically the current implementation seems fine: only "pack" the files for releases. The engine already for that, just need to change the way it packs.

It's most likely some variation on the best way to go for a moddable game. I think civilization IV uses some sort of similar system ("loose" or readable files are cached into binary files for later uses). 0 AD should definitely allow something like that. Given the architecture of the "mods" folder, I'm thinking keeping a separate "pack" cached for each mod. The engine would need to be able to handle either the original files and cache them, or the packs, or a combination of both. Keeps moddability, readability, and efficiency. (obviously takes more time to do but or the sake of argument, ill assume your time is infinite )