wraitii

I don't really think this is survival of the fittest, though. That's a fundamentally PVE game, I'm definitely thinking of some added element to PvP or PvAI.

Ran some more profiling, it seems PerformGather is really slow when the Fogging activates (which isn't really surprising), generally because copying information is slow (presumably because querying all those interfaces is a little slow). Not entirely sure how we could improve the data copy though. For the rest, the Gathering timer could be improved by removing a few checks in the beginning (in particular CanGather check seems useless since PerformGather does it again). Sending stuff to stats and the AIProxy takes around 0.2ms per turn, which isn't trivial but doesn't seem worth optimizing. Looking at other Timer calls, the important UnitAI ones are Combat.Attacking and Repair.Repairing, so those are the targets.

Looking more specifically at Gather, it seems the slower parts are the following: -The initial call to "CheckTargetRange and CanGather". Can probably be slightly optimized. About 1ms at turn 1500. Probably slow because called so often and may call CPP -The case where we need to find a nearby resource or dropsite (haven't looked in detail but most likely the first). -The call to return to the nearest dropsite when we're full (about 2ms) -The actual call to cmpResourceGatherer.PerformGather. This is the big one, 5ms on average at turn 1500. This is annoying because this last call is slow mostly because of: -The PostMessage "ResourceCarryingChanged" -The PostMessage in ResourceSupply "ResourceSUpplyChanged" -The call to Fogging We should consider directly calling AIProxy for those, perhaps that would make it faster. Another thing we could do is wait for turn end to send the resource supply message, since often a resource is worked on by many entities and we might avoid sending repetitive messages that way (unsure). Another thing is that in MP in particular the timer may get called several times per frame, and we are not "clever", we call everything twice, which is wasting resources. We might win some time by calling the timer less often. Not sure exactly how often that happens with resources though, but if it happens it's late-game where lag is already bad, compounding the issue.

Opening this thread to post some profiling results for my own sake and if anyone cares. It's not going to be very formatted Results after the 2000 turns, some at 3000 (i.e. after the 15th minute, we can reasonably assume players are skirmishing and have a solid amount of pop by then) I was looking at which components and messages take the most time. Using an MP replay of a 2v2 for these. In terms of component Guard is an obvious bottleneck, taking often upwards of 5ms for no good reason. It needs to be rewritten. UnitMotion is fairly slow (7ms). UnitAI on its own is somewhat slow. And when they show up, range manager and mirage can be slow. The slowest is by far Timer.js though. Looking at that one specifically, it's because of unitAI. Though Damage calls can be slow too. For unitAI, it's mostly because everything is too slow. The Gather timer is easily 7ms per turn, Combat when it shows up is terrible, Repair is slow too. Everything needs to be sped up.

It's more like the barbarians in Civilization. You're playing normally, but if you have uncontrolled ground in the FOW, you can get barbarians that will harass you. In 0 A.D., we could occasionally spawn raiding parties of 3 or 4 soldiers. Enough to disturb your farms a little if you're not careful, or your miners outside the base, but not a huge deal.

I don't believe I've ever read this idea as I've put it below: How about we introduced the concept of "Barbarians" or rogue units in 0 A.D. ? If there are parts of the map that are not in any player's territory and under FOW, they can span small bands of gaia units hell-bent and going to your farms and raiding you. Nothing impossible to deal with, but would keep you on your toes. Possibly as a new game mode?

Completely agreed. I think we should lower their size (pathfinder and real), and make them use melee attacks against other ships.

This looks like a visualisation of the svn logs. Rather interesting.

Hey wow, thanks for the notice, points 2, 3 and 4 should be fixed now. Point 1 is actually much more difficult to do than you'd expect, so I let it out for now. Very interested to see what you can do with this, personally I think we should use this feature quite a bit. It sounds like it could lead to pretty good stuff.

What I mean is that we have "formations". So either you put those 150 units in a formation then moved that formation, or you just selected 150 units and told them to move. Either way this is probably the short range pathfinder and there's not much that can be done right now sadly :/

Were you playing SVN or Alpha 20? edit: nevermind, Alpha 20 it is. This is going to be more difficult to debug, but at a glance to the commands file the odd thing is the walk command is repeated 3 times. I'm guessing the slow framerate is because of the short-range pathfinder for 150 units running into each other. I'm assuming you did not use formations for this?

That sounds catastrophically worse than what you should experience. Can you provide us with the replay of that game?

Looks like that got back it in my latest commit. And it still doesn't happen on intel, so there's something at play here. I'll try changing it later.

I wouldn't use a function if I were you and just use global variables, depending on what you want to do. It's not like it's going to become a huge issue. Alternatively, you could return a tuple with all stuff in it. Or a dictionary, yes. But again, not a python expert.

python 3 recently added the int8frombytes and I find it more readable. No other reasons. Also not an expert in python, far from it. I don't understand your question overall.

Part of the problem is that our map are 16-bit grayscale, which is a bit of an odd format. Using Pillow on Python I used a 32-bit floating point texture, saved in tiff. FYI here's the bit of my script (taken as is), with the useless stuff removed: with open(os.path.join(sys.argv[1],i.replace(".xml",".pmp")), "rb") as f: f.read(4) # 4 bytes PSMP to start the file f.read(4) # 4 bytes to encode the version of the file format f.read(4) # 4 bytes for the file size patchSize = int.from_bytes(f.read(4), byteorder='little') heightmap = [[0 for x in range(patchSize*16+1)] for y in range(patchSize*16+1)] textureTile = [[0 for x in range(patchSize*16)] for y in range(patchSize*16)] # Height,qp is 16 bit integer divided by 92 to yield the real height for x in range(0, patchSize*16+1): for z in range(0, patchSize*16+1): heightmap[x][z] = int.from_bytes(f.read(2), byteorder='little') / 92 numTex = int.from_bytes(f.read(4), byteorder='little') #print(str(numTex) + " textures used.") texNames = [] for texI in range(numTex): # first 4 bytes are texture name length then read the texture name itself name = f.read(int.from_bytes(f.read(4), byteorder='little')) texNames.append(str(name,"utf-8")) #im = Image.new("L", (patchSize*16,patchSize*16) ) #pixels = im.load() for z in range(0, patchSize): for x in range(0, patchSize): for pz in range(16): for px in range(16): texA = int.from_bytes(f.read(2), byteorder='little') #pixels[x*16+px, z*16+pz] = (texA) f.read(6) #second texture (2) + priority (4), don't care #im.save(sys.argv[1] + "." + i + "output.png","png") #im = Image.new("F", (mapSize+1,mapSize+1) ) #pixels = im.load() #for x in range(0, math.floor(math.sqrt(mapSize))*16+1): # for z in range(0, math.floor(math.sqrt(mapSize))*16+1): # pixels[x,z] = heightmap[x][z] #im.save(sys.argv[1] + "." + i + "output.tiff","tiff") #EOF

If you need help with the python script, as part of my markov chains script I have a Python script to load a pmp map which could also be used to write a PMP map if needed.

Should be somewhat trivial to adapt to use in skirmish maps as the "script" part, then you could load it up, open the map on Atlas, save it, remove the script. I agree it would be a nifty Atlas tool also, but I don't think Atlas supports running arbitrary JS, so it'd have to be C++, and that would make it much more complicated to implement. As for creating a more complete map using real markov chains, it sounds possible but I think it'd require a different, more "top down" approach.

1. Only actors, not sure what you mean exactly otherwise 2.It's for SVN (and should also work for alpha 20)

I've been adding more stuff. First thing is I now open the PMP files, which allowed me to sort which actors go over water and which go under (or both), to avoid weirdness. I also added the possibility of adding "rules", for example a map might decide to not add entities around a certain fauna/flora element, or forbid some actors from appearing (useful on volcanic lands for example). Finally I've added a beautifier based on the terrain texture, but it only works on some textures (check out Oasis) so it's not as useful as I'd hoped. It could probably be modified slightly to become a more uniform way of placing actors around a map though. At this point I'm not entirely sure what to add next, so I invite everyone to play a bit with it and see if they like it. Generally speaking I think it looks good, needs a few tweaks. Just download the mod at https://github.com/wraitii/0-AD-RM-beautifier

I don't think a beautification pass like that is really easy to do during the map generation. It might be helped by better integration with the random map generator though. I've kept the work up, now working on adding more "meta" info. I've also added a function to add a given amount of actors, so you can easily set your density. Results at "2","10" and "100".

New comparison screenshot over SVN RM Syria. I'll let you guess which is with and without.

Yes, the script currently loops over entities (and only entities) to add actors (and only actors). Ultimately I think it should also run over actors to add actors. I don't see adding entities working very well overall.

FYI, the mod runs the beautification 4 times when it should really just be run one time, to overdo it. Here are some screenshots from a modded Kerala with no beautification. I added a flag for all added props. As you can see it adds a lot of props, particularly under dense amount of trees. I've also updated the github repo with the latest version of the script and mod. It now takes basic distances into account so the result is overall better.

Added a mod to simply see the effects: https://github.com/wraitii/0-AD-RM-beautifier Just use the beautifier mod. Basically this automatically adds beautification elements around entities. The mod overdoes it so you can see the changes.