Howdy...

[fredsanford]

I figure it's time to stop in and say hello... I like your game a lot and would like to help with what polish I can provide.

I'm a C++ (and C) programmer with about 22 years of experience in a lot of different industries, except the gaming industry. A lot of my work time in in the past was spent incrementally optimizing things and removing code while leaving the same or improved functionality. I both enjoy this type of work and do it well for the most part.

I've taken an interest in the path-finding and formations, formation movement area and over the last few weeks I've come up with some (small) patches.

PM me if you'd like me to post them. Note that the patches are small and may actually be wrong but each change did make at least a small improvement to either run-time or for mitigating pathological behavior.

Note that I have some issues which prevent me from working on anything for long uninterrupted periods of time. It seems my body is forcing me to make up for all the sleep I missed in my 20s and 30s. This generally means it takes me anywhere from 1.25 to 2.0 times as long to do something.

Anyway...

I have some questions if anyone is willing to take the time to answer.

1) The way formations reform as they turn drives me crazy. Is there anyone who can give a brief overview of how it should work and how to find the related C++ and JS code? JS is not my forte, but I can probably figure it out.

2) Why is the vast majority of the code using fixed point? Is it a historical artifact? By using floating point and one of the better Template Math libs (that uses SSE type instructions), the math could likely be sped up by a big margin. If you point me to an isolated smallish area of the code to change to floats, I will do the changes, testing and profile etc... IOW, I'm willing to put my money where my mouth is.

3) In the file CCmp_Pathfinder_Tile.cpp there is a fair amount of redundant info spread across several data structures. An example is state.terrain (which looks like it could be combined with state.tiles) that contains only 2 bits of data related to impassable tiles. I'd be willing to clean this up, but not to waste time on it for no effect.

4) Was any of the code written by Ykkrosh/Patrick?/Phillip? saved anywhere? He seemed to write in the forums about some nice improvements, especially related to path-finding, but I do not see code attributed to his svn/git IDs in the repository.

5) Is there any way to setup a game that will play itself? I've seen references a few times to a command line option (-replay) to replay from a text file, but I have not figured out how that text file is created.

Thanks for listening to the geezer ramble.

Edited October 10, 2012 by fredsanford

[feneur]

Hello and thanks for your offer, I'm sure we've got a lot of things you can do and we're certainly grateful for any help we can get in this specific area

Now, I'm not a programmer, so I can't say much about the details I'm afraid, hopefully one of the programmers will have time to reply in more detail soon.

The fixed/floating point issue, again, not a programmer, so, yeah... But if I recall correctly with floating point it's harder to get the exact same result on different OSs/computers, and for the simulation to work the game needs to be perfectly synced. There might be ways to make sure it's the same for all players etc, so again, we need a programmer to reply, but the general concern of course applies to all aspects of programming, we need to make sure the code executes the same on all computers or we'll have OOS (out of synch) errors

In short pathfinding/formations is a real mess the way they are currently, as you have obviously seen. Philip (or Ykkrosh as he's known on the forums) did indeed spend quite some time working on the game pathfinding, and while I can't recall him posting any code (I might be wrong, in which case I hope someone else remembers where to find it.) he did at least post this document: http://trac.wildfiregames.com/export/11728/ps/trunk/docs/pathfinder.pdf which should at least give some basic information.

For the command line reference see the readme.txt file in binaries/system And also, just so I'm sure you're aware of it, we do have quite a lot of information in the wiki at: http://trac.wildfiregames.com/

[zoot]

About formation turning, there is a bit of discussion here: http://www.wildfiregames.com/forum/index.php?showtopic=16310

I only have a partial understanding of it myself, but maybe you can find some pointers there.

[fcxSanya]

<...> while I can't recall him posting any code (I might be wrong, in which case I hope someone else remembers where to find it.) <...>

I remember seeing only this one (from 2012-02-10-QuakeNet-#0ad-dev.log):

12:27 <@Philip`> (http://zaynar.co.uk/0ad-pub/pathperf2.patch is the incomplete unusable patch as of a couple of days ago, though)

Which is apparently quite old. But we can ask Philip for the latest version.

[fcxSanya]

5) Is there any way to setup a game that will play itself? I've seen references a few times to a command line option (-replay) to replay from a text file, but I have not figured out how that text file is created.

The text file is a simulation log, which as far as I understand is created for each played game under name 'commands.txt' in sim_log/<pid> subfolder of logs folder (see GameDataPaths wiki page to find it).

To setup a game without a human player you can run it like:

./pyrogenesis -quickstart -autostart="Acropolis 01" -autostart-ai=1:qbot-wc -autostart-ai=2:qbot-wc

Edit: also you can speed it up from the in-game console (opens on ~ or F9) with:

Engine.SetSimRate(<number>)

where <number> is 1 for the normal speed and something bigger for a faster speed.

[quantumstate]

For the fixed point stuff, we need to have perfect compatibility between all systems which run the game for multiplayer as feneur said. It also needs to be converted to javascript and back safely. I don't know much about this, but if you try changing it you will need to be pretty sure about it (backed up by some sources so whoever reviews your code can be sure). The code that is not network synchronised (anything outside of simulation2, and some of the stuff in simulation2) can be changed safely, but I don't know if there is any fixed point stuff in there.

Also please put your patches on trac with [PATCH] in the ticket title. Someone should eventually get round to looking at them. If you think they are ready to be applied to svn you can add the review keyword.

With formations, we plan to change how they work significantly. I don't think anyone quite knows how we want them to behave yet though. It might not be worth spending a lot of time on them now.

Similarly with the pathfinder it is probably worth waiting until we have the latest patch from Philip to save wasted effort on code which will be replaced.

Also another thing to avoid is CCmpRangeManager since I have been working on optimising that (changing the map to a vector for a good memory/speed trade).

I look forward to seeing your work, we do need some nice performance improvements .

[fredsanford]

Hello and thanks for your offer, I'm sure we've got a lot of things you can do and we're certainly grateful for any help we can get in this specific area

Now, I'm not a programmer, so I can't say much about the details I'm afraid, hopefully one of the programmers will have time to reply in more detail soon.

The fixed/floating point issue, again, not a programmer, so, yeah... But if I recall correctly with floating point it's harder to get the exact same result on different OSs/computers, and for the simulation to work the game needs to be perfectly synced. There might be ways to make sure it's the same for all players etc, so again, we need a programmer to reply, but the general concern of course applies to all aspects of programming, we need to make sure the code executes the same on all computers or we'll have OOS (out of synch) errors

In short pathfinding/formations is a real mess the way they are currently, as you have obviously seen. Philip (or Ykkrosh as he's known on the forums) did indeed spend quite some time working on the game pathfinding, and while I can't recall him posting any code (I might be wrong, in which case I hope someone else remembers where to find it.) he did at least post this document: http://trac.wildfire.../pathfinder.pdf which should at least give some basic information.

For the command line reference see the readme.txt file in binaries/system And also, just so I'm sure you're aware of it, we do have quite a lot of information in the wiki at: http://trac.wildfiregames.com/

Thanks Feneur, you've given me some nice stuff to chew on. The pathfinding document answers a lot of my detailed questions.

I suspect you're right about the fixed point math as well. The sync problem did not occur to me even though it's been a problem in the past.

[fredsanford]

About formation turning, there is a bit of discussion here: http://www.wildfireg...showtopic=16310

I only have a partial understanding of it myself, but maybe you can find some pointers there.

Thanks Zoot.

I remember seeing only this one (from 2012-02-10-QuakeNet-#0ad-dev.log):

Which is apparently quite old. But we can ask Philip for the latest version.

Hi fcxSanya. Is Philip still around and working on stuff? If so, I'll leave the pathfinding and formations alone as he has spent huge amounts of time on pathfinding at least.

I assumed he stopped working on it as I see nothing from him since 02/2012 or so. Proving yet again that assumptions are bad.

The text file is a simulation log, which as far as I understand is created for each played game under name 'commands.txt' in sim_log/<pid> subfolder of logs folder (see GameDataPaths wiki page to find it).

To setup a game without a human player you can run it like:

./pyrogenesis -quickstart -autostart="Acropolis 01" -autostart-ai=1:qbot-wc -autostart-ai=2:qbot-wc

Edit: also you can speed it up from the in-game console (opens on ~ or F9) with:

Engine.SetSimRate(<number>)

where <number> is 1 for the normal speed and something bigger for a faster speed.

Thanks for this as well. Getting the pathfinding to the point where it slows down is quite time consuming.

Hmmm, this forum software appears to hate Google Chrome... Copy/Paste fails and quoting just spins in place... Time to break out firefox...

Edited October 11, 2012 by fredsanford

[fredsanford]

For the fixed point stuff, we need to have perfect compatibility between all systems which run the game for multiplayer as feneur said. It also needs to be converted to javascript and back safely. I don't know much about this, but if you try changing it you will need to be pretty sure about it (backed up by some sources so whoever reviews your code can be sure). The code that is not network synchronised (anything outside of simulation2, and some of the stuff in simulation2) can be changed safely, but I don't know if there is any fixed point stuff in there.

Also please put your patches on trac with [PATCH] in the ticket title. Someone should eventually get round to looking at them. If you think they are ready to be applied to svn you can add the review keyword.

With formations, we plan to change how they work significantly. I don't think anyone quite knows how we want them to behave yet though. It might not be worth spending a lot of time on them now.

Similarly with the pathfinder it is probably worth waiting until we have the latest patch from Philip to save wasted effort on code which will be replaced.

Also another thing to avoid is CCmpRangeManager since I have been working on optimising that (changing the map to a vector for a good memory/speed trade).

I look forward to seeing your work, we do need some nice performance improvements .

Thanks Quantumstate. Looks like the little bit of code I've messed with is redundant and/or made obsolete by Philip's changes. Since there have been no publically visible changes from Philip since Feb 2012 or so, I thought I'd start picking at the edges. If he's completely rewriting things my changes/bug fixes are kinda useless.

Also another thing to avoid is CCmpRangeManager since I have been working on optimising that (changing the map to a vector for a good memory/speed trade).

One big speedup for generic STL code I've had success with in the past is to use a fixed-size allocator for vector<whatever> that only goes to the system for memory once. This is something I was experimenting with in the pathfinding code, especially where I estimated vector.size was less than 100 or so.

You can see an example of this in use here. Look at fsa.h and stlastar.h.

[fcxSanya]

Is Philip still around and working on stuff?

This is a hard question. What I can say for sure - Philip is significantly busy with the real life now.

From quick grepping the logs, it looks like the last time when he said something about the pathfinder and his plans was this (2012-08-28-QuakeNet-#0ad-dev.log):

18:04 <@Deiz> Do you have any idea when you'll have more free time?

18:05 <@Philip`> I almost have some free time now

18:06 <@Philip`> If anyone can summarise the past ~2 months of activity on IRC and forums and Trac, then I won't have to bother reading the archives/logs/etc, and I could probably start looking at some actual code again :-)

About his general availability (2012-08-12-QuakeNet-#0ad-dev.log):

17:28 <@Philip`> fcxSanya_: I always read email, though I rarely get around to responding :-(

17:28 <@Philip`> (I'm present on email whenever I'm not asleep)

17:29 <@Philip`> (unless I don't notice I've got messages)

Also IRC client on his server connected to our IRC channels 24/7, but Philip is not always connected to it.

If so, I'll leave the pathfinding and formations alone as he has spent huge amounts of time on pathfinding at least.

He indeed spent a lot of time on this, but the problem that to finish/integrate those changes a lot of time more is needed. And we can't be 100% sure that Philip (or someone else) will be able to finish this work at all, and especially we can't confidently expect any results soon (e.g. for the next alpha). And the bad performance is #1 in our known issues list, so if you are able to provide any optimizations it would be very welcome.

Hmmm, this forum software appears to hate Google Chrome... Copy/Paste fails and quoting just spins in place... Time to break out firefox...

We have some issues after the last forum upgrade. You can add your one here.

[quantumstate]

One big speedup for generic STL code I've had success with in the past is to use a fixed-size allocator for vector<whatever> that only goes to the system for memory once. This is something I was experimenting with in the pathfinding code, especially where I estimated vector.size was less than 100 or so.

You can see an example of this in use here. Look at fsa.h and stlastar.h.

In this case allocation is not an issue, there is one entry for every entity in game and not many entities get created in a game. There is lots of random access though, since this is what is used to work out nearby entities.