Yves

I think the wooden cage needs something to fixate the staffs at the crossings, probably ropes. The metal cage looks too perfect. It should be less symetric and maybe rusty.

Does anyone know how much overhead there is to wake up a thread? Unfortunately I have no Idea about that and can't tell if it's even worth waiting 10 seconds instead of 1 or 0.1. My impression was that it should probably be a few seconds, but I don't really know it. Restarting the music if we had a buffer underrun is no problem. We can simply check if the source is still playing after loading additional buffers (we did that before I removed it in the patch).

Did you try the free mesa driver? As you can see above, this problem only happens with fglrx for me. Maybe we could find a workaround but since this is a freeze of xorg related to the driver, it's definitely something wrong with (one of)these components. A driver should never freeze, no matter what the application does.

It really looks like the main function wasn't able to call "g_SoundManager->Update(false);" for at least 15 seconds. Yes I think we should do that.

Thank you for testing! I've removed the workaround to start music again when it's stopped, which explains why it doesn't start again. It doesn't explain why it even stopped (never happened on my computer with the latest patch). Does it seem to be performance related? Please post mainlog.html. At the moment it measures how much time it can load into the buffer at once and then triggers reloading at intervals of half of that time. In theory if it can load 30 seconds into the buffer it should have about 15 seconds to load additional 15 seconds and to avoid a buffer underrun. I thougt that's enough even on systems with poor performance. Loading the buffers takes about 100msec on my system and there might be some msec left to complete the last frame (which shouldn't be more than 1 second even on very very slow systems). Maybe it can load less time into the buffer at once on your system for some reason, but that's something we should see in the log if you didn't remove the LOGERROR calls. EDIT: There's also some additional time we loose because a buffer hasn't completely finished playing. However that shouldn't be more than a few hundred milliseconds (30sec divided by 50 buffers). Maybe it would also help if you could check how long the soundmanager loop takes using the profiler. It should run at the interval you see in mainlog.html (~14 seconds on my system).

They said it in the video that those are placeholder models and they showed a few building of the other civ to make clear they usually look better.

It always happens on Laconia in turn 239 (-quickstart -autostart="Laconia 01" -autostart-ai=1:qbot-wc -autostart-ai=2:qbot-wc) You can set a conditional breakpoint for the condition "n == 239" on line 85 of Replay.cpp. I've lost the trace at a call to JS_CallFunctionName with the function name "HandleMessage" for player 2. Now I'm searching for a way to debug JS code properly (like singlestepping and such).

The multiplayer with no AI players usually works much smoother. However if at least one player has a very unstable and slow connection or a slow computer it can lag badly too. Another problem is the pathfinder which can become a problem on larger maps with a lot of units moving.

Hmm the game seems to be frozen when I try this. EDIT: It works with qbot-wc. ./pyrogenesis -quickstart -autostart="Isthmus" -autostart-ai=1:qbot-wc -autostart-ai=2:qbot-wc EDIT2: But it freezes later

Sounds perfect. I'll check the AI code. I think we all agree that it doesn't make sense to only loop one piece of music during a game if we have several others available. We don't necessarily have to move everything to the engine and I think we shouldn't. We could just make some new functions available for javascript and change the scripts to use them. My understanding of the "Design Committee" is that it's more focused on changes that affect the gameplay signifficantly or result in a lot of additional work. I don't think either of them applies here.

I have a new version. It calculates the thread's wakeup-interval based on how much time could be loaded into the buffer and makes sure that there's always enough data for the music and environment sounds. Additionally, I'm now using more than one mutex to avoid the unnecessary locking that causes performance issues. It's still not cleaned up, should probably be reorganized a bit and contains debugging code. @stwf How should we proceed now? This implementation is obviously not ready to commit yet and there's some more work required to get it to that point. If you have a lot of changes too, it probably doesn't even make sense to make it ready to commit, because you'd have to change it again to match your changes. How much do your changes overlap/conflict with my changes and when do you think you'll have the new version ready to commit? I have a lot of time this week and next week but then my vacation is over. If our changes affect too much of the same code, one needs to take the lead and merge it. It's fine for me either way. Another thing I noticed today. Did you know that there's only one piece of music looped over and over again even if more than one is specified in the [civ].json file? Also as far as I can tell there's just one ambient sound used (day_temperate_gen_03.ogg). A callback function that allows scripts to set the next piece to be played would be a good solution IMO. It would trigger when the end of the first peice is detected and will come soon (ogg->atFileEOF()), but not instantly (there's still some data queued in the buffers). soundmanager_threading_v0.5.diff

Welcome to the forums! I like watching videos of new players on youtube. Your accent was quite a challenge though .

That's most likely caused by the AI. Unfortunately it's not very well optimized for performance yet. You should not play with too many AIs, set the population limit to a low value and play on smaller maps. It will still take a while until that gets better.

Yes you can do something like that: ./pyrogenesis -quickstart -autostart="Oasis" -autostart-ai=1:qbot -autostart-ai=2:qbot Check: http://trac.wildfiregames.com/browser/ps/trunk/binaries/system/readme.txt However as far as I know you won't be able to watch the game in this case.

None of the soundmanagers' threading problems I discovered yet should cause problems with the UI. I would not say that it's impossible but I don't see any signs for that, beside the fact that music also stopped playing. EDIT: Hmm maybe if the mutex stays locked but I guess that would cause the whole main thread to stall.

Thank you, I hope you had a merry Christmas too! I think the data is alway uncompressed in the buffer and the code I posted above should always work. It returned about 27 seconds after calling CStreamItem::Attach for playing "Honor bound" on the main menu. I've disabled the reloading and could confirm that it stops playing exactly after 27 seconds. Indeed, we'll have to be very careful about that.

His biggest problem was the population limit until he figured out that temples can raise it. I don't know why he checked the description of all buildings except houses.

Yes it certainly varies how much play-time you get each time you load data into a CStreamItem, but since the speed of playing sound effects and musfic is fixed, it should be possible to know how much additional play-time you got after loading a new chunk of data. Probably we should make the thread's sleep-time variable and calculate it each time after reading additional data. However certain events like fading and playing sounds instantly should ignore this sleep time and wake the thread up immediately. That seems to be the cleanest solution for me. I'll try to make a proof of concept implementation to see if it works in practice. What's more safe if this is in the main thread? That's the first time I'm working with threads, so I might miss something. I've already mentioned my main argument for moving it to the worker thread above but I'll elaborate a bit. We use m_WorkerMutex to protect any data that is shared between the main thread and the worker thread. This mutex can block the main thread for quite a while under some circumstances. I think it's mainly when loading of sound data happens which involves relatively slow reads from the harddisk. The mutex is locked for the whole duration and if the main thread triggers any code that is at least partially protected by the mutex, it has to wait until loading (of all items!) is over. The main thread is the performance bottleneck and I would move as much as possible to other threads and avoid such locking problems. I've measured delays of about 10-20ms which is already too much IMO but could probably be worse depending on processor power, harddisc-load operating system etc... I wonder if this or that could be related to this locking problem, but unfortunately no one has been able to provide the required information yet. It seems to be more than 10-20 ms for these people but it could as well be something completely different. Maybe we should also use more than one mutex to lock different data and limit the locking to less lines of code. For example there's no need to protect access to m_DeadItems while we are only changing m_Items. EDIT: It doesn't read it from the harddisk everytime it queues new data... that was a wrong assumption. I've now written a test-function that checks the amount of playtime that was added. It's possible to get the data but I still have to check if it is useful for what I hope. float fSec = 0; for(int i=0; i<num_processed; i++) { ALint iFreq, iBits, iBytes, iChannels; AL_CHECK; alGetBufferi(al_buf[i], AL_FREQUENCY, &iFreq); AL_CHECK; alGetBufferi(al_buf[i], AL_BITS, &iBits); AL_CHECK; alGetBufferi(al_buf[i], AL_CHANNELS, &iChannels); AL_CHECK; alGetBufferi(al_buf[i], AL_SIZE, &iBytes); AL_CHECK; fSec += (float)iBytes * 8 / ((float)iFreq * (float)iBits * (float)iChannels); AL_CHECK; }

There are also a lot more settings in the configuration file: http://trac.wildfiregames.com/wiki/Manual_Settings And could tell us something about your system (Processor, graphics card, RAM, OS)?

I've created a first version that basically works. I haven't spent much time testing and cleaning it up but I'd like to know what you guys think about the approach. The main thread pings the worker thread every 10 seconds. If something needs more updates like when fading is started, the ISoundItem object sets a flag and triggers an immediate update. The worker thread then loops until the flag is false again(fading done) and goes to sleep only if no ISoundItem is left that has the flag set. This means that the CPU-Core is up to 100% during the time of fading(if not limited due to other load), but I guess there's no way around that if we want to be independent of the main thread's performance. I've moved the call to CSoundManager::IdleTask() from the main thread to the worker thread because apparently it can block the main thread for quite a while under some circumstances. I guess the blocking got worse because m_WorkerMutex is blocked much longer inside Run() when it isn't called that often anymore. I've also added some profiling macros, especially at places where time could be lost while waiting for the mutex. I did not add information about how long the thread is sleeping because IMO the profiler should only show load and not idling that has no impact on performance. The 10 seconds are currently hardcoded. In theory this time depends on how much play-time is left in the buffers that are queued in OpenAL. I tend to say that it's not worth the effort to calculate this sleep-time on the fly and a hardcoded value should be fine. However these 10 seconds are quite random and I didn't do any analysis to get to that value. What do you think? I'm not sure about this code: if (m_CurrentTune) m_CurrentTune->EnsurePlay(); if (m_CurrentEnvirons) m_CurrentEnvirons->EnsurePlay(); It seems to be a workaround because the playing stops sometimes. It that's true, it could take up to 10 seconds now until the playing resumes again. However I didn't notice such problems during testing. Here's the patch. EDIT: Hmm the music stops playing after a while. EDIT2: Uff I seem to have found the cause for the problem with music stopping and also why the workaround mentioned above is even needed... after searching for hours. CStreamItem::IdleTask() should call "theData->ResetFile();" also if proc_state is AL_STOPPED (plus some other conditions). While debugging this, I've found various other curiosities I'll have to examine more closely. I'll probably post a new version of the patch tomorrow. Now I must do something different. soundmanager_threading_v0.4.diff

I'm not sure if what I described above can be your problem. I've only seen it in an earlier version of my current development-code but it should theoretically also be possible that it causes problems in alpha 12. Maybe your problem doesn't happen for me because I have a stronger processor. If your willing to help finding the problem you need to get the newest development version. It would be ideal if you could compile the game from source, but getting the newest prebuilt version would still be better than nothing. Here are the instructions for building: http://trac.wildfiregames.com/wiki/BuildInstructions You need this part if you only want to get the newest prebuilt version: http://trac.wildfiregames.com/wiki/BuildInstructions#Acquiringthecode Do you think you can do that?

A vague guess is that it's related to the soundmanager problem I'm currently debugging. The IdleTask() function is called below (as part of) the render function and I suspect that it sometimes has to wait a bit long until the data/mutex it tries to access gets unblocked from the soundmanager. Give me a few hours until I have some more facts about that. You can try disabling sound in the game's configuration but that won't completely disable everything related to sound and I'm quite sure it won't solve the problem.

I think it should work everywhere as long as the .js files are in the same directory as profiler2.html.

Apply it using patch -p0 -i /path/to/patch/file Or if you already have a subversion 1.7 client svn patch /path/to/patch/file Or if you are on windows and using tortoise svn there should be a menu item to apply patches.

I think that it basically makes sense even though it's a bit simplified. Imagine you lived in village A with a lot of wood but very little stone. You'd bring some wood to village B far away which has too much stone but very little wood. They'd give you some stone in exchange for wood which you exchanged for wood again in another village © which is close to village A and also requires wood. After that you bring the wood back to village A. You were a bad trader if you wouldn't have more wood in the end . In this example you essentially just increased the amout of wood from village A's perspective even though there's still the same amount of wood "in the world".