Freeze at game start with boost 1.47

[Sylvain Gadrat]

Hello,

Recently upgraded my system from boost 1.46 to boost 1.47. Thank to svn revisions r9905 and r9907 it compile well, but get stuck when starting a game. Downgrading to boost 1.46 fix the problem, the compatibility patches don't hurt anything.

I use ArchLinux's packaged boost version. The code seems to stuck in the particles engine.

I have no more information for now, but if needed I'll can later reproduce and give logs, exact stuck location, core files,...

Note : it is not impossible that the packaged boost is buggy, if so, sorry for the dummy bug report.

System :

* Archlinux

* Nvidia's drivers

Steps to reproduce :

* Install all required dependencies for 0 A.D.

* "#pacman -Suy" (full update of the system, containing boost 1.47)

* build 0 A.D. from svn

* start pyrogenesis

* single player -> start game

Expected result :

* The game start

Current result :

* The game show your initial statement untextured, the cursor and get stuck

[feneur]

Untextured is to be expected as the textures aren't pregenerated in the SVN version. That should only be for a few seconds while the textures are being generated though, so probably not too relevant in this specific case. Just something to be aware of in general Hanging isn't to be expected though, so there sure seems to be a bug somewhere. I hope the programmers will be able to help you debug it, and if it's something we can fix, fix it.

[Ykkrosh]

It'd probably be most useful to run the game (preferably compiled in debug mode) in gdb, wait for it to freeze, do ctrl+c in gdb, then do "thread apply all bt full" to see what it's doing.

[Sylvain Gadrat]

Attached to this post : the big output of "thread apply all bt full"

I suppose that the only interesting thread is the main thread (Thread 1) as pyrogenesis get stuck in it.

I compiled with "make CONFIG=Debug" but some values still shown as "<optimized out>"

And a clue that does not appear in the attached file :

* 82 % of time is spend in "CParticleVarUniform::Compute(...)" itself

* 15 % of time is spend in "boost::random::mersenne_twister_engine<...>::twist()" called by "Compute()"

* 3% in the other componants of my system

gdb_output.txt

[janwas]

Because you mentioned that downgrading Boost fixed it, I would presume this really is a Boost bug.

That aside, I really cannot understand why the over-hyped, over-sized (we're talking a significant chunk of L1 dcache) and overly long (2**19337 possible states is FAR more than the number of atoms in the universe) "Mersenne Twister" is still in use today. One of its developers had a part in developing the more recent WELL:

http://www.iro.umontreal.ca/~lecuyer/myftp/papers/lfsr04.pdf

They also provide an implementation: just remove the final conversion to double:

http://www.iro.umontreal.ca/~panneton/well/WELL512a.c

A second, somewhat simpler implementation has been described in GPG7: http://www.lomont.org/Math/Papers/2008/Lomont_PRNG_2008.pdf

Regardless of the Boost bug, I would welcome replacing MT19337 with this seemingly superior in all respects generator.

[Ykkrosh]

I compiled with "make CONFIG=Debug" but some values still shown as "<optimized out>"

Did you also run it as pyrogenesis_dbg (the output of CONFIG=Debug builds)? It probably shouldn't optimise things out in that case.

I tried compiling myself with Boost 1.47, and get the same hang, so I'll try debugging it here.

Regardless of the Boost bug, I would welcome replacing MT19337 with this seemingly superior in all respects generator.

Hmm, interesting. I just used mt19337 since it's easily available (in Boost, which recommends it if "you have no idea which generator to use") and it's not rand48 (which is used for Math.random() in scripts to match SpiderMonkey's default Math.random() implementation, but has visible patterns when used for graphics stuff like actor randomisation and probably particles). I implemented a Boost-compatible WELL512 now (copying the seeding algorithm from Boost's MT (where it mentions Knuth) since I assume that'll do) which seems pretty straightforward so we could use that - the only concern is how to verify that the implementation actually works correctly.

[janwas]

Good golly gosh, the Boost seeding code is opaque as always

The Diehard battery of tests is frequently used to judge RNGs:

http://www.stat.fsu.edu/pub/diehard/die-c.tar.gz

There's also a "dieharder" which I'm not familiar with:

http://www.phy.duke.edu/~rgb/General/dieharder.php

[Ykkrosh]

Fixed. (The code was calling Boost's uniform_real with min==max, and in 1.47 that generates a number then returns if result < max_value else it tries again. Since it always generates a value equal to max_value it never returns and loops forever. I think that may be a Boost bug, since it requires only min <= max (not min < max), but I'm too lazy to check and to figure out where to report it.)

[Ykkrosh]

Added a WELL based on the GPG code that seems to match the behaviour of WELL512a.c; hopefully there's no blatant problems with it. None of the code uses it yet, but I guess I should move the mt19937s over to it, and probably switch the rand48s too (Math.random() in RMS and AI and simulation scripts) since why not.

[Sylvain Gadrat]

Successfully built, ran and played at revision r9956 with boost 1.47. Thank you

Did you also run it as pyrogenesis_dbg ?

Oops ! Sorry ! Happy to see that you fixed it anyway.

[janwas]

Added a WELL based on the GPG code that seems to match the behaviour of WELL512a.c; hopefully there's no blatant problems with it.

Looks good!

I guess I should move the mt19937s over to it, and probably switch the rand48s too (Math.random() in RMS and AI and simulation scripts) since why not.

That'd be great