janwas

Ooh, I'd forgotten about license.txt, thanks (did remember to include the .asm files, though, heh)

My worrying mind has come up with a bit of a legal concern about the code shared between my work and 0ad. The debug, CPU-specific and memory allocation stuff is of course especially useful for high-performance algorithms. I was mostly using it for research-only code that's not distributed, but there has recently been interest in a binary-only test app for purposes of reproducible research. Since we aren't allowed to release source code in this case, there is a conflict with the GPL. I remember reading Charles Bloom's rants about having to re-implement a big library of useful stuff for precisely this reason, and that's definitely something to avoid After discussion with Philip and Simon, who together with me logged 94% of the commits here, it seems advisable to switch to a more permissive license. The BSD license is frequently used, but the advertising clause seems unnecessary. ISC is slightly simpler, but has problematic wording according to the FSF. The Expat (MIT) license is essentially identical to the 2-clause BSD license and a tiny bit shorter, so that looks to be a good candidate. Basically, for the low-level stuff in lib/, the license says it can be used whenever and however as long as the copyright notice remains intact, and we disclaim all warranty. Since we'd like to release an EXE at work without running afoul of the GPL and given how hard it was to reach most contributors last time, I've gone ahead and committed this. If there are any objections, please don't hesitate to contact me - I'm happy to discuss it, but would also be willing to re-do the contribution in question if objections remain.

Glad to hear that it works now. Could you please post info about your graphics card and driver version (reported in %appdata%/0ad/logs/system_info.txt), so we can possibly automatically disable the offending code when running on such a configuration?

It looks like this is a sound-related issue (possibly due to a bug in OpenAL), see http://trac.wildfiregames.com/ticket/449 Do you by any chance have an onboard sound chip? (See %appdata%/0ad/logs/system_info.txt) It probably won't help in this case, but in future if you need to run 0ad with command line arguments, one way to do so is to open up a shell (cmd.exe), change directory to the 0ad/binaries/system folder, then type pyrogenesis.exe -quickstart -otherArgumentsAsNeeded and hit enter.

Makes sense (After all, MD5sum is a pretty minor snippet of code)

Bug reports via forum are fine by me. Spam isn't that big an issue (search capability is there anyway, and I don't see a relevant limit on the DB). In case you'd like to get in touch with us directly, IM contacts are in our forum profile - but it may not always be clear whom to ask about a particular bit of code. We are always grateful for testing, reports of build problems on other platforms/distros, and help tackling any open tickets In particular, those tagged as 'simple' (i.e. self-contained) might be a good place to start. Good luck with your exams!

Fair enough, but please keep in mind future encryption or authentication requirements of the network code - we won't want to roll that ourselves

Thanks for the suggestion! I see Philip went with the FIPS DLL version of cryptopp - unfortunately it looks like that fails (throws OS_RNG_ERR) on WinXP SP2 for reasons unknown. This happens before main is entered, so it's hard to debug. Maybe it is easier to go with OpenSSL after all. I gave building it a go: it requires Perl and NASM (which we use anyway), and results in an 800 KB libeay32.dll that contains MD5 and SHA1 functions. Want to use that instead? (If so: I installed the latest ActivePerl, grabbed openssl-0.9.7-stable-SNAP-20100105.tar.gz, added NASM to the path, copied nasm.exe to nasmw.exe, and followed the instructions in INSTALL.W32)

Greetings! hm, I've just successfully tested Win32 VC2005 debug and release builds of r7264, loading the default map with command line argument -onlyPublicFiles. Are you doing anything different? Are you able to see where exactly it is crashing in the debugger? (If you're getting our "Much to our regret.." dialog box without a meaningful call stack, you can comment out the __try and __except lines in source\lib\sysdep\os\win\wseh.cpp to let the IDE handle the exception - it has somewhat studlier handling of debug information)

I see newer versions of boost have added fpclassify. I'm happy to use that instead - think it's worth upgrading now?

Only 2x slower sounds decent (now at home, I can check my old implementation's quoted performance: apparently it required 11 cycles per byte on an Athlon XP). Don't worry, the code size police won't come after you for adding another DLL - quick definitely has its virtues, especially for a non-critical component like this

hm, my first thought/question is why you're focused on MD5. For example, SHA-1 hopefully isn't that much slower (I've written a fast implementation myself, the point being that it's possible) and is included in the DLL (reference: http://www.cryptopp.com/docs/ref/). Come to think about it, we could use my implementation, but it is written in assembly (then again, we already have the build system infrastructure to handle that). About other algorithms in crypto++: I agree we can live with zlib and don't need to turn to crypto++ for a slightly nicer interface. crypto++ would be useful for encryption, though. It seems it'd be good to stick with the FIPS-approved (and smaller) DLL. I don't have experience with other libraries and it's been awhile since I looked at alternatives. As to the warnings: not a problem We can wrap cryptopp's header in one of ours (see lib/external_libraries/*) - that allows setting warning level to W3 or temporarily disabling specific warnings.

Right, but here's what I'm thinking: any codepoints outside the BMP cannot be stored in NTFS, so how are Asian filenames represented? (unless they fit within the BMP, but the Han characters alone are 70k in number). OK, so much for that idea. hm, that's a copout (I bet some people would be surprised that they can't specify savegame names with special characters). However, even that doesn't yet solve the problem (see below). Correct: the user enters characters in a known encoding (wchar_t, i.e. basically just the Unicode codepoint); we can't just paste those bytes onto those received from the system. If the system returned latin-1 and we tack UTF8 onto that, we have screwed the pooch. Maybe Linux is successful at sticking its head in the sand in this regard, but we don't only receive and pass along pathnames unchanged. User-specified paths are one example; there are also cases where we have to take an existing data path, strip off some parts, and add some characters (specifying the cache directory). If the encoding of any of these path sources happens to be different, we are definitely screwed. I have a definite sense of unease about distinct path vs wpath - not only would we need new string processing functions (more than just strcpy, see above), but moving towards less certainty (i.e. not knowing the encoding at all) seems a step in the wrong direction. It looks like the only real source of trouble is Linux returning either latin-1 or UTF8. How about we add checks to wstring_from_utf8? If the bytes are actually Latin-1, we'd notice because it's hopefully (proof needed) invalid UTF8. In that case, we can assume it's latin-1 (or maybe even the current locale) and copy that to wchar_t without too much trouble.

hm, double fail - the patch apparently didn't get attached (probably due to .patch extension not being whitelisted by the forum?) and didn't include a newly created file, so I can't commit *sigh* Will try again tomorrow evening. Oh no :/ Come to think of it, I believe we have the same problem on Windows (NTFS not using UTF8). Is it feasible to determine the filesystem (and its encoding) on which our files are stored? (we'd have to do this for every mount point, but still) hm, sounds at least partially possible - however, we do need to do some string processing (not only appending blobs of characters, but also extracting the name of the mod / digging through pathnames to find out where to store the cached XMB). I have a bad feeling about this, at some point we'll need to understand the paths and their encodings. ATM I am holding out hope for the fs-encoding-query option (we already do something similar in wfilesystem.cpp). Right, at least this change moves us in that direction.

Thanks for your help! I see the problem: we're passing the path to Windows as UTF8, but Windows only understands CP1252. In the past, the problem wasn't noticed because all required characters were covered by CP1252. Looks like the previous design (providing a POSIX interface and doing some magic behind the scenes so that it also works on Windows) wasn't so great. I suppose we could go from wpath to UTF8 for the opendir wrapper and then back to UCS-2 for Windows' FindNextFileW, but that's a bit ugly/inefficient/risky. Instead, I've now added a layer of functions that dispatch to the Unicode variants on Windows and only convert to UTF8 on Unix - we'll now call wopendir instead of opendir, etc. I will commit this later tonight (caveat: Unix version only compiles, can't test); if you'd like to try beforehand, the patch is attached. wrap_oal is part of the OpenAL sound library. There may be any number of causes - the first thing to check is whether your sound drivers are up to date. What hardware do you have? (see binaries/logs/system_info.txt) In the meantime, we have yet another trick that will hopefully save the day: if you launch with -quickstart, sound will be disabled. (Caveat: the writing of system_info.txt will also be skipped, but you should already have that from the first run)

TBH, I thought these changes were harmless enough to not warrant running the tests. Wrong! Looks like the only things that would help are re-education or making the test run automatically again, but that was indeed too slow/inconvenient.

Ooh, interesting, I don't think we've ever seen special characters in the appdata path yet However, it looks like that path is retrieved and passed along as Unicode: wutil:281 SHGetFolderPathW -> boost::filesystem::wpath -> Paths.cpp:46 Could you please check if Paths.cpp:46's 'appdata' is still correct, or whether that's broken already? (Since I can't see anything obvious happening like temporarily converting the path to 8-bit, we need your help to find out where it's going wrong.) As a temporary workaround, you can use the -writableRoot command line switch to get the game to use a subfolder within the SVN tree - provided there are no special characters in that path, it should work then.

Fixed

Sorry, I introduced a bug while porting a function to the new, non-deprecated version. Our permission bitmask definition was setting all bits, but _wsopen_s added a check to ensure no unknown bits are set. I didn't notice because all files apparently exist already, thus skipping the file creation logic. Fixed now.

You could try running a gcc build under cygwin, but the VC route is much easier and the preferred way of doing things on Windows. I'm confident that the trouble you are seeing is due to outdated intermediate files after our JS library upgrade, and will go away as soon as you do the update-workspaces + full rebuild as mentioned above. Sorry, I don't know about that (am using VC2005 and 2008 Pro), but that's the first time I've heard of a countdown or trial mode. We'll get it working!

Excellent!

The same thing happened to me - updating the workspace and doing a clean build (or at least regenerating the PCH) fixed it here. As to the runtime error, could you please break on file_system.cpp!CreateDirectories and see what path is the culprit?

my bad, the recent changes weren't 64-bit clean. Fixed now. All graphics-related messages indicate your graphics card is probably too old. A GF3 or better ought to do it. But black? No, it should be a non-fancy looking shade of blue Do you see any other messages, possible in the main log?

Thanks for the info! -1 is an unhealthy amount of sources in use I figure that can happen due to a race condition, or due to a double-free. Both issues are hopefully fixed (or at least result in a debug_assert(0)) in the latest revision. Can you please test again?

Thanks for the report. Could you please add a printf before line 545 of snd_mgr.cpp to display the values of al_src_used and al_src_allocated? That would help understand the cause.