Build environment and deployment on the Mac

[Sonarpulse]

This reminds me, might 0ad ever move to git? track supports it, and IMO the way development is currently going, with the new sound system and this, would be better supported by it.

Edited May 29, 2012 by Sonarpulse

[k776]

I've pushed for Git since I joined the team :-) But we autobuild the pyrogenesis binary into the repo, along with many other DLLs. Git would store a copy of each binary per change, which would very quickly explode the Repo size :-( The Git mirror excludes the pyrogenesis binaries, and it's still nearly 2x the size of SVN checkout (though Git pulls the whole history, not just the current revision, so it's not too bad for 10 years of development :-) )

Edit: So bottomline: I want it to as well, but unless we find a good way to store binaries, it's not likely to happen anytime soon. Meanwhile though, we've just got another developer who moved from SVN to Git (making the count 4 out of the 7 regular committers), so at least we're slowly moving away.

[Sonarpulse]

Does trac handle the automatic binary compilation? maybe trac+plus git can be rigged up to do something separate from the git repository itself. Does SVN store the binaries' history too and it just doesn't matter because only the working copy is downloaded? or is the binary history not stored at all.

Also, does anyone by chance know if Apache's mirror setup (https://github.com/apache) is something any github organization can do? or something they paid for.

Edited May 29, 2012 by Sonarpulse

[k776]

The auto builds are managed by a limited access control script (starts an Amazon EC2 server, builds, and does an 'svn commit'). As far as I know, you're correct (SVN stores all binaries but only one is downloaded (HEAD)). I thought about using Github downloads API, but Philip doesn't want to split the binary from the sourcecode cause then the Art people won't be able to keep up to date by a simple 'svn up' type of command.

[Ykkrosh]

I thought about using Github downloads API, but Philip doesn't want to split the binary from the sourcecode cause then the Art people won't be able to keep up to date by a simple 'svn up' type of command.

Yeah, I think it's valuable to have a single step for "make my game up-to-date", since it saves effort and reduces the risk of people reporting bugs due to mismatched file versions. Also it's valuable for programmers to be able to jump to a certain point in history and get the matching source and executables, to debug minidumps from crashes reported by users. That probably could be done with some custom tools on top of Git, but someone would need to make those tools.

(There's other difficulties with moving to Git, like dealing with non-public files (data/mods/internal/ etc) and whatever else I've complained about before, so I think more work is needed before migrating would be a net improvement.)

[fabio]

Well, the non-public files are not so non-public, BTW: http://trac.wildfire...a/mods/internal

[historic_bruno]

Edit: So bottomline: I want it to as well, but unless we find a good way to store binaries, it's not likely to happen anytime soon. Meanwhile though, we've just got another developer who moved from SVN to Git (making the count 4 out of the 7 regular committers), so at least we're slowly moving away.

I wouldn't say I've moved to Git yet On Windows, I'm having nightmares with line endings with no working solution yet (the "core.autocrlf" option doesn't seem to work as expected) - such as files randomly showing as modified with no way to revert them, which has dampened my enthusiasm somewhat. Considering how many of our developers use Windows and I've not had similar problems with SVN, that seems a significant obstacle to switching to Git. I'm afraid to touch it, for fear there will be hundreds or thousands of newly modified files that no amount of resetting, cleaning, or removing will fix.

Edit: needed a different client, switched from Github for Windows to TortoiseGit.

[Sonarpulse]

Yeah, I kinda gave up on dealing with line endings as I use the same repositories in a duel boot setup, meaning that while it might commit right the thing never knows what to check out.

On the other hand I've never had much of a problem with then, as most everything I use seems to be able to handle either line ending scheme just fine.

[historic_bruno]

Has anyone got Mountain Lion yet? I'm curious if the 0 A.D. bundle will work on it

[Wijitmaker]

Can trac be integrated with Git?

[wraitii]

@HIstoricBruno: not yet, but I'll upgrade my iMac on 10.6.8 to 10.8 quite fast is no major problem is revealed.

[zoot]

Can trac be integrated with Git?

Yes, on some basic level.

[Sonarpulse]

Full support since march first this year.

http://trac.edgewall.org/wiki/TracGit

http://trac.edgewall.org/wiki/VersionControlSystem

http://trac.edgewall.org/milestone/1.0

[wraitii]

I've just installed Mountain Lion, will report about SVN buildabilty once I get the latest X-Code.

[wraitii]

Okay, so I've successfully built and run 0 A.D. on OS X Mountain Lion, compiling with Xcode's 4.4 LLVM 4.0 compiler (ie not GDB). Apart from a few "updates" that Xcode did automatically on each project, I did not encounter problems.

However, the fullscreen option bugs, and results to the game reverting to windowed mode but with issues: it's stuck in the bottom left corner and i have no actual window to speak of (no title bar).

Edit: ActorEditor fails during linking though, it's apparently linked to wxwidgets.

Edited July 27, 2012 by wraitii

[Loki1950]

Edit: ActorEditor fails during linking though, it's apparently linked to wxwidgets.

That maybe related to OpenGL getting some linking errors myself on Win7 with VCE2010.

Enjoy the Choice

[historic_bruno]

Okay, so I've successfully built and run 0 A.D. on OS X Mountain Lion, compiling with Xcode's 4.4 LLVM 4.0 compiler (ie not GDB). Apart from a few "updates" that Xcode did automatically on each project, I did not encounter problems.

However, the fullscreen option bugs, and results to the game reverting to windowed mode but with issues: it's stuck in the bottom left corner and i have no actual window to speak of (no title bar).

Edit: ActorEditor fails during linking though, it's apparently linked to wxwidgets.

What are you using for the dependencies: MacPorts, Homebrew or other? Maybe there are some updates available, or we may have to wait as we did for SDL supporting Lion.

Can you verify if the Alpha 10 bundle works? Preferably fullscreen and also Atlas. Then we'll know what if any changes need to be made for ML support.

[wraitii]

I'll check that. I used macports, I'll check for updates, and report back.

Edited July 27, 2012 by wraitii

[wraitii]

Okay, Alpha 10 bundle seems to work correctly in fullscreen, even when I start the executable directly. The scenario editor requires X11, which is no longer bundled with OS X Mountain Lion, so that might be a problem. However, there's XQuartz that's fairly easy to download and that fixes the problem.

Still can't compile the scenario editor myself or have the fullscreen work, though, no particular idea why. The problem on compiling the scenario editor seems linked with some functions not being available in the proper architecture, for some reason.

Edited July 28, 2012 by wraitii

[wraitii]

I've gotten a problem this morning in the game with libpng, so I've updated everything in macports (took a good 2 hours), tried to compile: failure. Boost filesystem has been updated to version 3, so I had to change a define and add some ".wstring()" where I got errors (iCmpAIManager, line 56 and TextureManager, lines 428/430. I thinks that's it). Then I had to manually add the linking flag "-lboost_filesystem-mt" and it compiled (I was getting linking errors) and started. it worked.

The ActorEditor also compiles cleanly, and macports updated some wx stuff so I guess that fixed it. I however get a fatal error at start when tried to actually start the actorEditor. Starting the scenario editor from the executable works.

So basically, update everything in macports, fix a few linking troubles/small errors, and it works on 10.8

Do note: with the new security stuffs in OS X Mountain Lion, the system will refuse to start Apps that are not from the mac App Store or identified developers (using code signing in Xcode and probably some stuffs on Apple's side). You might want to consider checking that out, and "code signing" the mac bundle. There's probably some documentation on Apple's website.

(This setting can be changed to allow any application to start, but "most people" will probably not change it).

Edited July 28, 2012 by wraitii

[historic_bruno]

Okay, Alpha 10 bundle seems to work correctly in fullscreen, even when I start the executable directly. The scenario editor requires X11, which is no longer bundled with OS X Mountain Lion, so that might be a problem. However, there's XQuartz that's fairly easy to download and that fixes the problem.

Hmm, that's not ideal and unintended. There shouldn't be any X11 dependencies But now that I do an "otool -L libAtlasUI.dylib" I notice: /usr/X11/lib/libX11.6.dylib is listed, it slipped under my radar somehow. Will need to look and see if the offending lib can be configured without X11 (everything should use native APIs instead). Requiring XQuartz would be a last resort. Thanks for testing!

Still can't compile the scenario editor myself or have the fullscreen work, though, no particular idea why. The problem on compiling the scenario editor seems linked with some functions not being available in the proper architecture, for some reason.

This is why MacPorts is evil and must be replaced I'm working on that. Meanwhile, maybe it was a broken port or some optional wxWidgets features weren't enabled (GLCanvas comes to mind).

I've gotten a problem this morning in the game with libpng, so I've updated everything in macports (took a good 2 hours), tried to compile: failure. Boost filesystem has been updated to version 3, so I had to change a define and add some ".wstring()" where I got errors (iCmpAIManager, line 56 and TextureManager, lines 428/430. I thinks that's it). Then I had to manually add the linking flag "-lboost_filesystem-mt" and it compiled (I was getting linking errors) and started. it worked.

This is a known problem: Boost 1.50 breaks the game. But if you've gotten it to build, please attach a patch with your changes. I haven't gotten around to messing with 1.50 yet.

The ActorEditor also compiles cleanly, and macports updated some wx stuff so I guess that fixed it. I however get a fatal error at start when tried to actually start the actorEditor. Starting the scenario editor from the executable works.

Actor Editor is broken on OS X anyway And on *nix it doesn't save all its changes...

Do note: with the new security stuffs in OS X Mountain Lion, the system will refuse to start Apps that are not from the mac App Store or identified developers (using code signing in Xcode and probably some stuffs on Apple's side). You might want to consider checking that out, and "code signing" the mac bundle. There's probably some documentation on Apple's website.

(This setting can be changed to allow any application to start, but "most people" will probably not change it).

Agreed, but the exact process isn't clear to me and I really haven't had any time to look into it. Probably the required certificate(s) are something to be managed by Wildfire Games and need coordination to set up. So users will have to manually enable the 0 A.D. bundle for now, if they use Gatekeeper in its default state. Short term I'll be happy to get the bundle working on a clean Mountain Lion system (no XQuartz, etc.).

There's a lot of useful info on code signing, starting here: https://developer.apple.com/library/mac/documentation/security/Conceptual/CodeSigningGuide/Introduction/Introduction.html#//apple_ref/doc/uid/TP40005929-CH1-SW1

[historic_bruno]

There shouldn't be any X11 dependencies

Found the culprit, there is a potentially useless linking of X11 for Atlas and it even tries on Windows builds! http://trac.wildfiregames.com/browser/ps/trunk/build/premake/premake4.lua?rev=12198#L939 I'll fix that in SVN. Also welcome to the team wraitii

[wraitii]

Okay, there's a twist: the update-workspaces script won't work because spidermonkey says the gcc compiler isn't working... Which may be true, Apple doesn't ship gcc on Mountain Lion, so that may have messed up a few stuffs. Apple now ships clang...

Trying to get around that for now. Weirdly, Fcollada does compile...

Edit: okay so everything but spidermonkey compiles. I'm guessing it's because I had gcc-4.2 previously installed.

I'm going to see if I can't change the checking, since it's obviously wrong.

[wraitii]

Allright, I updated my SVN, cleaned workspaces. So, notwithstanding the crash with spidermonkey, which I simply prevented from recompiling, update-workspaces works fine.

Here's my report for the compilation.

Start Xcode and the pyrogenesis project: "edit scheme" to put the build configuration to "Release" on every project (activates -O3, for some reason Xcode 4.4 thinks debug is the base configuration for "running").

First compilation: mocks_real fails. Error is invalid setting of BOOST_FILESYSTEM_VERSION.

Fix: go to source/lib/pch/pch_boost.h, line 28, #define BOOST_FILESYSTEM_VERSION 3

Recompile.

Simulation2 fails. Error is "ICmpAIManager.cpp:56:16: No viable conversion from 'const boost::filesystem::path' to 'std::wstring' (aka 'basic_string<wchar_t>')"

Fix: change line 56 of ICmpAIManager.cpp to std::wstring dirname = (*it).wstring();

Recompile.

Graphics fails. Two errors. the first is "TextureManager.cpp:428:10: Invalid operands to binary expression ('VfsPath' (aka 'Path') and 'const boost::filesystem::path')", the second is the same as in Simulation2.

Fix: in TextureManager.cpp, line 428, change to p = p / (*it).wstring();

Line 430, change to return m_TextureConverter.ComputeSettings(srcPath.leaf().wstring(), files);

Recompile.

Linking failed, mach-o linking errors that all target boost::filesystem

Fix: in the project settings, add to "Other Linking Flags / Release" : "-lboost_filesystem-mt"

Recompile. Build succeeded. 0 A.D. is up and running.

Now trying to compile ActorEditor (and hence also the scenario editor as a side effect).

"Edit scheme" to put the build configuration to "Release" (same issue).

First compilation: build succeeded. The actor editor starts properly though it's bugged on OS X anyway.

Pyrogenesis executable and the scenario editor are up and running.

So, apart from the spidermonkey problem (which is not too bad if precompiled libraries can be made available), it's all pretty straightforward, and it seems to work with the boost update.

I'm not entirely sure that the fix for line 430 of texturemanager.cpp doesn't break something, but it didn't seem so in-game.

[historic_bruno]

Okay, there's a twist: the update-workspaces script won't work because spidermonkey says the gcc compiler isn't working... Which may be true, Apple doesn't ship gcc on Mountain Lion, so that may have messed up a few stuffs. Apple now ships clang...

Trying to get around that for now. Weirdly, Fcollada does compile...

Edit: okay so everything but spidermonkey compiles. I'm guessing it's because I had gcc-4.2 previously installed.

I'm going to see if I can't change the checking, since it's obviously wrong.

Make sure you install the latest command line tools for Xcode (it's probably best if you uninstall any previous tools, including those packaged with old Xcode), then you'll have a GCC front end for LLVM which is the compiler Apple prefers now. To get SpiderMonkey to compile, you'll likely want to define environment variables CC=gcc and CXX=g++, as the config script expects gcc-4.2 by default.