Issues with Building on Mac M1 - Spidermonkey - "ld: Undefined symbols: _main, referenced from: <initial-undefines>"

[Ben Phegan]

Hi,

I am trying to get 0-AD building on an M1 Mac running Ventura 13.6.  I have had to apply various patches (D2573, D5092, D5093) as well as two additional patches, one for vorbis (no issue raised yet but related to compiler flags) and Spidermonkey `ld` issue (current version on Mac does not have --version as an ld parameter).

However, despite all this I am still getting a compilation failure on Spidermonkey.  It is probably some local variance that I have not accounted for.

The issue is when running `./build-osx-libs.sh` when it gets to Spidermonkey.  As part of the configure I get this:

checking for 64-bit OS... yes
checking for linker... 
DEBUG: Executing: `/usr/bin/clang -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -std=gnu99 --target=arm64-apple-darwin -Wl,-v`
ERROR: Failed to find a linker
ERROR: Error building spidermonkey

Previous to my patch it was failing due to passing `-Wl,--version` which is incompatible.  However now, if I rerun the command with `-v` I get the following:

➜  osx git:(m1) ✗ /usr/bin/clang -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -std=gnu99 --target=arm64-apple-darwin -Wl,-v -v
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
 "/Library/Developer/CommandLineTools/usr/bin/ld" -demangle -lto_library /Library/Developer/CommandLineTools/usr/lib/libLTO.dylib -dynamic -arch arm64 -platform_version macos 13.0.0 14.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o a.out -v -lSystem /Library/Developer/CommandLineTools/usr/lib/clang/15.0.0/lib/darwin/libclang_rt.osx.a
@(#)PROGRAM:ld  PROJECT:dyld-1015.7
BUILD 18:48:48 Aug 22 2023
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
will use ld-classic for: armv6 armv7 armv7s arm64_32 i386 armv6m armv7k armv7m armv7em
LTO support using: LLVM version 15.0.0 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 15.0.0 (tapi-1500.0.12.3)
Library search paths:
Framework search paths:
ld: Undefined symbols:
  _main, referenced from:
      <initial-undefines>

I have not been able to find many hits on this error that seem relevant.  My guess is it is something to do with my compiler/SDK install.  Does anyone have a working M1 build machine that would be happy to help with troubleshooting?

Thanks!

Ben

Edited September 26, 2023 by Ben Phegan

[phosit]

IIRC @wraitii and @Langbart use macs. But i don't know if they use M1.

[Ben Phegan]

Thanks @phosit hopefully they can help.  I noticed that there are Dockerfiles that are used for Jenkins, but there is no specific singular Dockerfile approach to developing under Linux.  Are most of the devs using Windows?  I could transition to a Linux dev process and use Docker if there was a reasonable cross compilation approach (not sure that this is currently possible though).

[Ben Phegan]

Just as an update to this, I have managed to get a lot of the build working via a lot of patching.  The sticking point is Spidermonkey.  I am using the following config:

MacOS: 14.0
Xcode: 15.0
Hardware: MacBookPro18,1

I have also tried building on an Intel Mac running 12.7.  Neither works out of the box, and so far neither are successful at all.

To get any further than the error above an an M1 requires linking with `lld` instead of the default `ld`, installing the LLVM package via `brew install llvm` and a bunch of mods to various build files.  It is just not supported as far as I can tell.

Is there anyone who _successfully_ builds on Mac _currently_?  If so, would love to know what their config looks like?  At the moment, I do not want to go too far down the path of fixing these issues as to be honest I cannot see a reasonable way forward getting patches in place if no-one is building on Mac.

[Ben Phegan]

Is 0-AD still buildable on modern Mac software/hardware?  Does anyone have this working?  

I would love to replicate, and I am happy to update documentation/build scripts/patches if necessary.

[vladislavbelov]

49 minutes ago, Ben Phegan said:

Is 0-AD still buildable on modern Mac software/hardware?

Currently we build with an older macOS & SDK because of compatibility (also https://trac.wildfiregames.com/ticket/6193). Ideally it should be compiled on modern macOS, but I'm not sure it works for newest updates.

49 minutes ago, Ben Phegan said:

Does anyone have this working?

I'll be able to test it on M1 or M2 in a few weeks.

[Ben Phegan]

@vladislavbelov is the required MacOS/SDK version documented anywhere?  I can always drop a machine back to an early MacOS and SDK to get building, or even just use a VM.  Also, can confirm that it does not work on later MacOS and SDK/toolchain (lots of breakage).  I have a branch locally tracking required changes for when you are looking at this.

[hyperion]

The CI uses 10.12 IIRC, that's also the one used for building the releases. Anything newer should work or it's a serious bug.

 

As for SM you might want to try 102, maybe you'll have more success https://code.wildfiregames.com/D5002

[Ben Phegan]

@hyperion you mean Mac OS X 10.12?  Sierra?  That hasn't had a security patch since 2019 though, right?  I have not tried anything that old, and I can say for sure that the later versions do not work for me out of the box due to changes in the default compilers/linkers.

Will try with the updated SpiderMonkey patch, but so I am clear, what is the future of that patch?  Is it likely to merge?  If so, what sort of timeframe?  I am finding it particularly difficult to track the future state of the project based on the state of the patches/issues.  I am already automating a _bunch_ of patches to get this far, and it is far from ideal.  Some of the patches I am applying are quite old.  Like, three years as a patch diff sitting on an issue.  What is the governance around these?

[hyperion]

@Ben Phegan

SM-102 just became EOL, meaning it won't change anymore and can be used without it changing right under our nose. So there is no fundamental blocker anymore, ofc needs more testing and maybe fixing. So your findings might be of some help pushing it forward.

As for CI, the built binaries they should work on later versions anyway. The issue with newer SDK is it causes issues with opengl, so the new vulkan support is probably the way forward. As for the other build issues, well, not that nice is all I can say, let's hope those patches can get merged sooner rather than later.

[Ben Phegan]

@hyperion - unfortunately it appears that the source files referenced for Spidermonkey in https://code.wildfiregames.com/D5002 no longer exist.  For reference, the build script will attempt to download https://github.com/wraitii/spidermonkey-tarballs/releases/download/mozjs-102.12.0/mozjs-102.12.0.tar.xz which does not exist.  

@wraitii - is there any documentation on how you grab and upload this file?  Is there a way to replace this with a source reference?

[hyperion]

To my knowledge those tarballs are exports of gecko-dev.git and are in essence stripped tarballs of firefox source releases for size reason, the following should work as drop-in.

https://ftp.mozilla.org/pub/firefox/releases/102.12.0esr/source/

[wraitii]

Yeah that's what those are. I guess I just forgot to upload the tarball.

I've done that now, should fix the link.

[vladislavbelov]

On 11/10/2023 at 1:34 AM, Ben Phegan said:

@vladislavbelov is the required MacOS/SDK version documented anywhere?  I can always drop a machine back to an early MacOS and SDK to get building, or even just use a VM.  Also, can confirm that it does not work on later MacOS and SDK/toolchain (lots of breakage).  I have a branch locally tracking required changes for when you are looking at this.

I was able to build with https://code.wildfiregames.com/D5092 and few fixes in premake. I was testing with M2 and Ventura 13.4. I will investigate further.

[xone47]

Is there any reason to not use pre-built libraries (from homebrew) for many of these premake things?

[Stan`]

1 hour ago, xone47 said:

Is there any reason to not use pre-built libraries (from homebrew) for many of these premake things?

IIRC version stability across platforms.

[xone47]

28 minutes ago, Stan&#x60; said:

IIRC version stability across platforms.

The build instructions for linux say "apt install libvorbis-dev" so there would be drift there too?  But in cases where it is required, it looks like homebrew has a way to pull specific versions.  I'll play around with it.

[hyperion]

Well, for Linux only sources are provided and distributions have to build 0ad themself. Also they wouldn't accept bundled deps and so the situation is different than for Mac where the goal is an all-in-one dmg. If you build for yourself fixing the build script to use homebrew packages is obviously possible. At least you get a working build script you can use as base unlike on Windows where the deps are prebuilt.

 

[Stan`]

IIRC @wraitiiused brew at some point but it was a mess. Also this way you can have it on an external hdd without worrying too much about where to find things.

Anyway @hyperion is right the main point is that if a distro decides to update their package and it breaks it's a them problem. If you do on macos and it breaks it's a you one and given the little amount of macos devs it's probably one that will be hard to merge especially if it touches code used by all the other platforms.

Stuff like the SDL has a huge tendency to break things between minor versions ^^

EDIT: licenses too e.g. openssl

[fabio]

@Stan`note that since 3.0 openssl is GPL compatible.