Building a 64-bit pyrogenesis (and deps) for Windows

[vladislavbelov]

10 hours ago, hyperion said:

After a little hack for pkg-config I got tons of issues with undefined stuff. Probably caused by legacy hack after legacy hack. So a first simple suggestion attached as well. Best to get rid of the whole header thou and use the standard instead of custom types throughout the codebase.

"lib/posix/posix_types.h" already contains #include <stdint.h>, so no need to replace. Instead it might be useful to find a reason of the issues.

Also using int*_t (and others *_t) directly has a rare caveat, it might be not defined on some platforms (according to the standard).

[hyperion]

10 hours ago, Stan&#x60; said:

standard not ensuring the siezs of uint8_t

The size is guaranteed, it's not guaranteed to exist thou.

 

8 hours ago, vladislavbelov said:

it might be not defined on some platforms (according to the standard).

If the target doesn't have 8bit bytes, yes. The only examples I'm aware of is hardware in the 70' which for example had 36 bit words, so you end up with either 9,12,18,36 bit bytes.

Conversely if you have a C99 compliant compiler without uint8_t you won't have a type that you can use for u8 either, so you are screwed anyway if you want to port it to such a target unless you make sure the code never makes use of the feature that u8 is actually 8 bit in which case u8 is a bad name anyway.

 

8 hours ago, vladislavbelov said:

"lib/posix/posix_types.h" already contains #include <stdint.h>

Not for windows. There it redirects to wposix which apparently broke mingw for some odd reason .

 

8 hours ago, vladislavbelov said:

so no need to replace

Indeed there are other ways to go about it. But this doesn't change the fact we include dozens of headers where one would suffice just to to create a new header that is potentially only ever useful for compilers supporting c++17 (required by 0ad) but for some obscure reason not C99 (not an explicit requirement for whatever reason).

 

8 hours ago, vladislavbelov said:

Instead it might be useful to find a reason of the issues.

Right, just that I'm faced with to many errors to care for now. Porting might become a major project by itself .

[vladislavbelov]

14 hours ago, hyperion said:

If the target doesn't have 8bit bytes, yes. The only examples I'm aware of is hardware in the 70' which for example had 36 bit words, so you end up with either 9,12,18,36 bit bytes.

Not only, there is a case with <cstdint> (which we also include in some places), which might not define ::*_t, only std::*_t ones. To avoid the question what to include we might use own types. But I don't mind to use std::*_t.

15 hours ago, hyperion said:

Not for windows. There it redirects to wposix which apparently broke mingw for some odd reason

It defines the *_t types by itself. Your issues might mean that it doesn't define all required types.

15 hours ago, hyperion said:

Indeed there are other ways to go about it. But this doesn't change the fact we include dozens of headers where one would suffice just to to create a new header that is potentially only ever useful for compilers supporting c++17 (required by 0ad) but for some obscure reason not C99 (not an explicit requirement for whatever reason).

For sure there is a lot of things to improve with includes and dependencies. Like CStr/CStrW definition.

15 hours ago, hyperion said:

Right, just that I'm faced with to many errors to care for now. Porting might become a major project by itself .

IIRC the biggest and most annoying part for me was building libraries (which I didn't finish), the code itself was pretty easy to build with mingw.

[hyperion]

10 hours ago, vladislavbelov said:

Not only, there is a case with <cstdint> (which we also include in some places), which might not define ::*_t, only std::*_t ones. To avoid the question what to include we might use own types. But I don't mind to use std::*_t.

So you prefer <cstdint> and not importing those types into global namespace? (well the implementation might still do it)

As for avoiding the question, I guess you are qualified to dictate the answer

 

10 hours ago, vladislavbelov said:

t defines the *_t types by itself. Your issues might mean that it doesn't define all required types.

It's indeed the source for this issue and also the cause for breaking the off_t family. The whole header might profit from spring cleaning. C++11 mandates C99 after all.

 

11 hours ago, vladislavbelov said:

IIRC the biggest and most annoying part for me was building libraries (which I didn't finish), the code itself was pretty easy to build with mingw.

Is that 32bit runtime and on windows?

 

On 28/07/2021 at 7:22 AM, Stan&#x60; said:

First one is now public, there was a mistake in the visibility, second one does need an account

Thanks, fully support the effort, pulled it, still need the CHOST hack thou.

[vladislavbelov]

11 hours ago, hyperion said:

So you prefer <cstdint> and not importing those types into global namespace? (well the implementation might still do it)

I prefer to avoid library types (external types, not defined by an application) or "using"s in a global namespace in headers.

11 hours ago, hyperion said:

As for avoiding the question, I guess you are qualified to dictate the answer

I'm not sure that I get what do you mean here  I don't know how to check properly that I'm qualified enough for some task

11 hours ago, hyperion said:

The whole header might profit from spring cleaning.

I'd like to clean up it as well.

11 hours ago, hyperion said:

Is that 32bit runtime and on windows?

Yeah.

 

[s0600204]

I myself have been looking into creating a script akin to the current build-osx-libs.sh file, but for Windows; with the hope that it would lead to us debundling the Win32-specific binaries and headers.

I'm curious to see where other people have got to in their own attempts. Anyone willing to share a link?

 

18 hours ago, hyperion said:

still need the CHOST hack thou.

The "CHOST hack"?

[Stan`]

27 minutes ago, s0600204 said:

The "CHOST hack"?

 

On 27/07/2021 at 11:43 PM, hyperion said:

Building 0ad seems to be a major headache thou. Premake pkgconfig module should check for ${CHOST}-pkg-config first and only fall back to pkg-config if not found like autotools does.

 

27 minutes ago, s0600204 said:

 

I'm curious to see where other people have got to in their own attempts. Anyone willing to share a link?

https://wildfiregames.com/forum/topic/26142-using-conan-to-get-windows-dependencies/?tab=comments#comment-376743 This is as far as I went with Conan, (of course it's now missing libfmt at least)

VCPKG didn't work well last time I tried, I think I couldn't specify the toolset to be the XP one, which was bad. (Now we don't care anymore) I think @Imarok hated it though

We could also switch to CMAKE / MESON if it makes things easier;

And of course there is still the git transition to settle.

 

[smiley]

56 minutes ago, Stan` said:

I think I couldn't specify the toolset to be the XP one

I think it's not possible. Ninja doesn't support those. I personally didn't try using it with anything besides vs2019 with win10 toolkits, so I am not too familiar with that, but I recall reading something like that, not entirely sure.

Edited July 30, 2021 by smiley

[hyperion]

On 30/07/2021 at 9:28 AM, vladislavbelov said:

I'm not sure that I get what do you mean here  I don't know how to check properly that I'm qualified enough for some task

Not in general, but as a project should stick to one way and the nominal lead immediately pinged you ...

 

Btw. wseh is also broken (32bit only) and in wgl.h ...

Quote

// RAGE! Win32 OpenGL headers are full of crap we have to emulate

... doesn't have enough exclamation marks. You only know after experiencing it.

[hyperion]

On 30/07/2021 at 4:34 PM, s0600204 said:

The "CHOST hack"?

As @Stan` quoted. I called it hack here because I patched in the value of CHOST I require to cross compile. No problem reading lua but doing things properly would take time looking up stuff and I'm fast approaching the original estimate of a couple hours, so I settled for the cheap solution for now to be revisited later

 

I love hearing debundling. I'd even throw out the stuff in libraries/source/,  cxxtest and valgrind should be easy, fcollada and nvtt are dead upstream so forking and putting up as fcolladaAD and nvttAD and treating them as external libs should be fine.

[s0600204]

1 hour ago, hyperion said:

I love hearing debundling. I'd even throw out the stuff in libraries/source/

Heh, one thing at a time. (There's also mongoose, tinygettext, and several other things that can be found in source/third_party/)

1 hour ago, hyperion said:

cxxtest and valgrind should be easy

Very easy in the case of valgrind: D3646. Haven't looked into cxxtest at present.

1 hour ago, hyperion said:

fcollada and nvtt are dead upstream so forking

We already have our own fork of fcollada: https://code.wildfiregames.com/source/fcollada/, although it's not pulled in like other external dependencies (and in fact there are some differences between that and the code that is currently bundled within library/source/). (There's also #5907 to take into consideration.)

Personally, I'd like to see FCollada replaced with something that's actually maintained (by another project I mean, not by us). (Ditto with nvtt.) But it's beyond my ken to suggest possible alternatives. And to refactor code to use said alternative.

 

1 hour ago, hyperion said:

On 30/07/2021 at 3:34 PM, s0600204 said:

The "CHOST hack"?

[...] because I patched in the value of CHOST I require to cross compile.

I was a little confused as I haven't had to do that in my own exploratory work. Still not too sure what ${CHOST} is supposed to contain. Then again, I'm largely unfamiliar with autotools, so if it's something specific to that then that might be why.

[hyperion]

2 hours ago, s0600204 said:

I was a little confused as I haven't had to do that in my own exploratory work. Still not too sure what ${CHOST} is supposed to contain. Then again, I'm largely unfamiliar with autotools, so if it's something specific to that then that might be why.

CBUILD, CHOST, CTARGET correspond to the configure args --build --host --target which you might be more familiar with. So I cross compile from linux to windows with CBUILD=x86_64-pc-linux-gnu and CHOST=x86_64-w64-mingw32, and I use x86_64-w64-mingw32-g++, x86_64-w64-mingw32-ar etc as well as x86_64-w64-mingw32-pkg-config.

Not really familiar with premake, so whether to respect CHOST, use PKG_CONFIG=, or a switch like --with-pkg-config= or all of them I'm not sure which is most inline with how premake does things.

 

4 hours ago, s0600204 said:

Very easy in the case of valgrind: D3646. Haven't looked into cxxtest at present.

Pulled, thanks.

[Stan`]

@s0600204 I did. But we can't debundle it cause our version is patched. https://code.wildfiregames.com/D3119

[s0600204]

@Stan` Ah, I'd forgotten about that.

 

13 hours ago, hyperion said:

CBUILD, CHOST, CTARGET correspond to the configure args [...]

That does clarify things, thank you.

[vladislavbelov]

On 31/07/2021 at 10:16 PM, hyperion said:

Not in general, but as a project should stick to one way and the nominal lead immediately pinged you ...

In that way I could say that I'm responsible for the C++ part, yes.

On 31/07/2021 at 10:16 PM, hyperion said:

Btw. wseh is also broken (32bit only) and in wgl.h ...

IIRC wseh might work with minimal changes. About GL, the most important part is to check GL* types (GLuint, GLchar, etc).

[Stan`]

13 minutes ago, vladislavbelov said:

About GL, the most important part is to check GL* types (GLuint, GLchar, etc).

And as you've seen, it wasn't done  

[vladislavbelov]

22 minutes ago, Stan&#x60; said:

And as you've seen, it wasn't done  

I mean when porting to 64bits.

[Stan`]

I mean it's good practise anyway ;D

[vladislavbelov]

3 minutes ago, Stan&#x60; said:

I mean it's good practise anyway ;D

Yep, and I prefer quality over quantity

[hyperion]

wdbg is problematic too, sal.h

/* FIXME: __in macro conflicts with argument names in libstdc++. For this reason,
 * we disable it for C++. This should be fixed in libstdc++ so we can uncomment
 * it in fixed version here. */
#if !defined(__cplusplus) || !defined(__GNUC__)
#define __in
#define __out
#endif

so I threw it away

 

The ancient mongoose copy also needed a minor fixup, now the project compiled, with all extras still missing. Linking is the next step, but I guess lot of changes to premake files are needed. Not yet figured out how to go about it.

[smiley]

48 minutes ago, hyperion said:

wdbg

Tangentially related since this has nothing do with 64 bit builds, but wdbg code is written in a really old win api and has not been updated since it was written more than a decade ago. Better ways to get a backtrace are now available in the win api.

Someone should really refactor those early 2000's code.

[vladislavbelov]

2 hours ago, hyperion said:

The ancient mongoose copy also needed a minor fixup, now the project compiled, with all extras still missing. Linking is the next step, but I guess lot of changes to premake files are needed. Not yet figured out how to go about it.

Are you using --without-* flags?

58 minutes ago, smiley said:

Someone should really refactor those early 2000's code.

On the way partly.

[hyperion]

2 hours ago, vladislavbelov said:

Are you using --without-* flags?

Yes, I know I'm not yet done

 

[Stan`]

1 hour ago, hyperion said:

Yes, I know I'm not yet done

 

I guess he wanted to know which ones

EDIT: It's just me then X)

[vladislavbelov]

54 minutes ago, Stan` said:

I guess he wanted to know which ones

Nah, just recommend to use

1 hour ago, hyperion said:

Yes, I know I'm not yet done

Cool! Best wishes to your work 

UPD. Cool that you use it, not that you're not yet done x)