Jump to content

Working On Upgrade To Premake4


Yves
 Share

Recommended Posts

Hi

I've spent the last few days converting the existing premake3 lua-scripts to the premake4 syntax. I'll still need some more time to finish and test everything, but it looks like I'm on the right way. :D

I just wanted to inform the people here, because it would be a waste if someone else worked on that too.

It could still take some weeks or even months because my vacations are over now, but I guess this doesn't hurry.

So... everyone keep it going and hopefully we'll soon be able to take advantage of premake4 :)

EDIT:

I've released of first version of the updated scripts in this post.

Edited by Yves
Link to comment
Share on other sites

  • Replies 108
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Posted Images

Splendid, will this (help with) supporting Xcode on the Mac in any way? Would be nice to make things easier for people using that platform both for the sake of Mac programmers and other developers using Mac, so if that's the case you can rest assured your work will be even more appreciated :)

Link to comment
Share on other sites

Won't promise anything, but ...

Premake is a build configuration tool. Describe your C, C++, or C# software project using a simple, easy to read syntax and let Premake generate the project files for:

* Microsoft Visual Studio 2002, 2003, 2005, and 2008, including the Express editions

* GNU Make, including Cygwin and MinGW

* Apple Xcode

* Code::Blocks

* CodeLite

* IC#Code SharpDevelop

* MonoDevelop

Link to comment
Share on other sites

A short status report:

More or less the whole premake-script is converted to permake4 syntax in a "quick and dirty" style.

My approach is getting it working quickly and improving it as soon as everything works. I didn't change most of the concepts used in the original scripts even though some things could probably be solved different (and better) with the new premake4 features.

At the moment I'm working on the test generation. I'm trying to implement it using prebuildcommands, which already works except with parallel builds. Premake has a bug with parallel builds and make (e.g. make -j5). In parallel builds, the compiling is started at the same time the prebuildcommands are run, which results in missing test sourcefiles during the first build-process.

My goal was not to patch the premake sources, but it looks like I'll have to fix this. In my opinion this is a bug in premake and not just a problem of how I'm trying to use it. Prebuildcommands should always be called before the build-process and not during the build-process. Maybe I can get a fix upstream so we won't have to use patched sources in the future.

@janwas

Thank you for offering your support! I guess there will be some things to discuss after I get the first version working.

Link to comment
Share on other sites

  • 2 weeks later...

I just wondered if there's any need for the .d files created by make/gcc.

At first view it looks like they aren't needed. Dependencies are recreated using premake which is invoked by update-workspaces.sh.

The -MF flag for creating dependencies is appended by premake, but the customized premake code used for .asm files tries to create a .d file too. Because of a little typo it doesn't create it properly, so I guess nobody needed it.

What do you think? Should I remove it in the premake4 scripts(if not needed) or should we fix it now(if needed)?

I'm talking about these lines in gnu_cpp.c:

				if (!g_verbose)
strcat(g_buffer, "@");

strcat(g_buffer, "nasm "); strcat(g_buffer, opts);
strcat(g_buffer, " -i"); strcat(g_buffer,input_dir );
strcat(g_buffer, " -M -o $@ $< >$(OBJDIR)/$(<F:%%.asm=%.d)\n");

There's a percent-sign too much.

Link to comment
Share on other sites

Hmm, I'm not quite sure what you mean. Premake3 doesn't output .d files by itself, so it has to get GCC to do it via -MF (so that incremental rebuilds work correctly). Do you mean Premake4 is different for C++ and computes all the inter-file dependencies by itself? (I'd be surprised since I don't really see how it's possible to do that correctly, and incremental rebuilds would fail if you added an #include to a file (which is very common) and then rebuilt without running update-workspaces again.)

About .asm files: Good catch! (Though is it one % too much or too few? The line for .c files says "$(<F:%%.c=%%.d)" but I don't know if that's right or wrong). The .asm files are edited very rarely, which is probably why nobody noticed, but it would be nice to do it correctly and generate the .d files if it's not much extra work.

Link to comment
Share on other sites

Sorry I got that wrong. This line is the missing piece:

-include $(OBJECTS:%.o=%.d)

I don't have much experience with make and gcc and didn't catch how these files are used.

Now I understand.

About the other thing I'm a bit confused. I've tested the .asm-part with both two and one percent sign.

It only works with one. I don't know why there are two here: "$(<F:%%.c=%%.d)".

EDIT: I guess the double percent sign was made with character escaping in mind, but it's not needed there.

At the moment, there aren't any .c files used in the build-process and the premake-scripts don't even search for *.c.

In my opinion this part can be completely removed or should be corrected with one percent sign instead of two (even if it doesn't matter at the moment).

			else if (matches(ext, ".c"))
strcat(g_buffer, "$(CC) $(CFLAGS) -MF $(OBJDIR)/$(<F:%%.c=%%.d) -MT $@ -o $@ -c $<\n");

Premake 3.7 does it this way in the original version:

			else if (matches(ext, ".c"))

strcat(g_buffer, "$(CC) $(CFLAGS) -o \"$@\" -c \"$<\"\n");

Edited by Yves
Link to comment
Share on other sites

For clarity and convenience I've created a patch of the suggested changes.

The dependencies of objectfiles created by nasm are now written to .d files correctly instead of a file with .asm ending. I didn't test the part with .c files because there are no .c files at the moment, but I'm quite sure double percent signs are wrong there too. The script uses the same functions to write these strings to the makefile.

Feel free to apply this patch... or just wait for me to finish the update to premake4 :).

Patch.txt

Edited by Yves
Link to comment
Share on other sites

I've just executed the first build of 0ad created using premake4 and make. (y)

Unfortunately there was no way around extending premake. It's a bit easier with premake4 than it used to be with premake3 because nearly all the logic is now in lua-scripts instead of c-code and because some features now work out of the box.

I had to customize premake for meeting the following requirements:

- Using the output of pkg-config and sdl-config for library dependency input

- Integrating assembler-code using nasm assembler

- Using cxxtestgen

- Using special linker options to work around circular references (-Xlinker --start-group $(LDDEPS) -Xlinker --end-group $(LIBS))

The next step will be getting visual studio solution/project creation working (at least 2005, 2008 and 2010 I guess) and testing it on a mac (make-projects first, xcode later).

I hope I can get the required hardware and software for testing...

Link to comment
Share on other sites

Sounds like great progress (y)

In the long term, it'd be nice if the changes we need were merged into the standard Premake distribution - the main developer has seemed interested in extending Premake based on real use cases, so hopefully that should be possible. But our main priority is just to have it work for us, and if we have to distribute a customised version of Premake then that's fine for now (and better than waiting months for a new upstream release).

I don't think we really need to keep VS2005 compatibility, but we should support 2008 and 2010. But I think the 2005 format is almost identical to 2008, so if it's easy to support both then that'd be worthwhile.

If you need help with testing, I have access to various environments and other people here can probably help too, so you could upload your code somewhere (attach here or on a Trac ticket or whatever) for people to try out. (Also it'd be nice for us to have a copy of the work-in-progress in case you discover later that you don't have time to complete it, so someone else could finish it off - hopefully that won't happen but it's good to be safe :P)

Link to comment
Share on other sites

It's a bit too much "work in progress" at the moment and there are stupid "TODO" comments everywhere, some in german and some in bad english (y)

I'd prefer to wait with a first release until visual studio is supported.

Something else...

	if OS == "windows" or not options["without-pch"] then
package.pchheader = header -- "precompiled.h"
package.pchsource = source -- "precompiled.cpp"
...

It looks like on windows the option "without-pch" will not disable precompiled headers.. does anyone know the reason for that?

Link to comment
Share on other sites

I expect that's just because there was no good reason to disable PCH on Windows, and supporting it would mean more build configurations to test and fix, so it's easier to consistently require PCH. (GCC PCH can cause problems, at least with old versions or certain external tools, so it was worth supporting without-pch for GCC.)

Link to comment
Share on other sites

  • 2 weeks later...

Today/yesterday I've finished the vs2010 scripts. Vs2005 and vs2008 aren't working yet, because they have a different syntax for their project files.

During the last week, I've been looking a bit closer at premake's development. The good thing is that there's quite fast progress.

On the other hand, it means that the source is changing very frequently and it will be hard to get something upstream.

Porting changes from the current 4.3 stable release to the development release and making them "final" would be a lot of work and requires deep knowledge of premake's internals and the different IDEs and their build-systems.

The main developer has announced that he is trying to make contributions easier, mainly by creating a more unique code-structure... we'll see.

For now I'm NOT focusing on getting changes upstream.

Maybe I can give it a try again after premake4.3 is integrated and premake4.4 is released.

Good night... :)

Link to comment
Share on other sites

Good news.

About pushing stuff upstream: one upside of the rapid development is that others might have the same need, and it might get taken care of automagically ;) So waiting sounds good; as Philip says, living with a custom version of Premake is fine for the mid-term at the very least.

As for VC2005 support, I agree it's no longer needed, but IIRC the only difference is the version number at the top of the XML files.

I was recently suggesting to Philip that we only support VC2010 for simplicity (one big upside is avoiding the .manifest hell). Since the hwdetect framework now also reports the compiler used, we'll soon see how many people still need VC2008.

Link to comment
Share on other sites

  • 1 month later...

I've been working on xcode support the last weekends, but encountered a lot of difficulties.

It was the first time I've really tried working with MacOS, which didn't make it easier. One of the difficulties was setting up a VM with an acceptable low frequency of kernel panics :D.

After some kernel patches, manually entered bus-ratio-settings and other ugly stuff, it works more or less, but It's still a pain to use.

Make works, but xcode support is a bit more difficult. There were problems with architecture detection, pgk-config and sdl-config, assembling, linking, bugs in the code that got fixed in newer revisions and more.

Currently I have a working xcode project and I was able to run 0ad. I've made many changes to the project manually and there's still work to be done until these changes are all integrated into premake.

I keep working on it and hopefully there will be more progress in my vacations in the week after next week.

Link to comment
Share on other sites

Ah, good thing I didn't start working on this, too :D

I'm using Visual C++ 2010, and with our current version of premake I have to convert 2008 projects every time a source file is added or removed (requires several steps and 2008 projects seem to be broken anyway). That was enough for me to at least look at the new premake, but as the structure is totally different I didn't pursue it. It's great that someone else has! This will make a huge difference in ease of development on different platforms.

Link to comment
Share on other sites

  • 2 weeks later...

... a detail:

What do you think about changing the shebang of update-workspaces.sh from #!/bin/sh to #!/bin/bash?

I've found out that some of the arch-detection problems could be related to the missing HOSTTYPE variable.

According to the manpage of bash this variable is defined by bash.

On my Ubuntu 10.10 system /bin/sh points to /bin/dash which does not define HOSTTYPE.

Often /bin/sh is a link to bash anyway.

On my Snow Leopard installation both /bin/sh and /bin/bash are the same version of bash.

Link to comment
Share on other sites

If the architecture detection is currently only a problem on OS X (is that the case?), then using /bin/bash won't make a difference (since OS X always uses bash) and there's a danger it might introduce problems on other platforms that currently work, so it'd be safer not to change it unless the change fixes a real problem.

(Using HOSTTYPE is a terrible hack anyway - probably what we ought to do is something like autoconf (maybe actually use autoconf) to detect what the compiler is really targeting.)

Link to comment
Share on other sites

Currently this situation only exists on my Ubuntu and other systems where /bin/sh points to a shell that does not define HOSTTYPE.

It isn't really a problem because the check with "gcc -dumpmachine" works on my system, but could be a problem if both checks fail (happens rarely I guess).

There's a new function in Premake 4.4(not stable yet) that could be interesting:

http://industriousone.com/osis64bit

Currently architecture detections is nothing more than choosing between 32 and 64 bit, so if it's reliable we could replace a lot of code with calling os.is64bit() in the future.

I also had a look at "platforms" introduced in premake 4.1 but currently I think that's not needed.

Maybe I'll look at it later when premake 4.4 is released, but probably it's not worth the efforts.

http://industriousone.com/platforms

Edit:

gcc -dumpversion won't work on the mac, but currently it looks like HOSTTYPE-detection will work.

Edited by Yves
Link to comment
Share on other sites

It says "The os.is64bit function determines if the host is using a 64-bit processor" which doesn't sound like the right idea - some users run 64-bit kernels on 64-bit processors but with 32-bit userspace, so the game needs to be compiled in 32-bit mode. (Looks like that's what this change was for). I don't know whether that means Premake's code is doing the wrong thing or just that the documentation is imprecise, though.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share


×
×
  • Create New...