A "psychic" shader mod; development begins...

[hyperion]

@DanW58

Well, for A24 it's late anyway, probably a matter of hours till release.

A new set of shaders with an incompatible texture stack sounds like a plan to me. Though I suggest to discuss a proposal at least with @vladislavbelov and @wraitii for the technical aspect and with @Stan` as the artist representative, without their agreement there won't be an inclusion. They might have done some work into that direction already or at least might have some plans or requirements. Also there might be additional needs for mods, like glow or transparency. So consulting major modders like @wowgetoffyourcellphone or @The Undying Nephalim (Hyrule Conquest) might be a good idea as well. Anyway easy import/export and intuitive mapping I think are prime concerns. 

I had a deeper look at the result of the shaders in the other thread and there is often an improvement, but sometimes a digression. The reason might be poorly baked ao, interaction of shadows with ao or something else, I can't tell. The most important thing is it introduces a variation of quality in visuals which makes me think it's about as good as doing nothing. Sure this can be fixed by revisiting many assets, but is it worth it considering when there might be a new stack?

Another thing to consider is whether baking ao is still preferable over using some post processing algorithm.

[DanW58]

Well, the new stack is in the future;  this shader is now, and it doesn't just make frequent improvements;  it also makes a big general improvement in the lighting of all scenes, from an arcadish, too much ambient light, half way between 3D and cartoon, and scenes that look more realistic, more colorful.  You can't say that darkening saturated terrains and applying ambient occlusions with correct gains amounts to just "often beneficial".  You also seem to be looking for reasons NOT to adopt the shader, and ignoring any reasons to adopt it.  But in so doing you know you hurt me, and then you have to consider what might happen if I leave.  I have many years of experience in graphics, and not just years of doing what everybody else does, but years of pioneering work.  I've alredy corrected many small mistakes and a big one in the current shader, namely that shadows were being applied to specular reflections.  The version of the shader you are testing doesn't have that fix.  The version you have doesn't have ambient mapping with specular occlusion;  I offered you to test it but you didn't care.  It seems to me I'm being subtly attacked from all sides here.  First I tried to contribute a nice code cleanup of the VectorXD and FixedVectorXD classes, hiding away public members, etc.  Wraitii rejected my patch on the basis of taste;  he doesn't like get() and set() functions.  Now I come up with great improvements to your shaders, and you just look for any absurd fears and excuses to reject my work.

Though, to your credit, this is the story of my entire life.

But this may just be the end of it.  I frankly can't take it anymore and my health is not getting better.  But really, other than for Elon Musk, I think this world belongs to @#$%s.  All they care about is their bloody who's who, who is supposed to brown nose to whom, and such garbage.  And they all hate perfectionists, AND gang up on them.  They all agree behind the scenes, this guy is a perfectionist;  let's get rid of him.

 

 

Edited February 14, 2021 by DanW58

[nifa]

I'm pretty new here as well, but I read across a lot of stuff in the forum, also your posts. As far as I can see the people here have many different interests, there are artists, historians, gamers, programmers and so on, and for many of them it's not their professional field (my field of studying is actually completly unrelated to this :D). I can only speak for myself, but I had a hard time trying to understand what you were writing (due to my lack of knowledge and since it's not my mother tongue), but I like your dedication and I read out of your posts that you seem to have a lot of professional experience, so I'm pretty excited for the results

Anyway, what I'd like to say is that a huge part of the community might not follow you on every step but when you present a final result you will get the recognition for it. So keep it up

 

[hyperion]

First of all I'm not part of the project and don't represent it so you can just ignore me all together if you want. Secondly if you already know that you fixed the issue that I might have encountered no need to get upset as you obviously agree with me that there were and still might be pending issues. I have other things to do so reaction time is bound to be slow,  if you want me to always use your latest changes clone the git repo and push a branch to github or wherever so I can add your repo as a remote. Then there is no need to copy unversioned files from different threads witch might or might not be part of the same effort around and no need to notify me that there are changes in the first place. Also with proper commit messages I already know what got changed since the last test and I can reference a commit hash when I comment. All I can say is the direction is worthwhile however the version I looked at clearly isn't there yet. That there is a need to touch assets is obviously just a guess, I'm happy if you prove me wrong, but if you have to touch them it's a lot of work. With a new stack right after this all might be in vain. This is also not just about vanilla but also all assets in mods, so even if you fix everything in vanilla others still have to fix the rest.

Collaboration is always more about communication than coding, if you are completely averse to this you better stop earlier than later. This has nothing to do with brown nosing, bashing or the like but is just the nature of it.

As for a new stack, me thinks it's a good thing. If undertaken don't expect it to land within days. Planing, discussion, implementation and finally reviewing will take a lot of time. This is far from a one-liner after all and involves the interests of many.

Perfectionism is a prime source of stress. Nothing will ever be perfect. Anyway take care of your health, that's most important.

[Nescio]

2 hours ago, DanW58 said:

[...]

You also seem to be looking for reasons NOT to adopt the shader, and ignoring any reasons to adopt it.  But in so doing you know you hurt me, and then you have to consider what might happen if I leave. 

[...]

It seems to me I'm being subtly attacked from all sides here.

[...]

Though, to your credit, this is the story of my entire life.

But this may just be the end of it.

[...]

And they all hate perfectionists, AND gang up on them.  They all agree behind the scenes, this guy is a perfectionist;  let's get rid of him.

Please don't take things personal.

People read and reply to your posts, which shows they're interested in your work and interact with you. Everyone has a different background and way of expressing themselves; people here are all volunteers with limited time; and most of us aren't team members (the team is actually quite small, and currently very busy with getting the next stable (A24) released), but contributions from others are certainly welcome. Criticisms are part of the process to get things improved. The fact people look at your proposals and give feedback indicates you're appreciated.

Proposals are rarely embraced immediately, therefore it's advisable to:

• be patient;
• try to explain it differently;
• improve it further and address concerns raised by others.

That said, if you want to see something included, you should upload (and maintain) patches to https://code.wildfiregames.com/ In a forum post it's very hard to keep track of everything, go into detail, or review changes.

[wowgetoffyourcellphone]

I am already doing a lot of work on an entirely new terrain set for the game, so the current cartoony look of the terrains will be completely gone by Alpha 25. Example:

 

Spoiler

Alpha 23-24

Spoiler

Alpha 25

If you want to delve further into the development of 0 A.D. and see how your work can be a part of the whole, then join some of the other discussions and take a look where progress if being made elsewhere. Some of the work is being done far away from the main repo. It probably feels pretty chaotic from a professional's point of view, but this is an entirely different kind of development environment than probably what you are used to. Every few years, someone comes onto the forum and tells everyone how things are going to go and runs into a brick wall of frustration and leaves. That doesn't have to be you. We are all interested in the work you are doing, but we are all also ruthlessly honest with our opinions. In a professional environment this kind of brutal honesty could be a detriment to a harmonious office, but here we welcome it. Fragile egos will often burn out and leave. I'm not saying you're fragile, just probably not used to the kind of critique we offer. Again, we are all interested in what you're doing here. Iteration, iteration, iteration, has been WFG's mantra since its inception.  

[Rahman 1441]

12 hours ago, DanW58 said:

12 hours ago, DanW58 said:

Well, the new stack is in the future;  this shader is now, and it doesn't just make frequent improvements;  it also makes a big general improvement in the lighting of all scenes, from an arcadish, too much ambient light, half way between 3D and cartoon, and scenes that look more realistic, more colorful.  You can't say that darkening saturated terrains and applying ambient occlusions with correct gains amounts to just "often beneficial".  You also seem to be looking for reasons NOT to adopt the shader, and ignoring any reasons to adopt it.  But in so doing you know you hurt me, and then you have to consider what might happen if I leave.  I have many years of experience in graphics, and not just years of doing what everybody else does, but years of pioneering work.  I've alredy corrected many small mistakes and a big one in the current shader, namely that shadows were being applied to specular reflections.  The version of the shader you are testing doesn't have that fix.  The version you have doesn't have ambient mapping with specular occlusion;  I offered you to test it but you didn't care.  It seems to me I'm being subtly attacked from all sides here.  First I tried to contribute a nice code cleanup of the VectorXD and FixedVectorXD classes, hiding away public members, etc.  Wraitii rejected my patch on the basis of taste;  he doesn't like get() and set() functions.  Now I come up with great improvements to your shaders, and you just look for any absurd fears and excuses to reject my work.

Though, to your credit, this is the story of my entire life.

But this may just be the end of it.  I frankly can't take it anymore and my health is not getting better.  But really, other than for Elon Musk, I think this world belongs to @#$%s.  All they care about is their bloody who's who, who is supposed to brown nose to whom, and such garbage.  And they all hate perfectionists, AND gang up on them.  They all agree behind the scenes, this guy is a perfectionist;  let's get rid of him.

 

 

 

I am very impressed with the ideas you offer. it can make this game that much better.

[Genava55]

15 hours ago, DanW58 said:

Well, the new stack is in the future;  this shader is now, and it doesn't just make frequent improvements;  it also makes a big general improvement in the lighting of all scenes, from an arcadish, too much ambient light, half way between 3D and cartoon, and scenes that look more realistic, more colorful.  You can't say that darkening saturated terrains and applying ambient occlusions with correct gains amounts to just "often beneficial".  You also seem to be looking for reasons NOT to adopt the shader, and ignoring any reasons to adopt it.  But in so doing you know you hurt me, and then you have to consider what might happen if I leave.  I have many years of experience in graphics, and not just years of doing what everybody else does, but years of pioneering work.  I've alredy corrected many small mistakes and a big one in the current shader, namely that shadows were being applied to specular reflections.  The version of the shader you are testing doesn't have that fix.  The version you have doesn't have ambient mapping with specular occlusion;  I offered you to test it but you didn't care.  It seems to me I'm being subtly attacked from all sides here.  First I tried to contribute a nice code cleanup of the VectorXD and FixedVectorXD classes, hiding away public members, etc.  Wraitii rejected my patch on the basis of taste;  he doesn't like get() and set() functions.  Now I come up with great improvements to your shaders, and you just look for any absurd fears and excuses to reject my work.

Though, to your credit, this is the story of my entire life.

But this may just be the end of it.  I frankly can't take it anymore and my health is not getting better.  But really, other than for Elon Musk, I think this world belongs to @#$%s.  All they care about is their bloody who's who, who is supposed to brown nose to whom, and such garbage.  And they all hate perfectionists, AND gang up on them.  They all agree behind the scenes, this guy is a perfectionist;  let's get rid of him.

 

 

Personally I found your insights interesting. The only thing is that it is really different from what is done currently, from a visual perspective. So it has huge impacts. This is why other people are prudent and skeptical. I am absolutely not competent to give any opinion on this matter, but I really like the final render of your shader. As Nescio said, try to argue differently and to explain your point of view. I understand your frustration but going in a personal and political argument is not helping you.

 

[gameboy]

@DanW58 My friend, let me see the latest progress in your graphic production.

[DanW58]

Thank you all, gentlemen.  I'm recovering my cool, slowly.

But really, all this business of racing against the clock, working all-night'ers to try to make it to an upcoming release (an arbitrary deadline at that;  needed just to implement "crisis management", probably), and then be told the work is not accepted, and for the most ludicrous reasons, such as a few art items with bad ao bakings not looking good... It is NOT FAIR.  And I'm losing respect for Open Source as a result.  Not losing respect for the ideals and values it embodies, but for the culture that's grown around it, where people who work for free are treated worse than paid labor in a mine somewhere in the third world.

You try and "not take it personally" if you were in my shoes.

I'll keep working on this shader, but no longer will concern myself with what wfg do with it.

 

EDIT:

I decided to cancel the psychic shader.

Still working on the metal detecting one.  You could say it is psychic enough;  and it is much better than it was a week ago, as far as avoiding incorrect metal detections.  It is solid enough I think to even expand detections to human skin and green plant material...

At the moment it is far from compiling, however;  but I've made a ton of progress (nothing to do with metal detection at this point;  mostly with fake environment mapping with specular occlusion and many other crazy new features).  As far as a future shader, I can simply remove the metal detection part, and this is good to go.  I'm computing fresnel for non-metallic materials;  plan to detect human skin and greens to give them the right dielectric and specular power for natural gloss.  I don't have index of refraction input, of course, but will fake it for now;  make it some function of the colors.  It will be very easy to rework this shader once a texture stack is agreed upon.

By the way, I was semi-wrong when I said shadows should not affect specular reflections.  A point on a surface that by its orientation would be reflecting the sun, cannot possibly reflect the sun if it is in the shadow of another object;  it is only specular reflections from other directions that are not affected by shadows, namely environment mapping.  So now I correctly have specular highlights modulated by shadowing, but my pseudo-enviro-map NOT affected.

The progress with this shader will not be wasted by added features, by the way.  For example, currently there's no environment mapping, but when there is, having the reflected environment mapping modulated by (however fake) specular occlusion AND ground reflecting occlusion, will be far superior to standard (unoccluded) environment mapping, which looks like crap.   In fact, I'm telling you: 90% of the reason why the graphics industry are working so hard to achieve real time ray-tracing is BECAUSE of the absurd look of environment mapping without specular occlusion --but don't realize it.  Having ANY specular occlusion at all, however ad-hoc, is a million times better than NOT having any.  Just a little bit of occlusion is enough to fool the eye;  people are not going to be looking at reflections off objects with a magnifying glass and judging whether a pixel is correctly or incorrectly occluded...  There just has to be a hint of occlusion;  that's all that is needed, which is very cheap to achieve, in terms of instructions cost.

Edited February 15, 2021 by DanW58

[Stan`]

40 minutes ago, DanW58 said:

But really, all this business of racing against the clock, working all-night'ers to try to make it to an upcoming release (an arbitrary deadline at that;  needed just to implement "crisis management", probably),

It's been 2 years! Linux distributions were starting to drop support from the game, some developpers were starting to give up, and a lot of people felt like we'd never amount to a release. So we decided that now was as good as ever, especially since we managed to put behind us the biggest roadblocks, in order to go back to normal development cycles of 4-5 months.

If anything, it's just a bad timing

 

Quote

and then be told the work is not accepted, and for the most ludicrous reasons, such as a few art items with bad ao bakings not looking good... It is NOT FAIR.  And I'm losing respect for Open Source as a result. 

I would not call (1000+ objects * time to render AO) few   But I suppose we will agree to disagree on this. Also I never said we would not do it. I said we would not do it now

Quote

Not losing respect for the ideals and values it embodies, but for the culture that's grown around it, where people who work for free are treated worse than paid labor in a mine somewhere in the third world.

We all work for free here, so in that regard we're the same.

40 minutes ago, DanW58 said:

You try and "not take it personally" if you were in my shoes.

I will share my experience with you then. I started here in 2011. It took me 7 years to make it to the art team. During which I spent my time working on a lot of 3D models, a lot of which were just thrown in the garbage can because they were not good enough. I spent years telling myself I was not good enough for any of it. And to this day, I still do. If you leave, I will probably also consider it a personal faillure. So trust me when I say I have been in your shoes. The hardest part around here is to deal with the frustration.

One of the persons who answered this thread had to leave and now makes his own mod, because the gap between what he wanted and what the other persons working on this project was too big. And at the end of the day, one of the goals of this project is for people to have fun working on it. If it's only a chore, then everyone burns out pretty fast (I've been there too)

 

Anyway I hope that helps you a bit, if not I'm sorry.

All the best,

Stan.

[DanW58]

1 hour ago, Stan` said:

I would not call (1000+ objects * time to render AO) few   But I suppose we will agree to disagree on this. Also I never said we would not do it. I said we would not do it now

Agreeing to disagree should be the LAST RESORT.  It represents complete dialectic failure.  There are two situations that justify agreeing to disagree:  In matters of taste, and when a belief is at stake which cannot be proven true or false.

I thought that what was reported was that "a few" AO's, literally, didn't look good.  All the ones I've seen look good.

Unless you are talking about our previous discussion regarding placing props to occlude ground for AO baking purposes.

I stand by my argument;  however I was not insisting that they all be fixed;  and my current shader compensates for that problem, in two ways:

First by the UP bias in ambient light:  This helps mask the the fact that the AO bakings incorrectly account for light coming from below the ground.

Second by computing ground occlusion for the reflection of the view vector.  Namely, I reflect the eye vector off the surface, and if the resulting vector points downards I say it is hitting the ground, and I replace ambient light by the product of ambient light and ground color, for reflecting purposes.

So, I've essentially semi-fixed the problem, both in terms of diffuse ambient light and specular occlusion concerns;  so most AO bakings look far better now than they used to.

I do, however, stand my ground in terms of future AO bakings.  It is not THAT difficult to place a big cube under a model prior to baking, then throwing it away after the baking;  is it?  It takes about 10 seconds + 5 seconds = 15 seconds (if you are as slow as I am).

I don't know which AO bakings hyperion referred to that don't look good;  you'd think he could have named them...  I haven't seen them, myself.  It may even not be true at all...  It's like Communist China;  if you are accused, you are convicted.

Edited February 15, 2021 by DanW58

[badosu]

Thank you for the exciting new work

Hopefully we will see this in A25, we got a feature freeze recently (the announcement was not public for a while) and am not sure this is too safe to pull in in a short timeframe. If the team deems so, great!

Anyways, just to say I appreciate the innovations you're bringing and that exercising patience is somewhat important.

[DanW58]

Thanks!  And, yeah, I was starting to get the notion that I came in here during a feature freeze festivity.  Thanks for confirming it.

I hope you're right;  and I hope A25 comes soon.  Isn't the OS mantra "release often"?

Edited February 15, 2021 by DanW58

[Stan`]

It is but things went horribly wrong :/

[DanW58]

Tell me.  I'm curious.  What happened?

[Nescio]

48 minutes ago, DanW58 said:

Thanks!  And, yeah, I was starting to get the notion that I came in here during a feature freeze festivity.  Thanks for confirming it.

I hope you're right;  and I hope A25 comes soon.  Isn't the OS mantra "release often"?

More frequent releases would certainly be nice, but doing a release is a lot more work than one might expect. The feature freeze was on the 21st of January (see https://trac.wildfiregames.com/wiki/ReleaseProcessDraft ), the decision to release was made a few weeks earlier. Basically it takes a month to do a release. Even though 0 A.D. is called an alpha, the team cares about stability; fixing bugs tends to take more time than expected. And time spent on the release process is time not spent on other things.

The real problem is the small number of active team members.

[DanW58]

17 minutes ago, Nescio said:

More frequent releases would certainly be nice, but doing a release is a lot more work than one might expect.

Well, here's a solution I found decades ago, and haven't seen anyone else implement.  Back then I was lead developer of a remake of a game of flying in space, trading and fighting off pirates, and it was running on the Vegastrike engine.  I was the main programmer, the main shader programmer, AND the main artist (2D AND 3D);   though I did get a bit of help now and then.  And I HATED releasing,  like everybody else;  plus, it took a huge amount of bandwidth...  My game was 900 megs, which in those days was huge, and we had a lot of players;  and I was paying for my own dedicated server, and bandwidth was limited (luckily never exceeded it).

But so I wondered:  why not put the game, as it installs, on SVN?  That way, all our gamers have to do to get the latest is svn up.  Mac and Windows users had to do a few extra things, as the installation in SVN was the more typical Linux install.

I got a lot of bitter complaining initially (they hated having to install Tortoise), and a lot of resistance from colleagues,  but once people tasted the honey of being able to update their game every day, they loved it, and never complained again.

So I had 3 SVN repos:

The Art repository.

The Code repository.

and

The Installation repository.

Edited February 15, 2021 by DanW58

[Stan`]

2 minutes ago, DanW58 said:

Well, here's a solution I found decades ago, and haven't seen anyone else implement.  Back then I was lead developer of a remake of a game of flying in space, trading and fighting off pirates, and it was running on the Vegastrike engine.  I was the main programmer, the main shader programmer, AND the main artist;  though I did get a bit of help now and then.  And I HATED releasing;  plus, it took a huge amount of bandwidth.  My game was 900 megs, which in those days was huge, and I was paying for my own dedicated server.

So I wondered, why not put the game, as it installs, on SVN?  That way, all our gamers have to do to get the latest is svn up.  Mac and Windows users had to do a few extra things, as the installation in SVN was the more typical Linux install.

I got a lot of bitter complaining initially, and a lot of resistance from colleagues,  but once people tasted the honey of being able to update their game every day, they loved it, and never complained again.

Well we've been trying for years, but for people on macOS it's usually too hard to compile as you need to install XCode and other things, for people on linux it's fine generally, and on windows you have to install the  40GB of Visual Studio if you want to use git (svn has the exe), which is usually a major turn off.

The biggest problem isn't here though. Development version is highly unstable and people have a hard time syncing (since the game is p2p that means you have to have the exact same version on all machines if you don't MP gets out of sync)

Art assets are compressed PNG becomes DDS DAE becomes PMD and PSA for releases, and if you use SVN it's done on the fly and adds a bunch of extra gigabytes in a cache folder plus overhead when running (it's in a separate thread though)

We also have people with very limited bandwidth e.g. venezuela.

 

tldr; We tried but didn't have the same positive encounter as you did.
 

[DanW58]

15 minutes ago, Stan` said:

but for people on macOS it's usually too hard to compile as

No;  you misunderstand me.  My players did NOT have to compile.  The Installation repo had the binaries.  It was 'svn up' and play.

For mac and windows I forget what we were doing, as other people took care of that;  but I think there was no compiling involved either.

The bandwidth argument holds no water, as it is only the FIRST time they check out that they have to get all the dds textures;  from then on each (daily even) release they will only need bandwidth for anything that is new or changed.

This solution takes less than 1% the bandwidth of traditional releases, where at each release people have to download everything again, even though less than 10% of it changed.

I is a HUGE bandwidth saver;  and it removes the headache of struggling towards a release, making sure there are no bugs, etc.  If you release a bug you get immediate feedback and can retract it.  You get used to releasing daily.

EDIT:

Frankly, I don't understand why every operating system and every piece of software doesn't work this way.  It is a testament to the clinical lack of imagination, creativity and innovation that somehow afflicts the entire world, --Elon Musk being the sole exception.  I toyed with the idea of writing my own operating system, a Linux fork perhaps, that takes over Internet communications;  makes them look normal, but uses SVN under the hood.

Edited February 15, 2021 by DanW58

[Stan`]

We have too many distros on Linux/BSD to provide binaries Ubuntu, Debian, Gentoo, Arch Linux, Mageia, Haiku, FreeBSD,OpenBSD, NetBSD to just name a few.

54 minutes ago, DanW58 said:

Tell me.  I'm curious.  What happened?

In December 2018 we released, and things seemed to get back to normal. However most of the team was weared down because of the GDPR compliance the long time without commit, and the rushed re-release due to segfaults.

Then, people also got very busy with life, leaving only a handful of developpers, most of which could not tackle the challenges at the time, either by lack of knowledge, time, or both.  Time passed by with little activity on the programming side, while the art part got a lot of changes.  At each big holidays we were waiting for someone to finally get the time to tackle the spidermonkey upgrade, which was the big roadblock, along with macOS issues (hidpi, sdl2 etc) and the occasional library upgrade e.g. NVTT and each time, he either ran out of time, or had to tackle other problems and could not deliver. this went on for at least a year.

In July last year, I burnt out. I had a big breakdown and left 0 A.D. for almost a month. After this sad episode, I decided that I wanted to give it one last shot, or let the game die on its own. So I took even more things on my shoulders than I ever did before, trying to micromanage everyone, get people in touch, fix some issues on my own, talk to maintainers, get help from other projects, look for new contributors. and generally try to move the project forward, even though forward was as blurry as "get a release done".

Around september wraitii got back and the heavy task of finishing spidermonkey was given to him.  In December  he was almost done, and since that made a release possible, all the energy was shifted towards fixing the game breaking bugs, and releasing. I think we're at the end of the tunnel here.

 

 

 

[DanW58]

Thanks for the story;  helps to know.

As far as the different distros, does each of them need a different binary?  I only ever used Ubuntu, besides Windows;  I know nothing else.

If that's the case, I'm sure there must be a way to persuade SVN to do a file selection.

For example,  I know there's a way for a repo to reference another repo.  I was doing that with the Boost libraries.

So, you could have a private shared Installation repo with 99% of the files;  then a dozen tiny public repos for the varius distros, each containing just the binaries, but referencing internally the shared repo.

 

By the way, last time I was discussing this with people it was mentioned a plan to separate the engine and game repos.

That is good, but doesn't remove the convenience of separating the Art repo also.

An Art repo would contain all the files needed to produce each game item, including high resolution versions of textures, XCF's, all the maps used to create cube-maps,  high detail 3D models used to produce normal maps, Blender files, etc;  which you don't want to include in the game install.

So artists would have access to the art repo, and work there, with all their materials and props and multi-layer XCF's.  When they finish working on some item, they move the final product to the game repo.

Just as people working on the engine, when done, move validated sources to the game repo.

The game repo itself is where most of the javascript work would happen, as well as game-testing, and compiling for various platforms.

Final product from that, excluding sources and including binaries, could then move to the network of Install repos.

 

You guys should do this.  Bit of work at the beginning, but it will make your life much easier in the long term;  telling you from experience, because I did it.

 

Edited February 15, 2021 by DanW58

[Stan`]

57 minutes ago, DanW58 said:

As far as the different distros, does each of them need a different binary?  I only ever used Ubuntu, besides Windows;  I know nothing else.

Some are compatible, some aren't (e.g. you might get away with the same binary on Kubuntu Lubuntu Ubuntu and Linux mint, but it won't work on Fedora)

58 minutes ago, DanW58 said:

If that's the case, I'm sure there must be a way to persuade SVN to do a file selection.

For example,  I know there's a way for a repo to reference another repo.  I was doing that with the Boost libraries.

So, you could have a private shared Installation repo with 99% of the files;  then a dozen tiny public repos for the varius distros, each containing just the binaries, but referencing internally the shared repo.

Having scripts to download binaries is the goal of this ticket https://trac.wildfiregames.com/ticket/1814 (It's only for windows for now)

59 minutes ago, DanW58 said:

By the way, last time I was discussing this with people it was mentioned a plan to separate the engine and game repos.

Yep https://trac.wildfiregames.com/ticket/5366

1 hour ago, DanW58 said:

An Art repo would contain all the files needed to produce each game item, including high resolution versions of textures, XCF's, all the maps used to create cube-maps,  high detail 3D models used to produce normal maps, Blender files, etc;  which you don't want to include in the game install.

So artists would have access to the art repo, and work there, with all their materials and props and multi-layer XCF's.  When they finish working on some item, they move the final product to the game repo.

Just as people working on the engine, when done, move validated sources to the game repo.

https://trac.wildfiregames.com/browser/art_source

 

 

[DanW58]

Ahhh, good! 

[Samulis]

Just a few thoughts (been following this conversation with interest!)-
The modern, end-user-friendly solution for something like you're describing might a game install/update service like Steam or GOG Galaxy. There are many work-in-progress commercial games (M&B II: Bannerlord comes to mind) which pursue a highly-active (near daily) update schedule on such services for their Beta branch, while preserving a monthly 'Main' branch for players seeking a less buggy and more consistent experience; both separate from their internal development branches of course. Granted, we're talking about projects made by teams 5-10x larger than the active dev team on 0 A.D., who are also full-time paid employees of a company, but I digress.

In my experience making music software, a not-insignificant number of (non-Linux) users struggle even to do basic things like copy/paste or renaming files. I once sought a simpler solution for sharing FOSS musical sample library projects while in development and even went so far as to advocate the mediocre but relatively simple GUI-based Github Desktop app; yet still the vast majority just preferred to download the .zip from the mirror despite my best efforts and tutorials... so, personal rule of thumb, if setting something up is any harder than 'download thing and click a button', it's not going to fly for many if not most end-users. Even proper GUI-based installers can be problematic for many, especially if they click too quickly through and do not read the steps.

I like your suggestion of separating the repos for art and such out. Already many of the original raw game sound effect assets are completely lost to time, so we're stuck with either replacing them outright or working from lossy file formats. Now we mostly do sound development over on a separate Git which functions as a Mod for the game, so a small group of us can iterate quickly on a few ideas (and share them with others) before committing anything to the game itself.

Returning to the topic at hand, I think what you are doing looks great so far, especially if combined with the aforementioned new textures. I think the proper solution here would be to fully implement texture maps/keys for metallic as you suggested earlier, rather than trying to auto-detect. That'd probably take an enormous effort, but it is the sort of thing which needs to be done honestly. In the sound department, for A24 we mostly just tried to patch problems up with 'meatball surgery', but for A25 we're already at work on deeper, more fundamental changes to make the sound assets and their implementation consistent and "correct".

Keep in mind 0 AD has been in development for well over a decade. There are brilliant, clever implementations of things, and sloppy messy hacks right next to each other in every corner of the game. New features and changes are not checked by a singular Producer or Director, but by the team as a whole- an experience which can seem at times like an inquisition, when the actual intention is to ensure there are no weaknesses in the proposal and that it will provide a suitable foundation for future development. The deeper the change, the more things are at risk, and as a result the game tends to move more slowly the deeper you want to go. It is easy for me to replace a few bad or poorly made sound effects from a decade ago, but when we get into stuff like changing (or fixing!) the way the sound engine works, it takes a lot more planning and discussion to make sure things work and what we leave behind will be workable for many years to come.

In my opinion: Having said that, it also means it is fundamentally necessary that any code changes really need to be done the "right" way. Layers of hacks upon hacks is what has caused numerous issues on the project, so I believe it's essential whatever is decided must be done fully and in such a way that it doesn't impede future work. This means in a perfect world, changes shouldn't compensate for, override, assume, or guess about assets or previous work because that creates a feedback cycle of spaghetti going back into spaghetti. Ideally every change should be at the root of the issue, not treating or concealing the symptoms. If I understand your proposal and the architectures involved correctly, that means implementing this requires fixing textures so they give you explicit information on what is metallic, and integrating that in the workflow so all textures present and future have that.

Edited February 15, 2021 by Samulis