A "psychic" shader mod; development begins...

[Stan`]

2 hours ago, DanW58 said:

 

I would understand a couple of switches having to do with graphics settings;  but why should there be a switch for whether to use specular, another for whether to use specular map, another for whether to use normal map ...  Why would you NOT want to use these things?

Because of the material quality slider. With the lowest settings some materials have the only base texture. Also sampling 2 textures was slower than sampling 1 texture on old hardware.

[DanW58]

4 hours ago, Stan` said:

Because of the material quality slider. With the lowest settings some materials have the only base texture. Also sampling 2 textures was slower than sampling 1 texture on old hardware.

Materials SHOULD NOT affect shaders.  They SHOULD NOT cause different compilations of the same shader.  It is a performance disaster to have shaders proliferate to each meet exactly a material's needs and nomore.

The change from one shader to another,  the "context switch", is a very expensive operation;  equivalent to tens of thousands of polygons worth of rendering.  It is insane to do that.  There should be switches associated with graphics quality settings, but NOMORE.  This business of having materials customize shaders is INSANE, both in terms of performance, in terms of complexity added to the art pipeline having to specify so many stupid things via xml files, in terms of code complexity ... and what is the advantage of it?  Is there ANY?  Is there any advantage in starving the shaders of input, so that trying to add a feature implies not only glsl work but engine work, to try to get the uniforms it should be getting but isn't?  Ridiculous.

 

Edited March 4, 2021 by DanW58

[Stan`]

I don't think it causes recompilation for the simple reason that those flags are set for a game. You don't play with the slider during the game.

[DanW58]

Alright, then I'm not understanding anything.  What slider is this?

[DanW58]

8 minutes ago, DanW58 said:

Because of the material quality slider. With the lowest settings some materials have the only base texture. Also sampling 2 textures was slower than sampling 1 texture on old hardware.

Wait:  look at what you said.

You implied that different materials produce different shaders.

Edited March 4, 2021 by DanW58

[Stan`]

[DanW58]

Gottcha.  At fist I was assuming this was a slider for artists to use in Atlas or something.

I still don't understand your earlier statement, though, namely,

5 hours ago, Stan` said:

Because of the material quality slider. With the lowest settings some materials have the only base texture. Also sampling 2 textures was slower than sampling 1 texture on old hardware

Does the slider setting really affect some materials differently from other materials, yes or no?  Straight dope, please;  this is the crucial thing.  If some materials can dictate some parts of the shader to compile or not, differently from other materials, then we are back to Square Zero.

[Stan`]

basic_trans_ao

<?xml version="1.0" encoding="utf-8"?>
<material>
  <alpha_blending/>
  <alternative material="basic_trans.xml" quality="2"/>
  <alternative material="alphatest_ao_parallax_spec.xml" if="CFG_FORCE_ALPHATEST"/>
  <define name="USE_TRANSPARENT" value="1"/>
  <required_texture name="baseTex"/>
  <required_texture name="aoTex" define="USE_AO"/>
  <shader effect="model_transparent"/>
  <uniform name="effectSettings" value="1.0 50.0 0.0075 0.85"/>
</material>

If quality is lower or equal to 2 it will use basic_trans

<?xml version="1.0" encoding="utf-8"?>
<material>
  <alpha_blending/>
  <alternative material="alphatest.xml" if="CFG_FORCE_ALPHATEST"/>
  <define name="USE_TRANSPARENT" value="1"/>
  <required_texture name="baseTex"/>
  <shader effect="model_transparent"/>
</material>

(Slider goes from 0 to 10)

3 minutes ago, DanW58 said:

Does the slider setting really affect some materials differently from other materials, yes or no?  Straight dope, please;  this is the crucial thing.  If some materials can dictate some parts of the shader to compile or not, differently from other materials, then we are back to Square Zero.

It does.

Materials dictate shaders which as you said is expensive. By lowering the quality you lower the number of possible variants, thus reducing the number of shader recompilations because most flags are off.

As I said earlier, we needed to do that because sampling textures was more expensive than the switches.

 

 

 

[vladislavbelov]

35 minutes ago, DanW58 said:

The change from one shader to another,  the "context switch", is a very expensive operation;  equivalent to tens of thousands of polygons worth of rendering.  It is insane to do that.  There should be switches associated with graphics quality settings, but NOMORE.

Binding is a very interesting area to measure and discuss. Approximated numbers of 7 years old talk:

 

For pretty old card you can switch up to 5K shaders per frame.

And the interesting part is that sometimes the shader switching might be faster than using a single shader, especially on 2K/4K displays, because of branching/limited texture cache.

[DanW58]

2 minutes ago, Stan` said:

As I said earlier, we needed to do that because sampling textures was more expensive than the switches.

I find this hard to believe to put it mildly.  It was 20 years ago I was working full time on shaders, and people had texture lookup solutions for everything from Fresnel to Phong and whatnot, BECAUSE IT WAS CHEAPER than computing them, but every effort was put on trying to minimize the shaders, and minimize context switches.  This would imply that we have regressed in terms of texture units...  But you are using past tense...

Are you talking BEFORE 20 years ago?  Or did something really bad happen 10 years ago?

[DanW58]

3 minutes ago, vladislavbelov said:

Binding is a very interesting area to measure and discuss. Approximated numbers of 7 years old talk:

 

For pretty old card you can switch up to 5K shaders per frame.

And the interesting part is that sometimes the shader switching might be faster than using a single shader, especially on 2K/4K displays, because of branching/limited texture cache.

Alright, I rest my case;  but I just can't work with so many #IF's in the code.

[DanW58]

I specially cannot work with such discrepancies between shaders as v_normal and v_lighting being vec4's in some shaders but vec3's in others.

And I cannot work with them if I have to do engine work just to get a uniform in one shader that is available in another, such as a skybox.

 

[DanW58]

And I can't understand, given I have my shader effects slider maxed out,  I cannot understand why some variables declared inside #IF USE_SPECULAR are not getting defined, and such.  I spend hours puzzling over each of these leg-traps;  and if I succeed at solving them it is usually without understanding what I did to solve it.

I'm falling into a bear-trap at every step, and the work takes forever like this.

Edited March 4, 2021 by DanW58

[Stan`]

1 hour ago, DanW58 said:

And I can't understand, given I have my shader effects slider maxed out,  I cannot understand why some variables declared inside #IF USE_SPECULAR are not getting defined, and such.  I spend hours puzzling over each of these leg-traps;  and if I succeed at solving them it is usually without understanding what I did to solve it.

I'm falling into a bear-trap at every step, and the work takes forever like this.

That's because I think you are missing a piece here. You looked at shaders, you looked at materials, you looked at the C++ code but I believe you didn't look at the actors and their props using such materials. (Files in binaries/data/mods/public/art/actors/**/*)

Those files can use the most advanced material (e.g basic_trans_ao_parallax_spec) with all the defines but also the most simple (default.xml)

And the shader slider only decreases quality it doesn't increase it. So if a model uses default.xml it will use that no matter the quality. If you use the fancy material above at 10 it will have all the defines but at 0 it will use basic_trans which has only the transparent define.

 

 

 

[DanW58]

I thought actors were things like John Wayne.  I'll have a look;  thanks;  but I'm afraid knowing this won't help terribly much trying to deal with all these #IF's, and won't bring me the uniforms I need.

[DanW58]

28 minutes ago, Stan` said:

(Files in binaries/data/mods/public/art/actors/**/*)

I can't find "actors" for terrains or maps anywhere in there.

If I want to turn features on for Acropolis Bay, such as specular and normalmap, where would I go?

[Stan`]

22 minutes ago, DanW58 said:

I can't find "actors" for terrains or maps anywhere in there.

If I want to turn features on for Acropolis Bay, such as specular and normalmap, where would I go?

terrains are in art/terrains/**/*.xml

[DanW58]

EACH PATCH OF TERRAIN can customize its own shader?!?!?!  OMG...

That's it, I give up.

This project is only open source in name.  I mean, you can have a long chain of little decisions each of which can be justified;  but if the end result is an incomprehensible mess, then the end result is an incomprehensible mess no matter how right all the intermediate arguments may be.

Edited March 4, 2021 by DanW58

[Stan`]

I'm not really sure what to tell you ^^"

[wraitii]

17 minutes ago, DanW58 said:

This project is only open source in name. I mean, you can have a long chain of little decisions each of which can be justified;  but if the end result is an incomprehensible mess, then the end result is an incomprehensible mess no matter how right all the intermediate arguments may be.

Lol

You could maybe have a bit of respect & understanding for literally more than twenty years of work by literally hundreds of people.

Yes, multiple parts of the game & the engine are imperfect, flawed, or need improvement. I'd be hard pressed to find an area that could not use improvement, in fact - thus is the nature of these projects.

I am getting annoyed by your antics. You do something, ask for feedback, and then when you actually get feedback, you react as if you were persecuted by a shadowy cabal of fake open source people or 'the powers that be' as you wrote.
The way I see it, you're not being productive, you're just wasting everybody's time and energy.

[Stan`]

Maybe that we have the same material for most of terrains, like 300/470.

[DanW58]

2 hours ago, wraitii said:

Lol

You could maybe have a bit of respect & understanding for literally more than twenty years of work by literally hundreds of people.

Yes, multiple parts of the game & the engine are imperfect, flawed, or need improvement. I'd be hard pressed to find an area that could not use improvement, in fact - thus is the nature of these projects.

I am getting annoyed by your antics. You do something, ask for feedback, and then when you actually get feedback, you react as if you were persecuted by a shadowy cabal of fake open source people or 'the powers that be' as you wrote.
The way I see it, you're not being productive, you're just wasting everybody's time and energy.

Regarding my "productivity", that is a circular definition you're using.  I come up with good work in the engine and you don't accept it.  Does that constitute low productivity on my part?

I come up with great shader mods, and they are not adopted for whatever reasons.  Is that low productivity on my part?

What I will take from your personal attack post is about hundreds of people working on this.  I didn't know.  But I've been trying to make things better, but I get no encouragement from ... the powers that be, and not much help, except from Stan.  You guys would know how to give me the uniforms I need, but nobody offers to help;  I'm left all alone to struggle with this.

EDIT:  Reminds me of one boss I had that kept asking me how long the software was going to take, and I kept telling him "I don't know;  stop asking."  One day, when I was almost finished, half way through testing, I told him "3 more weeks, pessimistically", and the next day he cancelled my development and put a new guy (who said he could do it in a week) to work on the project, from the beginning again.  Then, 6 months later, when the project failed and he got sued by the client, he fired ME;  not the new guy;  and not himself...  Circular definitions of productivity, or of success or failure.

EDIT2:  As far as your criticism that I ask for feedback and when I get it I react in some inappropriate way,  I don't think feedback such as "I don't like get() and set()" functions is really "feedback", though you have a right to have preferences;  and I don't think saying "I hate what your shader does to terrains" without any specifics can really be considered "feedback";  and even "your video is of terrible quality" (when in fact it looks exactly like my in-game screen" can seriously be considered "feedback".  I have, AS A MATTER OF FACT, been "persecuted by a shadowy cabal".

Edited March 4, 2021 by DanW58

[Panther]

One thing people in this thread need to start realizing is; that everyone is working on a similar side. The goal is producing an excellent game, which many other, much bigger and financially supported companies have failed at creating. We all desire it to be nice, but everyone has diverse ways and thoughts of how he processes these actions. Why don't we set them together instead of declining the opposite and create a 'middle - solution for all those, that appreciate the game and aren't fixated on establishing their mindset?