A "psychic" shader mod; development begins...

[DanW58]

@wowgetoffyourcellphone

In that screenshot I see no specularity whatsoever.

But that is not surprising, because the only specularity present in the official shaders right now is sun reflections.

And the sun cannot reflect off building walls if we are looking down on them.

If you were to turn the camera so that the sun is on the other side of those roofs, however, I bet you'd see a ton of reflected light.

But to get back to the problem, you will soon, even if I were to leave, you will soon have environment mapping.  It is mandatory;  no game can NOT have enviromap for very long.

Environment mapping is when you have a "sky box" or "environment box" or "environment spheremap" and the shader computes the reflection off of every point on the surface of objects, fetching what point in the sky-box they are reflecting, and show that as the specular part of the color.  Once you do that, you will see clouds and whatnot reflecting off everything as if everything was wrapped in heat-shrink clear plastic.  And it WILL look ridiculous.

Right now, the lack of environment mapping is shielding you from seeing the problem.

In my experimental shader, I don't have environment mapping yet, as there are no environment maps yet in the game, that I know of;  but I implemented a little hack:  I used the AmbientColor, as the sky to reflect,  and I compute a rough specular occlusion to limit the reflection boundaries, and I implement a ground intercept, to reflect ground color when the reflection goes downwards.  And with this simple hack I already see problems on building walls reflecting the ground, which they shouldn't.

In other words, there is no specularity to speak of in the game yet;  that's why the problem of excessive specularity doesn't become apparent.  The worst part is that when a shader tries to implement environment mapping, everybody is going to jump and blame the shader for looking like crap...

EDIT:  Which brings me again to the same conclusion that this shader I'm working on will have to become the legacy shader,  and future art will have to target a new shader without hacks, so that the art, the materials, will be tested on the right shader, and be right.  So what I'm doing now with this shader is to either detect metallic specularity and improve it, or else turn it off completely.  It is the best I can do to enable pseudo-environment-mapping eye-candy without running into problems with excessive specularity everywhere.  It is the best way to support existing assets.

The next shader, though, should not support existing assets as they are, but as they should be.  Walls should be black in specular;  not gray;  and the same goes for women's dresses;  unless they are Hollywood star dresses full of metallic stuff...  But if they are they should indicate a specular power of like 2.0... super low!  Otherwise they will all look like the "liquid metal" terminator from the movie, once environment mapping is enabled.

Edited February 16, 2021 by DanW58

[Stan`]

31 minutes ago, DanW58 said:

Having separate shaders for all these texture combinations is a nightmare.  Means that every time you tweak one shader you have to tweak them all !!!  That's shooting yourself on the foot with a silver bullet.

Not really most of them use model.fs

 

[Stan`]

19 minutes ago, DanW58 said:

 

In my experimental shader, I don't have environment mapping yet, as there are no environment maps yet in the game, that I know of;  but I implemented a little hack:  I used the AmbientColor, as the sky to reflect,  and I compute a rough specular occlusion to limit the reflection boundaries, and I implement a ground intercept, to reflect ground color when the reflection goes downwards.  And with this simple hack I already see problems on building walls reflecting the ground, which they shouldn't.

What about the skybox ?

 

[DanW58]

5 minutes ago, Stan` said:

Not really most of them use model.fs

Okay, if they are modifications done on top of model.fs, that's even more evidence for my argument that they can be (and should be) incorporated into model_common.fs.

4 minutes ago, Stan` said:

What about the skybox ?

I don't understand the question.

EDIT:  Do you meant there IS a skybox?

EDIT2:  darn!  Of course there is;  how would the water work otherwise?

Edited February 16, 2021 by DanW58

[wowgetoffyourcellphone]

Just now, Stan` said:

What about the skybox ?

That's what I was thinking. Can use the skybox for the experimental shader?

 

21 minutes ago, DanW58 said:

In other words, there is no specularity to speak of in the game yet;  that's why the problem of excessive specularity doesn't become apparent.  The worst part is that when a shader tries to implement environment mapping, everybody is going to jump and blame the shader for looking like crap...

Well, yeah, I think that's part of the pushback you are getting. While your screenshots may showcase an improvement here and there, overall it makes the game look bad so one will wonder if the "improvement" is worth it. To fully illustrate what you are attempting to accomplish, perhaps you could partner with an artist to make a couple buildings with all the proper mapping to take advantage of your new shaders, or import some pre-made artwork into the game to show what your shader can do for the engine.

[wraitii]

There is a skybox, but no "terrain box", nor environment mapping, nor screen-space reflections, etc.

This is largely why I abandoned my original shader: with the camera looking down, many things will reflect the ground, which is quite tricky to do correctly.

We also just don't have that much that's actually reflective metal.

[DanW58]

2 minutes ago, wowgetoffyourcellphone said:

That's what I was thinking. Can use the skybox for the experimental shader?

Yes,  I just realized how dumb I am.  Yes;  I will.

2 minutes ago, wowgetoffyourcellphone said:

To fully illustrate what you are attempting to accomplish, perhaps you could partner with an artist to make a couple buildings with all the proper mapping to take advantage of your new shaders, or import some pre-made artwork into the game to show what your shader can do for the engine.

I will do this soon;  I want to finish this shader first, and then I will start removing all the hacks, settle on a new texture stack with hyperion or whoever, and partner with an artist to demo the new capabilities.

[DanW58]

21 minutes ago, wraitii said:

We also just don't have that much that's actually reflective metal.

You are right;  specular reflectors are few and far between;  but that is precisely what makes them catch the eye.  A metallic little pot here,  a helmet there and a shield is all you need, and it adds a TON of realism and depth to the entire scene.

EDIT: And you probably have never seen what a bit of Fresnel can do for paints, skin and plant leaves.

EDIT2:  And you've definitely not seen what spec power modulation PLUS fresnel can do for wood representation.

Edited February 16, 2021 by DanW58

[DanW58]

I need help.

I don't understand what's going on.

I have simple code to detect whether the reflected view vector hits the ground.

 

 

float vector_hits_the_ground( vec3 vector, float specpwr )
{
  float blur = 0.5/(specpwr+1.0);
  return smoothstep( -blur, +blur, -vector.y );
}

..................................................................

  vec3  refl_view = -reflect( eyeVec, normal );
  float reflecting_ground = vector_hits_the_ground(refl_view, 10.0*specPow);

.................................................................

#define TEST
//#define A_COLOR
#ifndef TEST
  gl_FragColor.rgb = color;
#else
  #ifndef A_COLOR
    //float temp = is_metal;
    //float temp = 1.0-normal_hits_the_ground;
    float temp = 1.0-reflecting_ground;  /// <<<<<<<<<<<<<<<<<<<<<<<<<-----
    gl_FragColor.rgb = vec3(temp,temp,temp);
  #else
    //vec3 temp = specCol;
    vec3 temp = texdiffuse;
    gl_FragColor.rgb = temp;
  #endif
#endif

You don't need to understand the code though... My question is:  How in the world could it be working just fine for the building and the trees, BUT NOT for the obelisks, or the people, or the camel?

My first reaction would be, some of the items lack normals...  But I have another test for whether normals hit the ground, and that test works fine for the entire scene!  I'm dumbfounded...

 

EDIT:  Adding a screenshot of displaying normal orientations (Y projections) from black (pointing down) to white (pointing up);  thus, the terraces are white.  As you can see, the normals are perfectly normal.  So why in the world reflecting the view vector should work differently for different assets?  Do some assets mess with, or disable, the view vector through compilation switches?

One thing I notice is that the parts of the civic center that don't work are the obelisks and the lions, both of which I noticed were packed inside the civic center, in the mesh.  Are they being moved from inside the building to outside without regard for the normals moving or rotating or scaling with them?  And what's with the women?

But the thing is, the second picture shows that the normals are fine.  I don't understand...

EDIT2:  Another screenshot: sky-biased ambient multiplied by ambient occlusion. The obelisks seem to have no ao bake;  but otherwise this looks fine.

EDIT3: Another screenshot:  This is the Y projection of my reflected view vector;   obviously not working for the people, lions, camels or obelisks, but working fine for the building...  Why?!!!

EDIT4:  Okay, here's a hint:  See the last pic with a green building?  What I'm displaying is the view vector:

    vec3 temp = vec3(0.5)*(vec3(1)+eyeVec);
    gl_FragColor.rgb = temp;

The bit of math is to fit the range, as the view vector spans -1 to +1, while colors span 0 to 1.

Anyways, I would expect the view vector to change slowly across the screen in x and y, and to display for all assets.  However the problematic assets show black.  I can't imagine how...  But that's it, if there is no view vector, how could I get the reflection of the view vector?  My worst fears are confirmed;  some of the assets are able to destroy view vector info through evil options!

EDIT5:  View problem fixed.  The very creation of the v_eyeView vector in the vertex shader was buried in bloody conditionals;  I took it out of the conditional morass, and it worked.  Can anyone explain to me why the use of parallax is a necessary condition for there being a view vector?

EDIT6:  Much better...

EDIT7:  ReflectingGND() now works as it should.

Edited February 17, 2021 by DanW58

[DanW58]

There's WAAAY too many options and conditionals in the shaders;  it is an incomprehensible rats' nest.  I mean, if something does not need a normalmap, just give it one texel normalmap.  If something doesn't need a specular texture, just give it a single black texel specular.  What is the need to have compiler switches everywhere, gazillions of different shaders...?

[badosu]

Remember, for each incomprehensible seemingly redundant and void code path there's a use case

[DanW58]

1 hour ago, badosu said:

Remember, for each incomprehensible seemingly redundant and void code path there's a use case

Is that something I should remember or try to believe?  Well, I've been forced to move dozens and dozens of things out of conditionals, or else it won't work.  One asset denies me having a normal;  another denies me the half-vector, another denies me the view vector...  I can't see the USE of denying information to the shader.  If the shader doesn't use it, nobody hurts;  but trying to get something done and having problem after problem after problem, THAT hurts.

Anyways, I'm making some progress now.  Look at this picture, and look at those women's arms...

See the whitish specularities?  That is natural skin Fresnel sheen, my friend.  The opposite end of the spectrum, vis a vis metals.  Now we got the full range of materials.

Too many hacks, though;  I wrote a skin detection hack, but it detects parts of the building as being human skin, so the sheen is applied to some of the trim...

I also wrote a blue dress detection hack that makes their dresses a bit shinier.

 

Edited February 17, 2021 by DanW58

[Freagarach]

7 hours ago, DanW58 said:

I like this ^ one the most, it is very:

Spoiler

 

[gameboy]

@DanW58 Nice work, friend, next step, what surprise will you bring me, keep working hard, I look forward to your real graphic scenes, let me immerse myself.

[DanW58]

Thanks;  I'm just putting together a few more screenshots.  Tomorrow I'll tackle using the skybox as enviromap.

This is HUGE, though.  Even though as wraitii pointed out, there aren't a lot of metallic objects, the whole point is that the few there are really look like metal.  And then with Fresnel for non-metals we can show the natural shine of human skin and plant leafs.  Even if it is a few tiny places in a given scene, it is soooo refreshing to see them well represented, instead of everything looking like plastic dolls and toys.

This also goes with the water.  The water in this game is exquisitely well portrayed;  and terrains too;  but then it makes a stark contrast to see most materials look like cheap plastic.  Getting masonry to look like masonry, metal look like metal, paint look like paint, and human skin that glows in the sun;  these are tremendous contributions to realism and immersion.

Edited February 17, 2021 by DanW58

[DanW58]

Here's more screenshots.  Keep looking at specular highlights on metals and skin.  Opposite ends of the spectrum.

The potted plants in the Persian CC,  I believe they got a bit of Fresnel by accident.  They will give you an idea of how I want plant leaves to look like, shiny;  though I did not intend it yet.

Tiled roofs and cobblestone grounds are getting Fresnel also, leaking from my skin detection hack, so they look like varnished...

This will not happen with the final shader, where a texture channel clearly says "this is a metal", and if it says it is a non-metal, another channel says "this has Index of Refraction such and so".  Right now it is very difficult to have the shader isolate items on the basis of their color alone.

Heck, even the palm trees behind the Persian cc have Fresnel gloss...

 

Edited February 17, 2021 by DanW58

[Stan`]

It's still quite dark. Is it because of the AO? or lack thereof?

[DanW58]

It's just right.  If you want more light, make the sun brighter, or increase ambient.  I don't have any hacks to alter the physics or the optics;  it is what it is, according to the lights and the materials;  no more;  no less.  It just follows physics and optics.  Users have brightness AND contrast controls on their monitors;  so do I;  so do you.  How do we know that our controls are all adjusted exactly the same?  This is a subject we should explore, like how bright or dark should it be shipped, so that it doesn't fall out of the range of users' adjustments.

One thing I would hate, though, is in-game brightness controls, before anybody asks;  because then you run into the trouble of "should I increase the game's brightness?, or my monitors?, or my videocard's?  Too many adjustments in a long chain.  It's like if you are a musician, and you got 5 effects pedals, each having a gain pot, followed by the amplifier's gain;  and you also have a volume pot on your guitar or bass;  you end up with like 10 volume pots, and you don't know which needs adjustment.

Let me ask you another thing, though:  In the real world, would you go about complaining it is too dark or too bright?  I mean, today we would, because we have electric lighting we can control.  But in the real world it happens often you're near a building at dusk, and the building looks super bright with golden radiance;  and the rest of the place looks rather dark.  No?  Why should it be different in-game?  Obviously I'm not alone feeling like this way, as someone created an Acropolis at Night map, designed to be dark;  --even challengingly dark.  I played that map several times.  As a gamer I enjoy realism far more than I do unrequested catering to my incorrectly presumed preferences.

If you want to know how it all got darker than before, well, part of that is that a lot of the stuff has bright specular textures, most of which I'm suppressing.  The ambient occlusions were not being used in many cases, and where they were used, their gain was turned down to as low as 0.6 for no reason whatsoever (in those xml files);  and then the shader was multiplying the ao's by 2.0, also for no reason whatsoever.  Last but not least, some of the grounds had bright specular textures, causing the color of the ground to go to saturation.  Once I turned off specular in the terrain shader, the diffuse textures, specially the Acropolis map, were still too bright, so I worked at detecting grayscale color (in the terrain shader) and dimming it while increasing contrast, and lo and behold, I discovered for the first time that the ground in the Acropolis map had nice square tiles.  I had never known that before.  The same goes for some of the stone paths, like in the Badlands, and Belgian Bog;  now they can really be appreciated in their full beauty;  before they were just whitish blurs.  So yes, if you try to go brighter than monitors can display, you lose a lot of detail :-)  You might care to sue Samsung rather than blame me.

So right now everything is right, by physics and optics.  The way to increase light is by cranking up the light, if that's what you want;  but it looks good to my eyes.  I want to see contrasts.  I want to see bright sun reflections contrasting with darker surrounding objects.  If you make everything be as bright as to be in the goldilocks of visual perception, you lose realism and beauty overall;  it all becomes a cheap arcade.  You might as well ask Goya why his paintings were so dark.  Well, he was showing realities that other painters were ignoring.  The question is, does it look real?  Forget about passing judgement on how bright or dark it should be;  does it look real?

Having said all that, I do enjoy direct sunlight, in the VERY FEW maps that feature it.   In the pictures I just posted, only the first one shows some hint of nice rays;  the rest are all ambient light based, which is boring as hell.  More maps should be like Belgian Bog and Oceanside.  Beautiful suns...

Ambient light should be reduced in most maps;  it is so bright it doesn't let you appreciate the sunlight;  and it reduces the visibility of shadows, which are important for 3D perception.

 

EDIT:

Speaking of brighness and contrast, I just remembered an issue we worked on at Vegastrike, namely "gamma awareness", which is a rather obscure subject... Monitors, for the longest time, have been very non-linear devices, and it's been customary to this day to compensate for the nonlinearity in the color values of assets and textures.  The theory is long to explain, but in practical terms, what it boils down to is that when a shader reads any textures, it should immediately square the value, to convert from standard texture gamma to linear space.  Once all the light computations are done, and the shader is about to output the final color for the pixel, it should take the square root first, then output, so as to go back to display device gamma space.

I'm going to try this tomorrow.

I mean today...  4 AM !!!

Edited February 17, 2021 by DanW58

[DanW58]

LOL

@Stan`  I think you'll be happier than me about this:  I just implemented gamma awareness, and it got brighter.

 

[DanW58]

So, here's a bunch of screenshots of the gamma-aware correction that brightened everything  ... 

Using the squares of the colors from the textures broke both my metal detection and skin detection hacks.  I had to adjust all my coefficients to get them to work again.

In the last picture, Romans, the metallic looking ornaments on the columns, I don't think they are metal detection, but rather fresnel put on them, sort of like varnish... "skin detection".

 

Edited February 17, 2021 by DanW58

[Stan`]

Yeah better now

[DanW58]

And as a result of making it more correct, not less.  So, no argument here.  For my taste, I'd lower ambient quite a bit, though.  But I'm happy in general, particularly with having got metal AND non-metallic specularities to work.  The rest is a mop-up operation.

EDIT:  Almost forgot.  Here's the current shaders:

 

model_common.fs model_common.vs terrain_common.fs

Edited February 17, 2021 by DanW58

[gameboy]

Looking forward to your final work.

[DanW58]

2 hours ago, Stan` said:

Yeah better now

Actually no;  it is way too bright.  Now I can't see the sheen on arms of the women like I did before.  The arms are already so close to white that sheen saturates.

There's a golden rule about brightness:  No part of a scene should saturate unless it is a sharp metallic sun reflection.

I'm going to tone down this gamma thing until I can see the sun's reflections on skin again.

Edited February 17, 2021 by DanW58

[DanW58]

This is better.  Still too bright for my taste, but at least I can see gradients of color on skin;  not just saturated white.

And of course I don't fail to notice the tiled roof of the house saturating, but that's due to unwanted gloss added to the tiles due to the skin detection hack which can't avoid detecting buildings as skin;  --almost the same color in diffuse.

 

Edited February 17, 2021 by DanW58