Next Shader Development

[DanW58]

1 minute ago, vladislavbelov said:

You mean refraction/reflection maps?

No, what I was trying to do was, well, yes, refraction;  the reading of the ocean bottom, I was trying to bias the texture read, to get a more blurry image as depth increases.  You have multiple fetches there, I think for that purpose, but I wanted to add a bit more blur by biasing the reads.

 

[DanW58]

@vladislavbelov

Done.

https://code.wildfiregames.com/D3603

 

[Loki1950]

RE: the GIMP and dds images version 2.10 has the dds plug-in installed by default it's available as a FlatPack on "buntu derivatives if it's not in the regular repos.

Enjoy the Choice

[vladislavbelov]

15 minutes ago, DanW58 said:

No, what I was trying to do was, well, yes, refraction;  the reading of the ocean bottom, I was trying to bias the texture read, to get a more blurry image as depth increases.  You have multiple fetches there, I think for that purpose, but I wanted to add a bit more blur by biasing the reads.

 

That texture doesn't have mipmaps, because it's generated each frame.

[DanW58]

Gottcha.  I suspected that was it.

[DanW58]

@Loki1950 I had GIMP 2.1 already;  didn't know it could open dds, LOL

I have a problem with GIMP, though;  every time I try to copy a selection, it quits on me like exit(0).

@Stan` Been looking at the skyboxes and some have LOD's;  some don't;  which is kind of perplexing.

All the ones that have LOD's are 512 in size, and include cumulus, dark2, mountainous, overcast, stormy, stratus, sunny, sunny1, sunset, sunset2 and twilight.

All the ones that do NOT have LOD's are 256 in size, and include cirrus, cloudless, dark, default, fog, rain, sunrise and sunset1.

So, you've said that the engine does something programmatically at runtime with the skyboxes, something along the lines of making them player-centered;  but my question is, @vladislavbelov whatever we have in the end, does it have LOD's?

I'm asking because all the work I'm doing of ensuring that blurriness of reflections and phong shading agree with each other, all the math for matching AO numbers to LOD bias, all is for nothing without LOD's in the skybox;  I absolutely need LOD's... for this shader to do its magic.

By the way, I just came up with the idea for a new "dynamics" uniform

uniform vec4 dynamics;

Where,

• dynamics.r = red-hot
• dynamics.g = gore
• dynamics.b = (not used (yet))
• dynamics.a = age

.r, or "Red-hot" would be a channel to communicate object temperature, which the shader translates into a black-body radiation color to be added to emissive color.

.g, or "Gore" is for fading in blood stains for units in battle.

.b, ...

.a, or "Age" is what I've been working on, for fading ageing looks as an object ages, such as rust on metals.

But the idea is not well formed yet;  there's only one Ageing channel, so I could do one of the three but not the others.  Still thinking...

EDIT:  No, that's right;  I can only represent one parameter, that is, a unit can age, or it can bleed, or it can glow red, but only one of them;  but the shader doesn't know how it should interpret the one channel, so the dynamics uniform tells the shader how to interpret or manifest the "Ageing" channel (renaming overdue).

EDIT2:  I also wanted to add a thin level of wetness to terrains, as an added Fresnel gloss, just above the waterline, to represent wetness from the waves, but I can't seem to find any height data, water or terrain, in terrain_common.fs...

Edited February 25, 2021 by DanW58

[Stan`]

1 hour ago, DanW58 said:

So, you've said that the engine does something programmatically at runtime with the skyboxes, something along the lines of making them player-centered;  but my question is, @vladislavbelov whatever we have in the end, does it have LOD's?

It only does so if the file is a png, and if there is a texture.xml somewhere either in the public or in mod mod.

HOWEVER skyboxes are a bit special since they do not go through that, everything is uglily hardcoded (Which means you can't use a png as a skybox for now)

1 hour ago, DanW58 said:

All the ones that have LOD's are 512 in size, and include cumulus, dark2, mountainous, overcast, stormy, stratus, sunny, sunny1, sunset, sunset2 and twilight.

All the ones that do NOT have LOD's are 256 in size, and include cirrus, cloudless, dark, default, fog, rain, sunrise and sunset1.

Those files are old, I haven't seen them changed in the past 10 years.

 

[DanW58]

22 minutes ago, Stan` said:

HOWEVER skyboxes are a bit special since they do not go through that, everything is uglily hardcoded (Which means you can't use a png as a skybox for now)

Yes, I was asking about skyboxes specifically.  I don't get the connection to .png.  All the ones I saw, with or without LOD's, were .dds files.  My question is still the same.  I tried changing the blurriness of the sky's reflections on water by LOD- biasing the texture cube reads, but it had no effect;  which may mean there were no LOD's.  So the question is, were LOD's missing in that particular map due to using an older sky without LOD's?,  and if I get the right sky with LOD's it will work?, or does the engine strip LOD info from sky-boxes?

Never mind, I guess I just have to look in the xml files to find a map that calls for a sky with LOD's and try it there...

I'm going to try implementing a wet shoreline in water_high.fs;  I think I can do it by draining a bit of the water ...

OMG, the water shader code is so complex;  there's depth as a uniform, depth as per refraction, depth read from depth.Tex texture stating buffer precision issues, so this is Z-depth?  I'm lost...

Edited February 25, 2021 by DanW58

[DanW58]

@vladislavbelov What's the difference between,

  varying float waterDepth;

and what comes through the x channel of ...

  uniform sampler2D depthTex;

?

I was trying to drain a bit of the water and implement a "wet shore" look for terrains other than sand, but I'm having trouble understanding the code.

Edited February 25, 2021 by DanW58

[Stan`]

36 minutes ago, DanW58 said:

Yes, I was asking about skyboxes specifically.  I don't get the connection to .png.  All the ones I saw, with or without LOD's, were .dds files.  My question is still the same.  I tried changing the blurriness of the sky's reflections on water by LOD- biasing the texture cube reads, but it had no effect;  which may mean there were no LOD's.  So the question is, were LOD's missing in that particular map due to using an older sky without LOD's?,  and if I get the right sky with LOD's it will work?, or does the engine strip LOD info from sky-boxes?

The game natively loads anything in art/textures/ whether it's TGA or PNG and turns it into a DDS file with mipmaps unless something else is specified in a texture.xml file.

BUT for skyboxes it's utterly hardcoded and only dds can be used and, as you noticed some of them don't have LODS (mipmaps)

https://github.com/0ad/0ad/blob/83e81362d850cc6f2b3b598255b873b6d04d5809/source/renderer/SkyManager.cpp#L54

Hardcoding

https://github.com/0ad/0ad/blob/83e81362d850cc6f2b3b598255b873b6d04d5809/source/renderer/SkyManager.cpp#L93

Funny comment

https://github.com/0ad/0ad/blob/83e81362d850cc6f2b3b598255b873b6d04d5809/source/renderer/SkyManager.cpp#L76

 

[vladislavbelov]

36 minutes ago, DanW58 said:

@vladislavbelov What's the difference between,

  varying float waterDepth;

and what comes through the x channel of ...

  uniform sampler2D depthTex;

?

I was trying to drain a bit of the water and implement a "wet shore" look for terrains other than sand, but I'm having trouble understanding the code.

It's better to draw:

[DanW58]

Thank both of you;  I know all I needed to know today, and it's only 2pm.

[gameboy]

@DanW58 I think I'll see your new graphics soon. Come on, friend.

[DanW58]

5 minutes ago, gameboy said:

@DanW58 I think I'll see your new graphics soon. Come on, friend.

Thanks.  I got myself momentarily distracted with the water shader, trying to make coasts look wet.  I couldn't do it in the terrain shader, as it has no information of water level;  and the water shader doesn't have information about the terrain.  I almost gave up, but then I thought, "what if I drain the water a bit?, lower the level, then I should be able to give the exposed rim a wet look.  So far I think I've succeeded lowering the water level by one yard, but I'm still looking for how how to repaint the exposed part.  All I got so far is arctifacts around the top of islands;  I mean edges of islands towards the top of the screen look like they have water gaps...

 

[gameboy]

Yes, it seems to feel a little better than before, I'm sure you'll finish it off, and I'm looking forward to your new graphic effects.

[DanW58]

This is too hard.  I got a level of darkening...  sort of under the water rather than above it...

...and what it took to get there is unspeakable.

These are the last few lines of

getRefraction()
{
.......................
  float term1 = clamp( 0.9999*worldPos.y - refrWorldPos.y, 0.0, 1.0 );
  float term = pow( term1, 240.0 );
  float temp = clamp( term, 0.0, 1.0 );
  vec3  mulfactor = vec3(0.382 * temp + 0.618);
    return vec4(mulfactor*refrColor, alpha);
}

It took a lot of experimentation to get there, and I've no idea why the numbers work. A power of 200 doesn't work;  but 240 does ... I think I'm banking on some arithmetic error;  this probably shouldn't work at all.  Of course I was trying to get a smooth function, but just couldn't make it work.  And every attempt at lowering the water level through optical tricks failed.

And I mean, I did get to lower the water, but the bottom went down with the water, leaving the land floating a few feet above the water like they have antigravity technology... Might work for Atlantis...

I'd better get back to work on the new shader.

Edited February 26, 2021 by DanW58

[DanW58]

My work on the water_high.fs shader is done;  and it is released as a mod together with the "metal shader" (metal and skin, really), and the terrain shader that reins in ultra-bright textures and adds anisotropic ambient light.  All of that as a package is available as a pyromod package in this forum post:

The "water patch" is also updated by itself here:

https://code.wildfiregames.com/D3603#157271

 

So, getting back to the new shader work, as I was saying many posts ago, what remains is really very little, and I'm going to include it below, though it's probably subject to change;  none of this is compiling yet...

  // Main algorithm begins.
  vec3 incident_spec_RGBlight = incident_specular_light(v3_mod_normal,gndColor,Mat_RGB_diff,Obj_RGB_ao,RGBlight_bnormSky,Mat_SpecularPower,reflecting_ground );
  vec3 fresnel_refl_color = SchlickApproximateReflectionCoefficient( is_metal, eyeDotNormal, 1.0, IndexOfRefraction );
  vec3 incident_diff_RGBlight = incident_diffuse_light(gndColor,fresnel_refl_color,ao_color,aniso_amb,normal_hits_the_ground,rayDotNormal);
  vec3  color = specularly_reflected_light( Mat_RGB_spec, fresnel_refl_color, is_metal, incident_spec_RGBlight );
  color = color + diffusely_reflected_light( Mat_RGB_diff, incident_diff_RGBlight );
  // Main algorithm ends.

  #if USE_SPECULAR_MAP && USE_SELF_LIGHT
    color = mix(texdiffuse, color, specular.a);
  #endif
  color = applyFog(color);
  color *= getLOS();
  gl_FragColor.rgb = mix(color,sqrt(color),1.0/3.0); // Re-gamma implicit de-gamma.
}

What I want to do next is dive deeply into the routines I never described or justified before.

I call your attention to the fact that I have four functions that sort of work together.  Two of them are for incoming light, namely incident_specular_light() and incident_diffuse_light();  and two "outcedent?" light returning functions, namely specularly_reflected_light() and diffusely_reflected_light().

These four functions are the heart of the shader.  It is immensely useful to separate diffuse from specular completely, and it is even more useful to separate incident light RGB, material RGB reflectance, and reflected RGB light.  Incident light multiplied by material color equals reflected light.  You can add or mix light values;  and you can mix or intermodulate reflective values;  but you cannot mix or add a light and a reflectance, even if both are "RGB".  This is why I like to tag my variable names with RGB or RGBlight... RGB alone stands for a reflectance, a material attribute, and its channels must span from 0.0 to 1.0, as materials can't reflect more light than what light hits them.  But RGBlight values can go to the millions;  there's no theoretical limit.

Now, if you are not familiar with computer graphics, you might be asking "incoming specular?!?! ... Isn't light just light?".  Yes:  In the real world photons leave light sources and travel at the speed of light to meet their destiny by knocking an electron somewhere, and along the way get reflected diffusely or specularly, and they would not know the difference after it happened, never mind before.  But a simulation of photons bouncing around is called a "photon mapper" and they are very good for some things, such as baking ambient illumination in complex scenes, but they make terrible solutions for real-time computer graphics.  Heck, even slow and ponderous ray-tracers aren't photon mappers.  What most computer graphics does is go the other way:  from the eye to the light;  yes, backwards.  It is far cheaper to do so.  And the difference between a real-time shader solution, and a slow ray-tracer, is basically the number and quality of bounces.  Most real time graphics I would say computes "one point one bounces".  First bounce being diffuse or specular.  The second bounce is done where it is cheap to do by some humongous hack;  and it is usually a diffuse second bounce from a diffuse first bounce;  specularity be dammed.  Add to that environment cubes and ambient occlusion bakes, and all together add up to about one tenth of a second bounce.  Ray-tracers can go 7 bounces deep, if you are sure you'll live forever.

But so we go from the eye, through the pixel in the screen we are rendering, into the virtual 3D scene, and hit the point on an object we are displaying currently.  At that point we reflect this (backwards) "eye ray", bounce it off the object, as if it was a mirror, to see what direction we should reflect specular-ly.  Once we know the direction, we can add up light coming from that direction;  and that is what I mean by incident_specular_light().

Diffuse light coming to the eye from that point on the object does not need a reflection direction;  it only needs to compute light arriving to that point from any directions, and how they angle relative to the surface normal.  So, this sum of diffuse light from all directions is the incident_diffuse_light().

There's a few tricks to all this that very few shader programmers get right.  For example, incident diffuse light includes environmental (ambient) light, and light from any light sources, typically the Sun.  Now, say we have an algorithm to compute where shadows fall... those are computed from the Sun, so it makes sense to switch incident Sun light on and off as per the is_in_shadow() test;  but it would make no sense to modulate ambient light by shadow test.  That's not a mistake commonly done;  but one mistake very commonly done is to modulate specular reflection by the shadow test, and that is a terrible mistake that looks awful, but few people can tell what's wrong with the rendering.  Imagine you are in a room, looking at yourself on a mirror, and sunlight is hitting the mirror.  Now your room-mate comes and makes a shadow falling on the mirror.  Does that affect your image on the mirror?  Of course not;  it is only if the shadow falls on you that your image in the mirror changes.

Some shader programmers are careful about that, and yet fail to be careful about a deeper subltelty:  It would be a mistake to say that the is_in_shadow() test has NO place in the specular pipeline.  Why?  Because the specular pipeline includes two things:  Environment mapping, where you read the pre-baked sky color that should be reflected, and Phong shading, where you add the Sun's reflection.  The Sun's reflection will NOT be there if the point is in shadow.

Another VERY tricky part is where the specular and diffuse pipelines sort of get joined at the hip, and that is with two-layer materials, such as paints, plastics, skin and green plant material.  It may seem hopeless to try to separate them given that light that refracts into the transparent layer bounces off the underlying opaque material and then wants to come out again, but part of it is reflected back down, to bounce and be colored yet again by the opaque base.

However, the multiple attempts at refracting back out can be accounted for by a single factor to multiply diffuse reflection by.

In traditional Fresnel modelling of glossy paint,
 

fresnel_reflection_factor = getFresnelReflectivity( ray, normal, n1, n2 );

fresnel_refraction_factor = 1.0 - fresnel_reflection_factor;

Most people leave it at that.  What I do is, when I'm in a hurry,

fresnel_refraction_factor = fresnel_refraction_factor * fresnel_refraction_factor;

Why?  Because it takes as much effort for light to refract out as it was to refract in, assuming a specular bounce, of course, but what better can we do?  Well, we CAN do better by considering that light that bounces back in gets colored again by the diffuse color base, and makes another run for the surface;  so you have an infinite series, and if you solve it it becomes a very simple fractional formula.  For a simple example,

1 + 1/2 + 1/4 + 1/8 + 1/16 ....  =  Sum[x=0~inf]{ 1/2^x }  =  2

But anyways, my point is that you CAN pre-calculate the fresnel factor for diffuse from a light source, as well as for diffuse from the environment map, and the two pipes don't need to exchange data between them.

In the next posts I will discuss each of these four routines.  Why?  Just so that there is documentation on them somewhere to be found.  Unlike C and C++, where you are encouraged (in most communities, anyways...) to write a lot of comments;  in glsl you are not so encouraged, since the file is compiled on the fly, at runtime, by the videocard's driver.  Too many comments would increase compile time.

 

 

Edited February 27, 2021 by DanW58

[DanW58]

So, I was talking about Fresnel, and how to calculate the fraction going to diffuse, for a perfect car paint.  (Everything else in the universe of shading dielectrics can be obtained by modifying the perfect car paint model.)  What we want is to deduce how Fresnel should affect the base color below the transparent dielectric layer, taking multiple bounces into account.  I deduced the math many years ago, but I don't have any records, so I'm doing it again even as I write these posts.

Let me draw a picture:

Okay, it took some time to draw that...

NOTE:  In the picture I say "here we assume diffuse", but the light bounces are showing specular trajectory.  I thought diffuse was going to be easier to analyze first;  but no;  specular is easier, so we start with that.

Starting at the bottom, we have a metal base (assuming it's a car body, but it could be terracotta, wood, or anything).

Above it there's a diffuse OR specular color base;  we'll tackle both.

Above that there's the lacquer "gloss" layer, with a Refractive Index of 2 (glass is 1.5;  water is 1.33), which I picked for no particular reason.  Mathematicians like to abstract things;  but I'm not a mathematician;  I prefer to work with an actual exmple first;  THEN abstract.

Above that is the air, with a refractive index of 1.0000004 or whatever, pretty much 1.0, like vacuum.  Assume 1.

Top left corner is the Sun, or a light source, and a sun-ray coming down at 45 degrees.

Schlick's approximation for amount of light reflected goes like...

float SchlickApproximateReflectionCoefficient( vec3 ray, vec3 normal, float n1, float n2 )
{
  float rayDotNormal = dot( ray, normal );
  float R0 = (n2 - n1) / (n2 + n1);
  R0 = R0 * R0;  //R0 is reflectivity at normal view
  float angle_part = pow( (1.0-rayDotNormal), 5.0 );
  return R0 + (1.0 - R0) * angle_part;
}

So, let's do the math:  If the light is coming at 45 degrees, rayDotNormal will be cos(45) = 0.7071.

R0 =2-1 / 2+1 = 1/3;  squared = 1/9.  So, looking at this paint vertically, the gloss layer reflects 1/9th of the light;  not enough to shave, unless the base diffuse color is black, such that it doesn't interfere with the reflection...

The angle part is (1-0.7071)^5 = 0.29289322^5 = 0.002155.

So we have 0.11111 + 0.88888 * 0.002155 = ... fresnel_reflection_factor = 0.1130271.

This means our ray 'b', our Fresnel specular reflection has 11.3% of the strength of sun-ray 'a'.

If we assume the light from the sun is RGB(1,1,1), ray 'b' has color b_RGB(0.113,0.113,0.113).

The amount of light refracting into the clear lacquer layer is 1-refl ...  fresnel_refraction_factor= 0.8869729.

So, ray 'c' has 88.7% of the strength of 'a', and therefore a color c_RGB(0.887,0.887,0.887);

Now, let's assume the color layer is a sheet of gold, specular RGB(0.9, 0.8,0.5):  ray 'd' will have a color of 'c' multiplied by gold's specular color, which turns into d_RGB(0.798,0.709,0.443).

Here comes a big question:  How much of d will reflect back as e, and how much will get out as f?

You'll have to trust me on this one, because the angle is different, having refracted, AND the refractive indices of the two mediums is now reversed.  However, I looked into this situation, did the whole math, and found that the reversing of the indices and the refraction angle all cancel out, and the reflectivity is the same for a mirror reflection of a refracted ray trying to refract out. Our calculation of reflection factor applies equally here:  11.3% will reflect down as 'e', and 88.7 will come out as 'f'.

So, e will be d multiplied by 0.1130271, or e_RGB(0.09,0.08,0.05) ... pretty dark already.

But 'f' coming out will be the difference, (d-e), or f_RGB(0.708,0.629,0.393).

Let's continue:

Ray 'e' now gets colored by the gold again, 'e'*gold, resulting g_RGB(0.081,0.064,0.025).

Ray 'h' will be 11.3% of that, so h_RGB(0.0092,0.0072,0.0028).

And ray 'i' is = g-h = i_RGB(0.0718,0.0568,0.0222).

My calculator is smoking...  Now, the first reflection, b, I submit it is not part of the multi-bounce series;  it is what we call the Fresnel specular reflection.  We can ignore it, except for calculating c = a-b.  Then d = gold*c;  then e = reflection_factor*d, and f=d-e.  What we want to find out is the c to e and e to h ... ratio.

c to d is gold's color;  d to e is fresnel_reflectivity.  So it is 0.113*(0.9,0.8,0.5) ... RRratio_RGB(0.1017,0.0904,0.0565), where RRratio stands for Re-Reflection ratio.  What comes out, f, is a constant times e, namely fresnel refraction to reflection ratio.  If fresnel refraction is 1-reflection, then this ratio is (1/reflection)-1.  Since at each iteration we have an emerging ray at this ratio relative to the re-reflected light, the total light coming out is this ratio times the total light re-reflecting.

So, at each iteration, re-reflected light is RRratioRGB = refract_factor * specular_color.  So the total light re-reflected after infinite bounces is RRratioRGB^1 + RRratioRGB^2 + RRratioRGB^3 + ...

This is solving an infinite geometric series.

Where r meets the criteria −1 < r < 1,  a + ar + ar² + ar³ + ... = a/(1−r)

So  total_re_reflected_light = c_RGB / ( RGB(1,1,1) - RRratioRGB )

and the total emerging light (rays f, i, etc.) is that times ((1/fresnel_reflection)-1)

but since we don't care about the light that dies bouncing back and forth, we ONLY care about the light that we see, we can simply put the formulas together and say that, for the specular base case of specColRGB, we proceed as follows, in pseudo-glsl now:
 

void FresnelSplit( vec3 ray, vec3 normal, float n1, float n2, vec3 specColRGB,
                   out vec3 bounce1, out vec3 bounceN )
{
  float raydotnormal = max( 0.0, dot( normalize(ray), normalize(normal)) );
  float R0 = (n2-n1)/(n2+n1);  R0 = R0*R0;
  float angle_part = pow( 1.0 - raydotnormal, 5 );
  reflectivity = R0 + (1.0-R0) * angle_part;
  bounce1 = vec3(reflectivity);
  float refractivity = 1.0 - reflectivity;
  float RRratio = vec3(reflectivity) * specColRGB;
  vec3 dead_light = vec3(refractivity) / ( vec3(1.0) - RRratio );
  bounceN = vec3((1.0/reflectivity)-1.0) * dead_light;
}

BINGO!

Next post I tackle Fresnel over diffuse,  i.e. paint.

[DanW58]

The thing with dielectric layer over diffuse is that angle is not conserved across multiple bounces.

Imagine, in the picture in the previous post, that we have incident light ray 'a', splitting same as before into a reflected 'b' and a refracted 'c'.  However, ray 'd' will never happen:  once 'c' hits the diffuse layer, it bounces off it in all directions spherically.  Some of those rays may come out;  some may reflect back in;  and when those rays going back in hit the diffuse layer again, each of them will bounce back spherically again.

We again have some kind of geometric series to solve, but the ratio of light going back in after each iteration is no longer connected to the original angle of incidence;  it is only dependent on the base color and the index of refraction of the dielectric layer.

The light that gets out gets out spherically;  not specularly.  This is interesting, because it means that an object with glossy paint does not reflect diffusely following the standard diffuse model at all.  Only light that penetrates the dielectric layer can possibly bounce off the diffuse layer.  Light coming at a shallow angle probably mostly bounces off specularly and never refracts in.  So the intensity of the color reaching your eye is not proportional to dot(v_light,normal), but rather the square of that, or a polynomial of that.  MUCH brighter towards the light source than a diffuse material.

Furthermore, you are more likely to see more of the diffuse layer reflection the more aligned your view vector is to the surface normal.  Why?  Because photons bouncing off in the direction of the normal have a much greater chance of making it out of the dielectric layer than photons trying to come out at a shallow angle.

For each index of refraction there must be an angle relative to the normal that photons coming out at that angle have a 50-50 chance of making it out, or reflecting back in.  Having that angle, I could use it as radius of a cone and calculate solid angle of light making it out versus reflecting back in.

Okay, so, if  R0 = ((n2-n1)/(n2+n1))^2, we can calculate R0 for a number of refractive indices, then ask at what angle is the reflectivity = 0.5.

We have,
 

refl = R0 + (1-R0)*(1-cos(a))^5
(1-R0)*(1-cos(a))^5 = refl - R0
(1-cos(a))^5 = (refl - R0) / (1 - R0)
1 - cos(a) = ( (refl-R0) / (1-R0) )^(1/5)
cos(a) = 1 - ( (refl-R0) / (1-R0) )^(1/5)
a = acos(  1 - ( (refl-R0) / (1-R0) )^(1/5)   );

Let's make a little table:
 

IOR  n2-n1/n2+n1  R0     refl-RO   1-R0   Div.  root5  1-5rt  deg.
===   =========   =====  =======   =====  ====  =====  =====  ====
1.0      0.0      0.0      0.5     1.0    0.5   0.871  0.129  82.6
1.5     -0.2      0.04     0.46    0.96   0.48  0.863  0.137  82.1
2.0     -0.333    0.111    0.388   0.888  0.44  0.847  0.153  81.2
2.5     -0.428    0.184    0.316   0.815  0.39  0.827  0.173  80.0
3.0     -0.5      0.25     0.25    0.75   0.33  0.803  0.197  78.6
3.5     -0.555    0.309    0.191   0.691  0.28  0.773  0.227  76.9
4.0     -0.6      0.36     0.14    0.64   0.22  0.738  0.262  74.8
4.5     -0.636    0.405    0.095   0.595  0.16  0.693  0.307  72.1
5.0     -0.667    0.444    0.056   0.556  0.1   0.631  0.369  68.3

Well, there must be some kind of mistake.  Angle at IOR 1.0 should be 90 degrees,  and should be getting smaller faster as the index of refraction icreases.  I'm too tired, though;  going to bed.  If you figure out my mistake, please post.

Going backwards for the first row, IOR of 1, the result should be 90 degrees.  That means, the cosine, or 1 - fifth-root yada yada should be 0.  So the fifth root yada yada should be 1.  Fifth power of 1 is 1, so the division of (refl-R0) by (1-R0) should have given us 1;  not 0.5 as it did.

Now, 0.5 is the arbitrary reflectivity threshold I set;  and this threshold should not matter in the case of IOR of 1;  we should get 90 degrees for pretty much any reflectivity.  Hrrrmmmmm.....

Edited February 28, 2021 by DanW58

[DanW58]

Okay, I'm tracing the problem to Schlick's Approximation.

Say we have a dielectric with a refractive index of 1;  same as air.  There should be NO reflection at any angle except maybe 90 degrees.  But what does Schlick's hack say?

R0 = ((n2-n1)/(n2+n1))^2  ... (1-1)/(1+1) = 0 ... 0^2 = 0.  So far so good;  reflectivity at 0 degrees to the normal is 0.

Reflectivity at any other angle is a different story ...

refl = R0 + (1-R0)*(1-cos(a))^5  =  0 + (1-0) * (1-cos(a))^5 = (1-cos(a))^5

So, say a is 60 degrees, cos(a) = 0.5, 1-0.5 = 0.5, 0.5^5 = 0.03125;  when  it should be exactly zero.

Small wonder, then, when I did this 20 years ago my Fresnel code was really complex, full of square roots... I was implementing real Fresnel back then for a reason, it seems.  It's funny how foggy my memory seems.  When I was researching stuff for this shader and I found Schlick's approximation, I thought I recognized it as what I had used 20 years ago;  but for one thing:  There were no square roots...  I thought I remembered there being square roots in my original work.  What to do?  Schlick's hack is obviously garbage.  Ok, here's from Wikipedia, real Fresnel starts with Snell's law, for refraction angle:

Call the incident angle i and the refrated angle t, for editing purposes.

n1 * sin(i) = n2 * sin(t)

Remember what we care about is the cosines of these angles;  but this is Snell's Law;  period. So,

sin(t) = (n1/n2)*sin(i)

and sin^2 = 1-cos^2, and viceversa, so we can compute cos(t) as

cos(t) = sqrt( 1 - sin(t) ) =

= sqrt( 1 - ( (n1/n2)*sin(i) )^2 ) =

cos(t) = sqrt( 1 - ( (n1/n2)* sqrt( 1-(cos(i))^2 ) )^2 )

If we make n1=1 for air,

cos(t) = sqrt( 1 - ( (1/n2) * sqrt( 1-(cos(i))^2 ) )^2 )

Then come the wave impedances:  Z1 = Z0/n1  and  Z2 = Z0/n2,  where Z0 is the impedance of free space, but it cancels out in the final formula, so you can make it 1.  So,

Z1 = 1/n1 = 1  and   Z2 = 1/n2

That's the easy part.

Now come the Fresnel reflectivity formulas, for S and P polarizations, which can be averaged for non-polarized light:

Fresnel_reflectivity = ( Rs + Rp ) / 2, where,

Rs = { [Z2*cos(i)-Z1*cos(t)] / [Z2*cos(i)+Z1*cos(t)] }^2

Rp = { [Z2*cos(t)-Z1*cos(i)] / [Z2*cos(t)+Z1*cos(i)] }^2

THAT is the Real Fresnel that I had implemented in a shader 20 years ago.  Now it's coming back...

But I remember also I was doing all this using vec3's,  3 calculations in parallel.  I can't remember why, but may be that I was using distinct indexes of refraction for glass for the 3 colors.  The index of refraction for materials changes with wavelength...

 

 

Edited February 28, 2021 by DanW58

[DanW58]

So, let's put Real Fresnel into code, assuming distinct indexes of refraction for the 3 colors for a give material, but assuming the index of refraction for air being 1 for simplicity.  I'm returning reflectivity as well as cosines of refraction angles, as vec3's.  Refraction sines will be good for refracted texture lookup's.  Refraction cosines will be useful for iridescence calculations.
 

const vec3 WHITE = vec3(1.0);

void RealFresnel( float NdotL, vec3 IOR_RGB, out vec3 Frefl, inout vec3 sin_t, inout vec3 cos_t )
{
  vec3 Z2 = WHITE / IOR_RGB;           // Assumes n1 = 1 thus Z1 = 1.
  vec3 cos_i = vec3( NdotL );
  vec3 sin_i = sqrt( WHITE - cos_i*cos_i );
  sin_t = min(WHITE, sin_i * Z2);      // Outputs sin(refraction angle).
  cos_t = sqrt( WHITE - sin_t*sin_t ); // Outputs cos(refraction angle).
  vec3 Rs = (Z2*cos_i-cos_t) / (Z2*cos_i+cos_t);
  vec3 Rp = (Z2*cos_t-cos_i) / (Z2*cos_t+cos_i);
  Frefl = mix( Rs*Rs, Rp*Rp, 0.5 );    // Outputs reflectivity.
}

Done!

I'm going to test this in the water_high.fs shader...

EDIT:  Fresnel for sunlight entering the water, which is averaged (doesn't use wave normals) could be off-loaded to the vertex shader.

Edited March 1, 2021 by DanW58

[DanW58]

The Fresnel function worked the first time;  no debugging needed.

It's now incorporated in the metal_shader_set version1.3 pyromod's modified water_high.fs.

It works a charm and a half.

So, let's get back to the problem of multiple diffuse light bounces within a dielectric glossy coat:  I was making a table that was failing due to using Schlick's Abomination,  er, I mean Approximation,  to try and visually educe a formula to relate how much diffuse light gets out of a dielectric at each bounce.  To re-do the table using real Fresnel would involve more columns than fit in my poor man's screen;  so I'm going to use LibreOffice Calc, and maybe I will figure out how to copy the relevant stuff to insert here ...

Darn!  To have "50% reflection ANGLE" as a function of refractive index involves a huge math operation on the Fresnel formula.  I'm not sure this is even possible.  I think my table is going to be a multiplication of refractive indexes and angles as a multi-page column, and resulting reflective coefficients in the last column, and then I can visually select input angle for 50% result.

Edited March 2, 2021 by DanW58

[DanW58]

Okay, it's been long enough that might as well put this in a new post.  I'm attaching the spreadsheet here so that people interested can look at it.

Note that we are talking about light bouncing diffusely (spherically) and then trying to get out of the dielectric medium.  So we are talking about going from a higher refraction index medium to a lower index of refraction medium.  In this case, there is an angle at which reflection is mirror-like.  If you've ever been underwater in a swimming pool with your eyes open, or with goggles on, you've probably noticed that looking up you see a circle through which you can see stuff above the water, but further away the surface of the water looks like mercury;  totally opaque, and reflective like a mirror.  Inside water, the angle of total reflection is 49 degrees from the vertical.  The angle of 50% reflection is 48.1 degrees. At 45 degrees, see-through is 95%.  So, it's not a very gradual transition;  it is smooth but almost step-wise in its rapidity.  So when we obtain our 50% reflectivity angle and use that to calculate solid angle and % of light that escapes, we don't need to worry about transition function feactoring in;  our result will be pretty much exact.

So here is the relevant results:  First column is indices of refraction, second column is the 50% reflectivity angle, and the third column is an approximation function I found to match the results, which function involves two odd powers.  If someone is good at matching data with functions and wants to contribute a better one, it will be much appreciated.  The goal is an f(x) that approximates the solid angle (hemispheres) column with the refractive index as input;  hopefully a polynomial, or something cheaper to compute than two powers plus two multiplications and one addition...

Example     Refractive   Angle for   Solid angle   f(x)= 0.593/x^6.5
Material      index      50% refl.  (hemispheres)    + 0.407/x^1.75    Error
========    ==========    =======    ===========    ===============
 Air          1.0000       90.00        1.0000           1.0000        0.0000
Water         1.3350       48.10        0.3322           0.3361        0.0040
Glass         1.5000       41.40        0.2499           0.2427       -0.0072
Cr2O3         2.0000       29.70        0.1314           0.1276       -0.0038
Diamond       2.5000       23.35        0.0819           0.0834        0.0015
Merc.Sulfide  3.0000       19.30        0.0562           0.0600        0.0038
Silicon       3.5000       16.47        0.0410           0.0456        0.0046
Germanium     4.0000       14.36        0.0312           0.0360        0.0048
Unobtanium1   4.5000       12.76        0.0247           0.0293        0.0046
Unobtanium2   5.0000       11.47        0.0200           0.0244        0.0044

 

FresnelTable.ods

Edited March 2, 2021 by DanW58

[nani]

Use taylor expansion for 1- cos(x) =~   1 - ( 1 -x^2/2! + x^4/4!) = x^2*(1/2 - x^2/24)

Taylor Expansion	error
0.9800	-0.0200
0.3317	-0.0005
0.2497	-0.0002
0.1313	0.0000
0.0819	0.0000
0.0562	0.0000
0.0410	0.0000
0.0312	0.0000
0.0247	0.0000

[DanW58]

Managed to improve the function, while using integer powers now.

Example     Refractive   Angle for   Solid angle   f(x)= 0.593/x^6.5
Material      index      50% refl.  (hemispheres)    + 0.407/x^1.75    Error
========    ==========    =======    ===========    ===============
 Air          1.0000       90.00        1.0000           1.0000        0.0000
Water         1.3350       48.10        0.3322           0.3361        0.0040
Glass         1.5000       41.40        0.2499           0.2427       -0.0072
Cr2O3         2.0000       29.70        0.1314           0.1276       -0.0038
Diamond       2.5000       23.35        0.0819           0.0834        0.0015
Merc.Sulfide  3.0000       19.30        0.0562           0.0600        0.0038
Silicon       3.5000       16.47        0.0410           0.0456        0.0046
Germanium     4.0000       14.36        0.0312           0.0360        0.0048
Unobtanium1   4.5000       12.76        0.0247           0.0293        0.0046
Unobtanium2   5.0000       11.47        0.0200           0.0244        0.0044

 

Thanks;  I had to submit this table I was working on, to look at your reply.  I'll check that out.

EDIT:  I don't get what you use as x.

 

Edited March 2, 2021 by DanW58