Next Shader Development

[DanW58]

@wowgetoffyourcellphone  Thanks.  For me it might as well be something between a sprint and a marathon, because I did it before.  Not as well organized as it is getting now;  it was a huge mess back then;  but many of these ideas I tried them before.  I have to say though, this is looking infinitely better.  I'm in love with the texture groupings and channel packings.  I never imagined something so good.  My old experiments needed about 10 textures.  Here we have 5.  (I'm talking run-time textures, as opposed to art-level textures).

The one-texel textures, that's not my idea;  we had a whole bunch of them in Vegastrike;  very useful.  Obviously what lead to there being so many compiler switches in the shaders is a misunderstanding or bad assumptions about what increases or decreases GPU performance.  Someone thought that stripping down a shader to the bare minimum code needed for each object was a good idea.  That's the opposite of the truth;  that implies that the same shader source file generates a different shader for every object, and you end up with hundreds of shader swithches.   Each shader switch has an equivalent cost of rendering about ten thousand polygons.  What you want to do is have as few shaders as possible, with NO compiler switches, and even then sort your draw calls so that all the calls that call each shader are together, so that for N shaders you only have N-1 switches.   The only compilation conditionals justifiable are those related to user settings, which don't change frame to frame;  but game assets should NOT have a way to tailor ther shaders.  Pick one by name maybe;  but not tailor them.

Please don't be shy to ask questions.   I try to be clear rather than mystical, but I'm probably taking for granted a few concepts.  I know when I was getting started with shaders one of my biggest disbelief items was about all of that code happening for every pixel on the screen.  Took a while for my brain to accept that.  Back in the days of Doom, the holly grail for game graphics was to get the rasterizer to work while executing less than 6 assembler instructions per pixel...  I had a book that chronicled someone's jurney of code optimization, and how he got to 8 instructions per pixel and could not optimize anymore... Until someone whispered another trick in his ear at a convention, and then he got to six instructions.  Here we are computing logs, powers, trigonometry, matrix and vector math, pages of it ... per pixel.  Per fragment, rather.

@asterix  Many thanks;  I'll check that out.

EDIT:  I just listened to the first video;  it's refreshing to hear Vladislav Belov speaking;  he knows his stuff.  One thing he might consider in terms of sorting is using something like bubble-sort with some tweaks.  It is traditional to use bubble-sort as a perfect example of un-optimized algorithm, but few people realize that bubble-sort is the fastest sorting algorithm for a presorted set, and that in fact quicksort is the slowest.  In the case of sorting objects by Z depth, from one frame to the next the sorting doesn't typically change much (unless the camera moves fast), so the set is usually pretty close to a pre-sorted set;  much cheaper to re-sort with bubble sort than quicksort.  And in fact, the engine knows when the camera moves and by how much, so it can set a movement trigger for a qsort call.

 

Edited February 22, 2021 by DanW58

[DanW58]

Okay, so here's the whole texture pack, again, with the latest changes.  And I'm going to throw in some examples of what it can do for materials and objects.

First the Material Textures set:

Red_channel       5 bits   "albedo.dds" (DXT5)
Green_channel     6 bits        "
Blue_channel      5 bits        "
Aged_Color        8 bits        "
MSpecularity      5 bits   "optics.dds" (DXT5)
SurfacePurity     6 bits        "
FresnelGloss      5 bits        "
Smoothness        8 bits        "

Aged_Color is for rust color, typically;  0.0 = black, 0.2 = burgundy, 0.4 = red,  0.6 = orange, 0.8 is no effect, and 1.0 is transparent rust.  This channel does not tell you where the rust is to be located in a material;  only what it WOULD look like if called for.  What calls for rust to manifest is the Ageing channel in the Zones texture;  see below.

The Object Textures set:

Normal_U   8 bits  "Forms.png" (PNG sRGBA; no mipmaps)
Normal_V   8 bits       "
Normal_W   8 bits       "
Height     8 bits       "
Emit_red   5 bits  "Light.dds" (DXT5)
Emit_grn   6 bits       "
Emit_blu   5 bits       "
AO_bake    8 bits       "
Faction    5 bits  "Zones.dds" (DXT3)
Ageing     6 bits       "
DetailMod  5 bits       "
Alpha      4 bits       "

The Ageing channel ushers-in rust --for metals ...

NOTE:  What Ageing and Aged_Color would or could do for non-metals is not clear to me.  I'd hate to waste an opportunity for yet another fancy feature;  but the problem is that we know many metals rust, and we know the color the rust will be in advance.  If we tried to use these two channels to usher some effect on non-metals, the question is what do we know in advance about those materials.  We could use this to progressively fade in blood stains on warriors as their health goes down, for example.  If anyone has any ideas, by all means, speak up.

The rules, as I'm thinking them so far, are:

Where Ageing is 0.0 (black), nothing is modified.

Where Ageing is 1.0 (white), modifications are maximal, as follows:

If Aged_Color is below 0.8 (light gray or darker) the corresponding color will be shown, matte, on top of the metal surface (overwriting albedo color, zeroing specularity, and zeroing Purity for good measure).   And if Ageing is, say, 0.5 (50% gray), the matte color will be alpha-blended at 50% on the metal.

If Aged_Color is 1.0 (white) and Ageing is any non-zero value, a dielectric, transparent film on top of the metal will be simulated, of a thickness of up to 10 microns, modulated by Ageing's value.  Thus, white in ageing produces the thickest dielectric film.  The dielectric constant of the film is whatever the Gloss channel specifies.  And rather than decrease specularity and purity, Ageing with Aged_Color at white will actually boost specularity and purity.  This clear rust film will be almost invisible;  hard to notice, unless you see something bright reflecting off the object, in which case you'll see moving bands of rainbow-tinted color, kind of iridescent.

What this whole thing allows is to encode an alternative look for each material via the material textures, and to then interpolate between the two apparences via a color channel in the object textures domain.

Let's create a few metals, just for fun:

ALBEDO.DDS:       SILVER     IRON    STEEL   CHROME  ALUMINIUM  GOLD
Red_channel        0.8       0.4      0.6      0.70     0.93     0.9
Green_channel      0.8       0.4      0.6      0.70     0.93     0.8
Blue_channel       0.8       0.4      0.6      0.70     0.93     0.5
Aged_Color         0.0       0.2      0.6      1.0      1.00     0.8

OPTICS.DDS:
MSpecularity       0.8       0.6      0.7      1.0      0.9      0.9
SurfacePurity      1.0       1.0      1.0      1.0      1.0      1.0
FresnelGloss       0.0       0.0      0.0      0.55     0.19     0.0
Smoothness         0.7       0.4      0.6      0.9      0.7      0.9

NOTES: Chromium's 70% reflectivity is by reference. Its oxides' refractive indices are hard to find, but I found
a refernce to chromium thin films having RI of 3.2. To encode 3.2, subtract 1 and divide by 4, ergo Gloss = 0.55
Aluminium's 93% reflectivity is by reference. Alumina's refractive index is 1.77. Gloss =1.77-1... 0.77/4 = 0.19
For other metals, the figures are fudged; pulled out of nowhere.
Smoothness is not a material attribute, traditionally speaking, but here it is;  thus polished silver and rough
silver would be different materials.  The materials here are assumed polished, but the smoothness varies by the
polish-ability of the material (equalized efforts; not results XD), and also by what we want the passivated rust
finish to look like.  Compromise...
The same can be said of MSpec where 0.9 means that 10% of the light reflects diffusely, 90% specularly.  Is that
the case?  Well, If I look at aluminium foil I think far less than 10% of it reflects diffusely, but then I look
at my aluminium pot and cry.
Aged_Color is dialing in black for silver, dark red for iron, orange-red for steel, and clear film of passivated
rust for both chromium and aluminium.  For gold, at 0.8, it dials in no change at all.
Gloss is not applicable to non-self-passivating oxide metals (unless you varnish them... or anodize aluminium).
Purity refers to absence of diffuse particles on a dielectric surface;  default is 1;  less than 1.0 values are
used to describe plastics or biological materials.
Nothing prevents you from creating a custom, multi-material material, and use it for a single object;  therefore
if you need a copper sarcophagus with green rust, you can create that in this system. The Ageing mechanism is a
trick to get a lot of art done for less effort, but it is not the only way to get rust.

But like I said, those rusts are dormant until you put some light into the Ageing channel in the Zones texture;  then rust comes alive.

This will save a lot of work.  Think about it.  Without this feature, to put one dot of rust on a metal surface you'd have to, a) put the dot of red on the gray of iron, and b) put the same dot but in black in the specular channel.  That's simple, okay;  but what about then a sprinkling zones of rust kind of alpha-blending on the metal?  Then you have to start creating masks and using them for specular and diffuse.  But then what about a film of refractive self-passivating oxide of varying thickness?  Now you have several channels to modify using masks.  Here you just air-brush the Ageing channel, and all the layer work is taken care of.

Furthermore, it has the benefit of keeping the definition of rust together with the metal, but the control of it with the objects' textures. So, if you change your mind and switch a sword from steel to chrome, or vice-versa, you just change the material it calls for;  the rust will be in the same places, just look different.

Conversely, if you find your sword has too much or two little rust, you don't have to change the material, affecting possibly dozens of other objects that use it;  you just dial down the Ageing channel of your sword's Zones texture.

 

 

Edited February 22, 2021 by DanW58

[DanW58]

Hmmm.... I've got myself into a conflict.

Gloss (refractive index) has two meanings now.

My original intent was for gloss to stay at zero for metals.  Zero gloss maps to 1.0, which is the refractive index of vacuum by definition, and of air, by luck.  So Fresnel specularity is computed for metals, but produces no effect.

But now, for metals like chromium, where a self-passivating oxide can happen by rusting, and needs a refractive index, if I use the Gloss channel to specify it, the whole thing will have a film of it, not just the rusted parts.

Unless I use MSpec to deny Gloss unless Ageing is non-zero.

This is doable, but would deny us the freedom to have anodized aluminum, or varnished metal;  this may not be an issue for 0ad but might be an issue for my Masters of Orion remake mod.

On the other hand, the latter problem could be solved by making the Ageing channel all white and forego ageing effects.

Think,  think, think ...

EDIT:  I actually like the last solution;  the problem is that it makes the shader a bit more complex to understand.  Artists will have to have a good grasp of the shader pipeline, but having Gloss denied by metallicity unless pushed by Ageing is a bit of a gordian knot.  Furthermore, there's no reason to deny Gloss by metallicity if the rust type indicated by Aged_Color is not 1.0 (transparent).  In other words, Iron rusts red, so Gloss would be at zero.  If Gloss is not zero, it could not possibly be for rusting, so it means we are varnished head to toe.  More complexity...   On the other hand, how often will this situation arise?  Need coffee ...

EDIT2:  With the last solution, the ONLY two situations that will be odd are a varnished chromium sword, and a heatsink of anodized aluminium.  In order for these to be covered in varnish or rust, respectively, they have to have their Aged channel whitened.  The problem with that is that they won't be able to rust;  but then again, neither WOULD rust, anyways!

 

Edited February 22, 2021 by DanW58

[hyperion]

Ageing is an over-definition/redundant, can be expressed by changing albedo, is metal, roughness, etc.

[DanW58]

 

@hyperion No, Ageing is far more useful than that.  I was editing and re-editing my second-last post, and maybe you missed the last paragraph.  Quoting myself,

11 hours ago, DanW58 said:

This will save a lot of work.  Think about it.  Without this feature, to put one dot of rust on a metal surface you'd have to, a) put the dot of red on the gray of iron, and b) put the same dot but in black in the specular channel.  That's simple, okay;  but what about then a sprinkling zones of rust kind of alpha-blending on the metal?  Then you have to start creating masks and using them for specular and diffuse.  But then what about a film of refractive self-passivating oxide of varying thickness?  Now you have several channels to modify using masks.  Here you just air-brush the Ageing channel, and all the layer work is taken care of.

Furthermore, it has the benefit of keeping the definition of rust together with the metal, but the control of it with the objects' textures. So, if you change your mind and switch a sword from steel to chrome, or vice-versa, you just change the material it calls for;  the rust will be in the same places, just look different.

Conversely, if you find your sword has too much or two little rust, you don't have to change the material, affecting possibly dozens of other objects that use it;  you just dial down the Ageing channel of your sword's Zones texture.

And as I've said elsewhere, ageing could be used for other things, e.g. modulated via a uniform, to control blood stains for units fighting as their health decreases.  Of course you wouldn't want to dynamically adjust rust, but you could make sure units using rust don't use blood, and viceversa.  This could have a lot more uses than just rust.  That's why I gave it the name Ageing.  It could also be called Dynamic, but be static by default.  Lots of possibilities...

EDIT:  Also, don't forget, this mechanism doesn't prevent you from making your own rust using the normal texture channels;  it merely gives you a shortcut way to do it, that is easy to modify, and lessens the amount of work in creating custom materials with delineated rust zones that may not be useful to other objects.

EDIT2:  Yet another advantage is that once the metal textures are created, each with its own Aged_Color, a misled or subversive artist cannot easily call for blue rust on iron, or red rust on gold.  Assuming your material textures are organized into larger "album" textures, I imagine there is some protocol to follow before someone can add a new member to the album, right?

EDIT3:  In fact, @hyperion , and I'm sorry I have this habit of editing my posts for hours...  I'm thinking of further uses of CPU control via a uniform.  This is not only good for effects that change over time;  it could be also an instance differentiator.  Just like when you train cavalry, and the horses are of different colors, instancing buildings and other assets perhaps could benefit of individualization via texturing effects.  Another possibility is for a uniform to bring in an exponent, between zero and infinity, to raise Ageing to.  Ageing has a range 0~1, and any values in between can be pushed down by raising to a power greater than 1, or pulled up by raising to a power less than 1, thus increasing or decreasing the "gamma" of Ageing;  so the CPU could control ageing this way.  The artist would sprinkle and blur the Ageing channel to get a "middle age" look, and the CPU could dynamically increase or decrease the age.  If we generalize this mechanism to actually represent chronological age, or weathering, building decay could be portrayed using it.

EDIT4:  Last but not least, if I were to NOT implement this Ageing feature, I'd have two vacant texture channels:  one in the materials;  one in the object.

EDIT5:  In fact, my worry is not that artists will find this redundant and not use it;  my worry is that it may be liked too much, and abused;  and/or more features like it be requested daily.

Edited February 23, 2021 by DanW58

[DanW58]

I consider myself lucky that I ran into this conflicting definition of how or when Gloss is applied.  I think that the solution I arrived to a couple of posts ago is pretty much the only solution, however convoluted the logic may seem.  I consider this lucky because it has just given me a clear direction to proceed forward.

Just as one of the first things you need to get operational when building a giga-factory is the switch-yard,  here we need to put all this complex logic in one place, and make sure it reads like poetry.

Let me first write down pseudocode of this whole mess;  just remember the conditions will be fuzzy booleans, and-ing is done by multiplication, and all code will execute: but results will be conditionally attenuated.  A sort of "analog switchyard", full of giant rheostats.  You don't have much support for code jumps in glsl.
 

// init defaults:
vec3  matte_rust_color = black;
float  ageing_throttle = 1.0;
float  ageing_effect_on_thick = 0.0;
float  ageing_effect_on_mspec = 0.0;
float  ageing_effect_on_gloss = 0.0;
float  ageing_effect_on_purity = 0.0;
float  ageing_effect_on_smooth = 0.0;
float  applied_Gloss = nominal_Gloss;
// Deny Gloss if it's metal and is not ageing in a passivated dielectric way...
// In other words, if it's a metal but its Aged_color == 1.0 and it IS ageing, then let there be Gloss on it
// or if it is a metal but it is a colored ruster, let there be Gloss on it, if so indicated,
// or if it is a non-metal then by all means allow a Gloss on it,
// but otherwise kill the gloss
if( mspec > 0.5 && Ageing != 0.0 && AgedColor != 1.0 && ..........
{
  applied_Gloss = 0.0;
}
if( Age_Color == 0.8 ) //NO color
{
  ageing_throttle = 0; //NO ageing
}
else_if( Age_Color < 0.7 ) // A real rust color:
{
  MatteRustColor = get_RGB_from_Age_Color( Age_Color );
  ageing_effect_on_mspec  = -0.9;  // Switch from specular to diffuse.
  ageing_effect_on_gloss  = -3.7;  // Peel off varnish first, if any.
  ageing_effect_on_purity = -0.4;  // Varnish made impure, whatever's left of it.
  ageing_effect_on_smooth =  0.0;  // Roughness of remaining specularity unaffected.
}
else // Clear passivated dielectric rust:
{
  ageing_effect_on_thick  = 10.0;  // Film thickness becomes 10 microns times Ageing.
  ageing_effect_on_mspec  =  0.2;  // Boosts specular to facilitate iridescence.
  ageing_effect_on_gloss  =  0.0;  // Refractive index is independent of thickness.
  ageing_effect_on_purity =  0.0;  // There should be no impurities.
  ageing_effect_on_smooth = -0.1;  // Smoothness affected in aluminium, but not chromium...
}

My head hurts.  That first conditional,

// Deny Gloss if it's metal and is not ageing in a passivated dielectric way...
// In other words, if it's a metal but its Aged_color == 1.0 and it IS ageing, then let there be Gloss on it
// or if it is a metal but it is a colored ruster, let there be Gloss on it, if so indicated,
// or if it is a non-metal then by all means allow a Gloss on it,
// but otherwise kill the gloss

  I can't translate the comments into pseudocode yet, not to speak of fuzzy float boolean ops ...  OUCH!

[DanW58]

Okay, getting my wits back together, slowly ...  Gyn seems to help, ironically.

The problem with the plain text conditional is that it is wordy, redundant;  it needs to be cleaned up

// Deny Gloss if it's metal and is not ageing in a passivated dielectric way...
// In other words, if it's a metal but its Aged_color == 1.0 and it IS ageing, then let there be Gloss on it
// or if it is a metal but it is a colored ruster, let there be Gloss on it, if so indicated,
// or if it is a non-metal then by all means allow a Gloss on it,
// but otherwise kill the gloss

can be simplified to ...

// If it is a metal
// {
//    Deny Gloss; //tentatively ... but ...
//    if( (aged_color == 1 && Ageing != 0) || aged_color != 1 )
//    {
//        UN-deny Gloss;
//    }
// }

Yes?

// If it is a metal
// {
//    Deny Gloss; //tentatively ... but ...
//    if( aged_color != 1 || Ageing != 0 )
//    {
//        UN-deny Gloss;
//    }
// }

Yes?

// If it is a metal
// {
//    if( !( aged_color != 1 || Ageing != 0 ) )
//    {
//        Deny Gloss;
//    }
// }

Yes?

// if( is_metal )
// {
//    if( aged_color == 1 && Ageing == 0 )
//    {
//        Deny Gloss;
//    }
// }

Yes?

if( is_metal  &&  aged_color == 1  &&  Ageing == 0 )
   Gloss = 0;

Yes?

float applied_Gloss = mix( nominal_Gloss, 0.0, is_metal * is_a_passivated_ruster * is_not_ageing_yet );

Yes?

So we can substitute that ...

// init defaults:
vec3  matte_rust_color = black;
float  ageing_throttle = 1.0;
float  ageing_effect_on_thick = 0.0;
float  ageing_effect_on_mspec = 0.0;
float  ageing_effect_on_gloss = 0.0;
float  ageing_effect_on_purity = 0.0;
float  ageing_effect_on_smooth = 0.0;
//if( is_metal  &&  aged_color == 1  &&  Ageing == 0 ) DenyGloss()!
float is_metal = clamp( 3.3*(Mat_MSpec-0.5)+0.5, 0.0, 1.0 );
float is_a_passivated_ruster = clamp( 7.7*(Mat_Aged_Color-0.85)+0.5, 0.0, 1.0 );
float is_not_ageing_yet = clamp( 9.9*(Mat_Aged_Color-0.15)+0.5, 0.0, 1.0 );
float applied_Gloss = mix( nominal_Gloss, 0.0, is_metal * is_a_passivated_ruster * is_not_ageing_yet );
// Yes?
if( Age_Color == 0.8 ) //NO color
{
  ageing_throttle = 0; //NO ageing
}
else_if( Age_Color < 0.7 ) // A real rust color:
{
  MatteRustColor = get_RGB_from_Age_Color( Age_Color );
  ageing_effect_on_mspec  = -0.9;  // Switch from specular to diffuse.
  ageing_effect_on_gloss  = -3.7;  // Peel off varnish first, if any.
  ageing_effect_on_purity = -0.4;  // Varnish made impure, whatever's left of it.
  ageing_effect_on_smooth =  0.0;  // Roughness of remaining specularity unaffected.
}
else // Clear passivated dielectric rust:
{
  ageing_effect_on_thick  = 10.0;  // Film thickness becomes 10 microns times Ageing.
  ageing_effect_on_mspec  =  0.2;  // Boosts specular to facilitate iridescence.
  ageing_effect_on_gloss  =  0.0;  // Refractive index is independent of thickness.
  ageing_effect_on_purity =  0.0;  // There should be no impurities.
  ageing_effect_on_smooth = -0.1;  // Smoothness affected in aluminium, but not chromium...
}

So, the final consequence of our convoluted fuzzy logic is that IF you want to represent varnished metal, or anodized aluminium, you will have to max-out ageing for the entire thing, which means you won't be able to age them, which should be fine because varnished metal and anodized aluminium can both last centuries.  But at least there IS a way to represent them.  In my old texture packing, 20 years ago, I was unable to represent varnished metals or anodized aluminum;  I was limited to either varnished diffuse materials (paints) or shiny bare metals.  So, at least here we have a way to represent these oddballs (a dielectric layer on top of a metallic specular surface), if in a bit of a roundabout way ...

Edited February 23, 2021 by DanW58

[DanW58]

Heck, NO!  What am I talking about?

The problem is actually MUCH smaller than what I just described.

You can have varnish over shiny metal all you want, as long as it is not a self-passivating ruster like aluminum or chromium.

You can varnish steel, you can varnish iron, or gold, without any hacks or tricks.

It is only in the case that you take a self-passivating, clear rust metal like chromium or aluminium, that, if you want to cover them in anodize or varnish, or in their own oxide, you can only do it by whitening (maxing out) the Ageing channel.

And you won't be able to specify a different Gloss (refractive index) from what their oxide is specified to be.  That's the only "problem".

EDIT:  Well, there are other subtle consequences.  When you are using the Ageing channel with a passivated clear ruster your Ageing value controls thickness, in a 0~10 micron range, which will give you iridescent effects when looking at a light reflection on the metal.  Regular "varnish" is treated as having zero thickness, therefore producing no chromatic effects.  This means that when you use the Ageing channel trick to represent anodized aluminium, you won't be able to NOT specify iridescent effects.  On the other hand, you will still be able to supress them by lowering the smoothness.  Low smoothness kills iridescence dead.

Edited February 23, 2021 by DanW58

[hyperion]

2 hours ago, DanW58 said:

EDIT: Also, don't forget, this mechanism doesn't prevent you from making your own rust using the normal texture channels;  it merely gives you a shortcut way to do it, that is easy to modify, and lessens the amount of work in creating custom materials with delineated rust zones that may not be useful to other objects.

So how are you gonna render it in blender?

 

2 hours ago, DanW58 said:

EDIT2:  Yet another advantage is that once the metal textures are created, each with its own Aged_Color, a misled or subversive artist cannot easily call for blue rust on iron, or red rust on gold.  Assuming your material textures are organized into larger "album" textures, I imagine there is some protocol to follow before someone can add a new member to the album, right?

Adding such an album to the shader doesn't scale. There are hundreds of materials used.

[DanW58]

6 hours ago, hyperion said:

So how are you gonna render it in blender?

Having a texture set, or before having one, you mean?

I can't imagine it will be problematic at all to have five texture input nodes, split their channels, and do the exact same math the shader does, using math nodes.

 

6 hours ago, hyperion said:

Adding such an album to the shader doesn't scale. There are hundreds of materials used.

I don't understand;  adding albums and shaders is like adding apples and bananas.

I don't know how 0ad organizes its textures and materials presently;  it may be that the second UV is being used ONLY for ambient occlusion, and that every model creates its own materials from scratch.  It's inefficient, but if people want to continue to do so, this shader will support their modus operandi;  it will not prevent it.

I wouldn't mind having a discussion on textures and materials organization.  And there are no hard and fast rules;  each game has its own predicaments to deal with.  Ultimately, you are after two things:  Minimizing texturing work-load, and minimizing video-memory use.  And there are secondary goals, such as minimizing texture units, minimizing context switches, minimizing texture loads and unloads ...   How to best serve these goals is a question with a million answers.  I suppose one solution I would look into is a per-faction materials library-texture.  This would be a Material Textures set (albedo.dds and optics.dds), say 4096 x 4096 divided into typically 8 x 8 = 64 regions, that about one quarter of it is pure materials, another quarter is factional doodads like standards, flags, paintings or hieroglyphs, another quarter is actual faction-specific art such as buildings and ships, and the last quarter is room for future additions.  This would take up a bit of room in the videocard, but has the advantage of avoiding loading and unloading many textures.  Eventually it would grow quite efficient due to reuse of parts of the texture.

In fact, pure materials, such as metals, don't need anywhere near 512x512 space;  being featurless, pure materials have no pixelation concerns, so if you have an item that is made of pure metal you can minimize its UV island into a tiny metal square.  Same goes for any pure material.

How an artist would work with this is, suppose you are about to begin work on a new, flying medical building for the Nephiim;  so you get a copy of the big Nephilim album texture, together with a slot for your custom things, say the upper right corner.  You can unwrap your model any way you want, but you can only add or modify textures in your assigned 512 x 512 lot.  So you see what's already in the album that parts of your medical center can use, and unwrap islands to those locations, and for things you need custom texturing you put them in your private lot.  Of course, you are working at the artist level texture set, probably at 2048, but each time you modify and want to test, you run a little app that takes your textures and packs them into your slot of the album.  However, your copy of the album is for testing purposes only;  the master texture is kept safe in the art repository.  Once you are done, you submit your model and the artist texture set at 2048, (as well as the second UV textures having the AO, emit and normalmap bakes, and zones), and the maintainer uses a similar little app to convert your textures and add them to the high quality master album, and produces a new scaled and compressed nephilim_albedo.dds and nephilim_optics.dds for the game repo.

This would be my solution;  but like I said, the shader won't insist that it be done this way.

 

EDIT:  Quite importantly, there needs to be a standard set for an exact scaling, i.e., for how many texels per yard are there in the final compressed texture.  Well not exact exact, as a surface can be spherical and refuse to flatten out nicely;  but none of that scale-by-eye should be permitted, as changes in the texture scaling across units is quite noticeable and quite odd-looking (except for items made of a pure material).  So, if it is a texel per inch or 50 texels per yard, but the number has to be agreed and adhered to across all assets, including terrains.  The second UV set, for the bakings, can have a considerably smaller pitch, say 20 texels per yard, but it should also be standardized.  (Sky boxes all have to be 1024, as I discovered about 10 posts ago in this thread.)

Edited February 23, 2021 by DanW58

[DanW58]

Finalizing the switchyard, just for now...

// THE SWITCHYARD

// external input variables:
uniform float age; // 0.0~1.0 mapping as from newborn to ancient

// external output variables:
float Mat_IOR;

// temporary variables and computations (in nameless namespace)
{
  float age_power = exp( 3.0*(age-0.5) );
  float pixel_age = pow( Ageing, age_power );
  vec3 matte_aged_RGB = vec3(0.25);
  matte_aged_RGB.r = 0.83 * Mat_Aged_Color; //to reach 0.5 at 0.6
  matte_aged_RGB.g = 1.11 * Mat_Aged_Color * Mat_Aged_Color; //to reach 0.4 at 0.6
  matte_aged_RGB.b = 0.0;
  matte_aged_RGB = mix( matte_aged_RGB, vec3(0.25), step( 0.8, Mat_Aged_Color ) );
  temp = 5.0*(Mat_Aged_Color-0.8)+0.5;
  float is_ageless_nonruster = clamp( 1.0 - temp*temp, 0.0, 1.0 );
  float is_colored_ruster = clamp( 3.5*(0.8-Mat_Aged_Color)+0.5, 0.0, 1.0 );
  float is_passivated_ruster = clamp( 1.0 - is_colored_ruster - is_ageless_nonruster, 0.0, 1.0 );
  float is_metal = clamp( 3.3*(Mat_MSpec-0.5)+0.5, 0.0, 1.0 );
  float is_not_ageing_yet = clamp( 9.9*(Mat_Aged_Color-0.15)+0.5, 0.0, 1.0 );
  float applied_Gloss = mix( nominal_Gloss, 0.0, is_metal * is_a_passivated_ruster * is_not_ageing_yet );
  // A Dark Forest of Magic Numbers:
  float spec_age_factor     = mix( 1.0, mix( mix( 1.0, 0.0, is_colored_ruster ), 1.0, is_ageless_nonruster ), pixel_age );
  float smooth_age_factor   = mix( 1.0, mix( mix( 0.4, 0.1, is_colored_ruster ), 0.6, is_ageless_nonruster ), pixel_age );
  Obj_Microns       = Obj_Microns       * spec_age_factor;
  Mat_MSpec         = Mat_MSpec         * spec_age_factor;
  Mat_Gloss         = Mat_Gloss         * spec_age_factor;
  Mat_Purity        = Mat_Purity        * spec_age_factor;
  Mat_SpecularPower = Mat_SpecularPower * smooth_age_factor;
  float Mat_IOR = ( temp4.b * 4.0 ) + 1.0; // Gloss to IOR.

I say "just for now" because I think the switchyard will grow.  I think whenever I need some new kind of fuzzy boolean I'm just going to throw the code into the switchyard.  I once read an article in a software magazine, can't remember which, about the problem with "boilerplate code".  The author was saying that basically every application has to deal with ugly, convoluted situations.  He advocated trying to put all the ugly code into a single file, if possible.  So, in the same spirit, the switchyard here will collect most ugly things, and allow the other parts of the pipeline look clean and elegant.

Dam!  I had COMPLETELY forgotten that functions could write to parameters by qualifying them as "out" or "inout".

So, I can make a routine of this, as well as of the texture loading operations.  I'm going to rewrite a lot of stuff and post an update of the whole shader so far in the next post.

Edited February 23, 2021 by DanW58

[DanW58]

Deleted, as the website software won't let me edit it anymore. Copied to the next post.

Edited February 24, 2021 by DanW58

[DanW58]

One tricky thing that's just come up again, is in the application of ambient light by fetching it from the skybox.

In some earlier post I said that, basically,

incident_ambient = ao * textureCube(skyCube, v3_raw_normal, LODbias_from_AO( Obj_AO ) ) );

There's an issue with that:   Suppose the ao is almost black for that point.  This indicates a great deal of occlusion;  narrow solid angle;  but in which direction?

The traditional answer would be "who knows?  Just assume it's the normal!".  But we DO know something very important!  However narrow the solid angle of visibility may be, it MUST include the camera, the eye-vector, that is, OUR direction, because if it didn't we wouldn't be seeing the very point;  we wouldn't be processing it.  So, it would be more correct to fetch the portion of sky that the "bent normal" (bent towards the camera) sees, rather than the raw normal.  Now, you might say "bah, it's probably the same color anyways!  A cloud is a cloud!".  Well, not so fast:  the sky box MAY include distant mountains and whatnot.  But there's another issue:  ambient light usually does not need to compute dot(ray,normal) because it is omnidirectional,  but in the case of a narrow solid angle occlusion and a bent normal, perhaps we should.

And the issue doesn't even end with view vector concerns:  Suppose we've calculated our bent normal.  Now, if the cone of visibility includes the ground, it probably should be bent up a little, as the ambient occlusion (if properly baked, with a ground occluder) should not be adding light rays coming from below the ground.  So the view direction ought to be from slightly more upwards than the normal.

Correcting these subtle issues will not only improve the quality of ambient lighting;  it will also improve the quality of our specular occlusion hack, which is far more noticeable.

So, how do we bend the normals to include the eye vector and exclude ground?

We have a formula for angular radius from AO, namely,

ao = 1 - cos(radius)     and the reciprocal     cos(radius) = 1 - ao

No need to solve for radius, since we'll be comparing cosines, rather than angles.

So, if the angle between the normal and the eye vector is greater than the visibility radius, we want to bend the normal.  And cosines go opposite way as absolute angle, so the logic reverses:  if( dot(eyevec,normal) < 1-ao ) { bend_normal(); }

Now, GPU's don't like conditional execution;  nor do I, frankly;  so we need to find a continuous function that bends the normal as needed, or more than needed sometimes, to ensure that a narrow vis cone includes the eye, but a wide cone is not disturbed.

vec3 bent_normal = normalize( mix( -eye_vec, normal, some_factor ) );

Now we just need to compute "some_factor".  A value of 0 will bend the normal completely towards us.  A value of 1 not at all.

The only situation we would need a value of 0.0 is when the ao is 0.0.  So, ao will probably be a multiplicative factor in some_factor.

More generally, if the radius from the ao is MUCH smaller than arccos(dot(-eye,normal)), in other words, if aoradius/eyenormangle is a small number we need a small factor.  Cosines reverse the logic, so, if cos(eyenormangle)/cos(aoradius) is a small number we need a small factor.  This would translate to,

some_factor = f{  dot(-eyevec, normal)/(1-ao)  }  //some relinearizing function of

Now we need to find what kind of linearity we are getting.

No, no need to test that;  I know it won't work.  For small angles, cosines are always close to 1;  they won't give me substantial factors.  I will try sines, as a last resort;  but first I want to try 1-cos terms.  The logic will reverse again, so...

some_factor = f?{     ao/(1.0-max(0.0, dot(-eyevec, normal)))    }

Alright, let's try a few values, see what happens.  First the extremes, such ao = 1, ao = 0, dot = 1, dot = 0...  At dot = 1 we get infinity;  not a good sign...  With ao = 0 we get 0, which is good.

With ao = 1, if dot is 0, we get 1 which is good!  ... ao = 1 means 90 deg vis; no need to correct even at 90 degree viewing.

With ao = 1, if dot is 0.5, we get a factor 2.0, which is weird;  means not only we don't bend the normal, but bend it away from us?!?!?!  Could be corrected by clamping the result to 1.0, perhaps;  let's continue:

With ao = 0.5 (which is 60 degree angle), if dot is 0.5 (60 degree angle also), we get no bending, which is okay, edge case.

Okay, ao = 1-cos(r), so 1-cos(dot) will ensure all our edge cases yield 1.0 --no correction.  We are making progress!  So what about when the edge case is exceeded?, such as ao=0.5 but dot=0.6 ? That yields .5/.4 = 1.25.  This is sort-of correct, in the sense that acos(0.6) is 53 degrees, which is smaller than 60 degrees.  All we need is a clamp now:

some_factor = clamp( ao/(1.0-max( 0.0, dot(-eyevec, normal) )), 0.0, 1.0 );

Now, if ao = 0.5 and dot = 0.4, factor = 5/6 = 0.8333.  Is this good?  Let's see:  arccos(0.4) = 66.42 degrees.  If the ao radius is 60 degrees, it does not include the camera, but must move towards us by 6.42 degrees, which is almost 10% change in angle.  However, the scaling is linear in vector space;  not in angle, and the tangent of 66.42 degrees is 2.3, so we are talking about a 23% move to give a 10% angular move.  Anyways, it looks pretty right to me;  not perfect, but pretty darn close!  I think this is a GO!

Now we need to repel the ground, somehow.  Won't happen too much, as the camera is above the ground, and already bent the normal towards itself, in many cases;  but in case of ao = 1 and a normal not pointing up, we have a problem.  Fortunately, I think we only need to look at the horizontal component of the normal and compare it to 1-ao.

So, here again we have a mix of normal and now an up vector,

bent_normal = mix( up_vector, normal, some_factor );

Except, "some_factor" now has to repel ground, rather than pull towards the eye vector.  How do we do this?

Note that when the normal's angle to the ground is less than the ao radius, that's when bending up is needed.

In cosine terms, the logic reverses:  When cos(ao radius) < cos(normal to ground), bend up.

In 1-cos terms again, the logic reverses yet again:  when 1-cos(norm-to-gnd) < 1-cos(aoradius), bend up.

But we know that 1-cos(aoradius) = ao, so,  when  (1-cos(norm-to-gnd)) < ao,  bend up.

And we know that cos(norm-to-gnd) is nothing but sqrt(norm.x*norm.x + norm.z*norm.z), so

when  (1.0-sqrt(norm.x*norm.x + norm.z*norm.z)) < ao, bend up.  Or ...

when sqrt(norm.x*norm.x + norm.z*norm.z) > 1.0 - ao, bend up

Now, to "bend up" we need a factor less than 1, so,

factor = clamp( (1.0 - ao) / sqrt(norm.x*norm.x + norm.z*norm.z), 0.0, 1.0 );

Let's see how this works:

Edge cases first:  ao = 0 yields factor = 1 or no-correction.  This is incorrect;  it is not considering downward-facing normal cases. Back to the drawing board...

Wait!  All we need to do is make sure that sin(normal-to-ground) is no less than sin(aoradius)!!!
 

bent_normal = normal.
bent_normal.y = max( normal.y, sqrt(1.0 - (1.0-ao)(1.0-ao)) );
bent_normal = normalize( bent_normal );

What's wrong with that?

So now we put the two bends together.  Away from ground first:
 

vec3 bent_normal = normal.
bent_normal.y = max( normal.y, sqrt(1.0 - (1.0-ao)(1.0-ao)) );
bent_normal = normalize( bent_normal );
float factor = clamp( ao/(1.0-max( 0.0, dot(-eyevec, bent_normal) )), 0.0, 1.0 );
bent_normal = normalize( mix( -eye_vec, bent_normal, factor ) );

DONE! 

Edited February 24, 2021 by DanW58

[DanW58]

This is the shader so far:

#version 777

#include "common/fog.h"
#include "common/los_fragment.h"
#include "common/shadows_fragment.h"

#if USE_OBJECTCOLOR
  uniform vec3 objectColor;
#else
#if USE_PLAYERCOLOR
  uniform vec3 playerColor;
#endif
#endif

// Textures:
uniform sampler2D TS_albedo;
uniform sampler2D TS_optics;
varying vec2 UV_material;  // First UV set.
//~~~~~~~~~~~
uniform sampler2D TS_forms;
uniform sampler2D TS_light;
uniform sampler2D TS_zones;
uniform sampler2D TS_detail;
varying vec2 UV_object;  // Second UV set.
//~~~~~~~~~~~
uniform samplerCube skyCube;

// Vectors:
uniform vec3 v3_sunDir;
varying vec4 v4_normal;
varying vec3 v3_eyeVec;
varying vec3 v3_half;
varying vec4 v4_tangent;
varying vec3 v3_bitangent;
varying vec4 v4_lighting;

// Colors:
uniform vec3 sunColor;
uniform vec3 gndColor;


// SUBROUTINES:

vec3 renormalize( vec3 input ) // For small errors only, faster than normalize().
{
  return input * vec3( 1.5 - (0.5 * dot(input, input)) );
}

float LOD_bias_from_spec_power( float spec_power )
{
  return clamp( 8.0 - ( 0.5 * log2(spec_power) ), 0.0, 7.78 );
}

float LOD_bias_from_AO( float AO )
{
  return clamp( 8.0 + (0.4373 * log2(AO)), 0.0, 7.78 );
}

void separate_albedo( vec3 albedo, float mspec, out vec3 diffuse, out vec3 specular )
{
  float mdiff = 1.0 - mspec;
  vec3 diff = albedo * albedo; // Boost saturation.
  vec3 spec = sqrt( albedo ); // Wash saturation.
  diff = ( (mdiff * diff) + (mspec * albedo) ) * mdiff;
  spec = ( (mspec * spec) + (mdiff * albedo) ) * mspec;
  vec3 temp3 = albedo / (diff + spec); // Renormalize results.
  diffuse = diff * temp3;
  specular = spec * temp3; // Done!
}

vec3 get_bent_normal__away_from_gnd_and_towards_camera( vec3 normal, vec3 eyevec, float ao )
{
  vec3 temp3 = normal.
  temp3.y = max( normal.y, sqrt(1.0 - (1.0-ao)(1.0-ao)) );
  temp3 = normalize( temp3 );
  float factor = clamp( ao/(1.0-max( 0.0, dot(-eyevec, temp3) )), 0.0, 1.0 );
  return = normalize( mix( -eye_vec, temp3, factor ) );
}

float vector_hits_the_ground( vec3 vector, float cos_spot_radius )
{
  return smoothstep( -cos_spot_radius, +cos_spot_radius, -vector.y );
}

float reflection_unobstructed( vec3 viewvec, vec3 normvec, float AO, float cos_spot_radius )
{
  float cos_halfangle = 1.0 - AO;
  float cos_view = max( 0.0, dot(viewvec, normvec) );
  return smoothstep(cos_halfangle-cos_spot_radius, cos_halfangle+cos_spot_radius, cos_view );
}

vec3 SchlickApproximateReflectionCoefficient( float rayDotNormal, float From_IOR, float To_IOR )
{
  float R0 = (To_IOR - From_IOR) / (From_IOR + To_IOR);
  float angle_part = pow( 0.9*(1.0-rayDotNormal), 5.0 );
  R0 = R0 * R0;
  float RC = R0 + (1.0 - R0) * angle_part;
  return vec3(RC, RC, RC); // Returns Fresnel refl coeff as gray-scale color.
}

vec3 incident_specular_light
(
  vec3  normal,
  vec3  groundColor,
  vec3  diffuse_col,
  vec3  ao,
  vec3  ambient,
  float matspecpwr,
  float unblocked_specular,
  float refl_ground
)
{
  vec3  eyevec = normalize(v_eyeVec);
  vec3  refl_view = -reflect( eyevec, normal );
  vec3  hafvec = normalize(v_half);
  float spec_pwr = max(1.0, matspecpwr);
  float point_is_in_shadow = 1.0-getShadow();
  float refl_view_dot_sun = dot(refl_view, sunDir);
  float blocker_in_sun = 0.5 + (refl_view_dot_sun * 0.5);      // Just a likelihood...
  blocker_in_sun = blocker_in_sun * (1.0-pow( max(0.0,refl_view_dot_sun),spec_pwr)); // We are shadowing blocker!
  float Phong_coefficient = pow( max(0.0, dot(normal, hafvec)), spec_pwr );
  float brightness_adj = 0.5 / ( 1.0 - pow(0.5, (1.0/spec_pwr)) );   // Sharper reflections do look brighter.
  vec3  light_on_blocker = mix( ambient*ao*ao, sunColor, blocker_in_sun*blocker_in_sun );
  vec3  spec_blocker_color = diffuse_col * light_on_blocker; // Assuming blocker's same material as this.
  vec3  spec_from_sun = mix( sunColor * brightness_adj, spec_blocker_color, 0.0*point_is_in_shadow );
  vec3  spec_from_env = mix( spec_blocker_color, ambient, unblocked_specular );
  spec_from_env = mix( spec_from_env, ambient * groundColor, refl_ground );
  return mix( spec_from_env, spec_from_sun, Phong_coefficient);
}

vec3 specularly_reflected_light( vec3 mspeccol, vec3 fresnelcolor, float is_metal, vec3 incident_light )
{
  vec3 final_specular_color = mix(fresnelcolor, mspeccol, is_metal);
  return final_specular_color * incident_light;
}

vec3 incident_diffuse_light
(
  vec3  groundColor,
  vec3  fresnelcolor,
  vec3  ao,
  vec3  ambientfetch,
  float bent_normal_dot_normal,
  float normal_hits_ground,
  float ray_dot_normal
)
{
  vec3 FresnelRefractFactor = vec3(1.0) - fresnelcolor;
  FresnelRefractFactor = FresnelRefractFactor*FresnelRefractFactor; // In and out.
  vec3 incident_ambient = ao * bent_normal_dot_normal * mix( ambientfetch, groundColor, normal_hits_ground );
  vec3 incident_direct = vec3(ray_dot_normal*getShadow()) * sunColor * FresnelRefractFactor;
  return incident_ambient + incident_direct;
}

vec3 diffusely_reflected_light( vec3 diffuse, vec3 incident_diff_light )
{
  return  diffuse * incident_diff_light;
}

void main()
{
  vec4 temp4;
  vec3 temp3;
  vec2 temp2;
  float temp;


  // MATERIAL TEXTURES:

  // Load albedo.dds data:
  temp4 = texture2D( TS_albedo, UV_material );
  vec3  Mat_RGB_albedo = temp4.rgb; // To be split into diffuse and specular...
  float Mat_alpha = temp4.a;

  // Load optics.dds data:
  temp4 = texture2D( TS_optics, UV_material );
  float Mat_MSpec  = temp4.r; // Metallic specularity % (vs diffuse).
  float Mat_Purity = temp4.g;
  float Mat_Gloss  = temp4.b;
  float Mat_SpecularPower = 1.0 / min( 1.0 - temp4.a, 1.0/256.0 );
  Mat_SpecularPower = SpecularPower * SpecularPower; // Smoothness.
  Mat_cos_spot_radius = pow( 0.5, 1.0/Mat_SpecularPower );


  // OBJECT TEXTURES:

  // Load forms.png data:
  temp4 = texture2D( TS_forms, UV_object );
  vec3 Obj_NM_normal = normalize( vec3(2.0, 2.0, 1.0) * ( temp4.rgb - vec3(0.5, 0.5, 0.0) ) );
  float Obj_ParallaxHeight = temp4.a;  // Any scaling needed?

  // Load light.dds data:
  temp4 = texture2D( TS_light, UV_object );
  vec3 Obj_RGB_emmit = temp4.rgb; // Emissive + self-lighting.
  float Obj_AO = temp4.a;  // Occlusion.
  vec3 Obj_RGB_ao = vec3( temp4.a ); // AO as color, for convenience.

  // Load zones.dds data:
  temp4 = texture2D( TS_zones, UV_object );
  float Obj_is_Faction = temp4.r; // Where to put faction color.
  float Obj_Microns = 10.0 * temp4.g; // Thickness of oxide film.
  float Obj_AO_detailMod = 1.0 - min( temp4.b * 2.0, 1.0 );
  float Obj_SP_detailMod = max( 0.0, temp4.b * 2.0 - 1.0 );
  float Obj_Alpha = temp4.a;

  // Load detail.dds data, and apply it:
  temp3 = texture2D( TS_detail, UV_object * vec2(11.090169945) );
  temp = dot( temp3, vec3(1.0) );
  Obj_RGB_ao = Obj_RGB_ao + vec3(0.0625 * Obj_AO_detailMod) * (temp3 - vec3(0.5));
  Mat_SpecularPower = Mat_SpecularPower * ( 1.0 + ( 0.0625 * Obj_SP_detailMod * (temp-0.5) ) );


  // VECTORS AND STUFF:

  // Sanitize and normalize:
  // v3_sunDir should not need renormalization
  vec3 v3_raw_normal = renormalize( vec3( v4_normal ) );
  vec3 v3_eye_vector = renormalize( v3_eyeVec );
  vec3 v3_half_vec   = renormalize( v3_half );
  // Tangent stuff ... I know nothing about it.
  // Normal-map-modulated normal:
  vec3 v3_mod_normal = renormalize( v3_raw_normal * Obj_NM_normal );
  vec3 v3_refl_view = -reflect( v3_half_vec, v3_mod_normal );
  // These numbers are precomputed, as they will be needed more than once:
  float upwardsness = v3_raw_normal.y;
  float rayDotNormal = max( 0.0, dot( -v3_sunDir, v3_mod_normal ) );
  float eyeDotNormal = max( 0.0, dot( v3_eye_vector, v3_mod_normal ) );
  vec3 fresnel_refl_color = SchlickApproximateReflectionCoefficient( eyeDotNormal, 1.0, Mat_IOR );
  // Compute a bent normal for ambient light

  // STUFF I KNOW NOTHING ABOUT:
  #if (USE_INSTANCING || USE_GPU_SKINNING) && (USE_PARALLAX || USE_NORMAL_MAP)
    vec3 bitangent = vec3(v4_normal.w, v4_tangent.w, v4_lighting.w);
    mat3 tbn = mat3(v4_tangent.xyz, bitangent, v4_normal.xyz);
  #endif
  #if (USE_INSTANCING || USE_GPU_SKINNING) && USE_PARALLAX
  {
    float h = Obj_ParallaxHeight;
    vec2 coord = UV_object;
    vec3 eyeDir = normalize(v_eyeVec * tbn);
    float dist = length(v_eyeVec);
    vec2 move;
    float height = 1.0;
    float scale = effectSettings.z;
    int iter = int(min(20.0, 25.0 - dist/10.0));
    if (iter > 0)
    {
      float s = 1.0/float(iter);
      float t = s;
      move = vec2(-eyeDir.x, eyeDir.y) * scale / (eyeDir.z * float(iter));
      vec2 nil = vec2(0.0);
      for (int i = 0; i < iter; ++i)
      {
        height -= t;
        t = (h < height) ? s : 0.0;
        temp2 = (h < height) ? move : nil;
        coord += temp2;
        h = texture2D(TS_forms, coord).a;
      }
      // Move back to where we collided with the surface.
      // This assumes the surface is linear between the sample point before we
      // intersect the surface and after we intersect the surface.
      float hp = texture2D(TS_forms, coord - move).a;
      coord -= move * ((h - height) / (s + h - hp));
    }
  }

  // ALBEDO AND THINGS:
  vec3 Mat_RGB_diff, Mat_RGB_spec;
  separate_albedo( Mat_RGB_albedo, Mat_MSpec, Mat_RGB_diff, Mat_RGB_spec );
  Mat_RGB_diff = mix( Mat_RGB_diff, playerColor, Obj_is_Faction );
                     
  // CUBE MAP FETCHINGS:
  vec3  RGBlight_reflSky = vec3( textureCube(skyCube, v3_reflView, LODbias_from_spec_power( Mat_specular_power ) ) );
  vec3  v3_bent_normal = get_bent_normal__away_from_gnd_and_towards_camera( v3_raw_normal, v3_eye_vector, Obj_AO )
  float bent_normal_dot_normal = dot(v3_bent_normal, v3_raw_normal);
  vec3  RGBlight_normSky = vec3( textureCube(skyCube, v3_bent_normal, LODbias_from_AO( Obj_AO ) ) );


  // THE SWITCHYARD

  // external input variables:
  uniform float age; // 0.0~1.0 mapping as from newborn to ancient

  // external output variables:
  float Mat_IOR;

  // temporary variables and computations (in nameless namespace)
  {
    float age_power = exp( 3.0*(age-0.5) );
    float pixel_age = pow( Ageing, age_power );
    vec3 matte_aged_RGB = vec3(0.25);
    matte_aged_RGB.r = 0.83 * Mat_Aged_Color; //to reach 0.5 at 0.6
    matte_aged_RGB.g = 1.11 * Mat_Aged_Color * Mat_Aged_Color; //to reach 0.4 at 0.6
    matte_aged_RGB.b = 0.0;
    matte_aged_RGB = mix( matte_aged_RGB, vec3(0.25), step( 0.8, Mat_Aged_Color ) );
    temp = 5.0*(Mat_Aged_Color-0.8)+0.5;
    float is_ageless_nonruster = clamp( 1.0 - temp*temp, 0.0, 1.0 );
    float is_colored_ruster = clamp( 3.5*(0.8-Mat_Aged_Color)+0.5, 0.0, 1.0 );
    float is_passivated_ruster = clamp( 1.0 - is_colored_ruster - is_ageless_nonruster, 0.0, 1.0 );
    float is_metal = clamp( 3.3*(Mat_MSpec-0.5)+0.5, 0.0, 1.0 );
    float is_not_ageing_yet = clamp( 9.9*(Mat_Aged_Color-0.15)+0.5, 0.0, 1.0 );
    float applied_Gloss = mix( nominal_Gloss, 0.0, is_metal * is_a_passivated_ruster * is_not_ageing_yet );
    // A Dark Forest of Magic Numbers:
    float spec_age_factor     = mix( 1.0, mix( mix( 1.0, 0.0, is_colored_ruster ), 1.0, is_ageless_nonruster ), pixel_age );
    float smooth_age_factor   = mix( 1.0, mix( mix( 0.4, 0.1, is_colored_ruster ), 0.6, is_ageless_nonruster ), pixel_age );
    Obj_Microns       = Obj_Microns       * spec_age_factor;
    Mat_MSpec         = Mat_MSpec         * spec_age_factor;
    Mat_Gloss         = Mat_Gloss         * spec_age_factor;
    Mat_Purity        = Mat_Purity        * spec_age_factor;
    Mat_SpecularPower = Mat_SpecularPower * smooth_age_factor;
    float Mat_IOR = ( temp4.b * 4.0 ) + 1.0; // Gloss to IOR.

Decided against subroutinizing the switchyard... Too many paramenters.

[Stan`]

#version 777 ?

Needs to be 130 or whatever we use else some drivers will crash because unknown opengl version iirc

[DanW58]

Ahhh,  that's OpenGL version!  I wondered why it was in # instead of //.  I must have seemed pessimistic about delivery.  Thanks.

Stan, by the way, do we use LOD textures at all?  I was trying to improve the blurriness of water by progressingly adding LOD bias to texture reads of the bottom terrain as depth increases, but found that the bias has no effect.

Edited February 24, 2021 by DanW58

[Stan`]

We have mipmaps in DDS files if that's your question.

Might or might not point you somewhere.

https://code.wildfiregames.com/D3023

https://code.wildfiregames.com/D3020

[DanW58]

Nice work!  I'm browsing through the terrain textures.  They are awsome!  I did find the cause of the Acropolis terrain being so bright problem, though:  medit_city_pavement.png is pretty bright, and medit_city_pavement_spec.png, though itself not too bright, brings the total light, the albedo, over the top.  Best thing would be to get rid of medit_city_pavement_spec.png.  The only way any specularity can work with such rough material is by having a super-low specular power;  and a physics based shader would dial down the brightness of specularities with low spec power, but most shaders don't.  (The more polished and smooth a surface is, the smaller the specular spot from a light source, but by the same token it is more concentrated;  the converse is also true;  but no shaders except mine account for that.)  You should take it out;  replace it with a black 1-texel texture;  and save it for when there's a better shader.

The skyboxes, I see there's 6 .dds textures in each folder.  I thought they'd be packed into a single file, but maybe I'm misremembering.

 

 

[Stan`]

They are brought together by code. I think we create a GL cube that's dependant of camera position

[DanW58]

24 minutes ago, Stan` said:

They are brought together by code. I think we create a GL cube that's dependant of camera position

Interesting!  Another question for you, Stan, I'm looking for where the maps are;  I can't find them.  I find the textures used in the maps, but not the maps themselves.  Unless they are the dds textures --I don't have any tools to open them yet, gotta get them...

[Stan`]

dds texture are images files (I believe gimp can open them) 

Maps are in binaries/data/mods/public/maps/ 

https://trac.wildfiregames.com/browser/ps/trunk/binaries/data/mods/public/maps

There are three types of maps, Random (Procedurally generated using javascript), Skirmishes, and Scenarios. The last too are mostly the same apart from the fact they are in a different folder.  Some maps have trigger scripts, which are scripts used to generate specific actions (eg spawning units regularly, or adding a different victory condition)

"Normal" maps use an XML file and a PMP file (The xml contains map settings, lighting, water height, entity positions). The pmp contains height data, and texture data.

wiki:PMP_File_Format

[DanW58]

16 minutes ago, Stan` said:

dds texture are images files (I believe gimp can open them)

There's a plug-in for GIMP.  I downloaded the compressonator suite, but it comes in a .tar.gz with ABSOLUTELY no instructions on what to do with it.  I uncompressed it into a folder but can't get it to work.  I guess I'll get the GIMP plug-in.

So now I'm starting to understand.  I thought there'd be a humongous texture somewhere, for each map.  So now the question is, what the engine puts together for a terrain, at run-time, does that have LOD's?

Edited February 24, 2021 by DanW58

[Stan`]

6 minutes ago, DanW58 said:

So now I'm starting to understand.  I thought there'd be a humongous texture somewhere, for each map.  So now the question is, what the engine puts together for a terrain, at run-time, does that have LOD's?

No the map references the individual terrain texture actors in https://trac.wildfiregames.com/browser/ps/trunk/binaries/data/mods/public/art/terrains

Those terrain texture actors reference the actual textures in https://trac.wildfiregames.com/browser/ps/trunk/binaries/data/mods/public/art/textures/terrain

Each of those textures has mipmaps (if they are png a dds file is generated in your cache folder) auto mipmap generation bv NVTT is pretty bad though and @vladislavbelov wants to fix it.

 

[DanW58]

Thanks, Stan!  I've learned so much today...

So when I tried to use LOD bias for ocean bottoms, why did it not work?  Hrmmm....

 

[vladislavbelov]

53 minutes ago, DanW58 said:

Thanks, Stan!  I've learned so much today...

So when I tried to use LOD bias for ocean bottoms, why did it not work?  Hrmmm....

 

You mean refraction/reflection maps?