A "psychic" shader mod; development begins...

[DanW58]

@Stan`  No;  this is not working.

I think the problem is that human skin is painted too light;  almost white.  Obviously, if I don't want to see saturation with an item that's too light, I have to darken everything too much.  It's a hard choice.  See the trees too dark?, or see the women's arms saturatingly white?  I'd rather make the trees dark.

Human skin should be darker in diffuse.  It is lightened up by the sheen of our epidermis index of refraction, and my shader will give you that;  but the diffuse tone needs to be darker, otherwise I'm adding white sheen to white base.

But actually, some of the people models have proper skin colors;  I think the Romans are one of the few that don't.

I'll put this bad example belolw, then go looking for which are the civs with good skins, see how this gamma thing works with them.

 

[DanW58]

Now I've removed all of the gamma stuff;  put back the detection hacks to what they were before, and I still can't get rid of saturating white arms.  Why can't I get back to how it was before?  This is too frustrating.

Anyways, detecting metals was trivial compared to detecting human skin here;  there is just no way of telling skin from roof, wall or piece of wood, or a rock.  If I give skin a sheen, everything gets a sheen.

It looks like I'm hitting the limit of what this shader can do.

I mean, I could add a green detector for plants;  that would be a lot easier.

And I could use the sky box as environment map;  but it is probably not an LOD texture that I can bias the fetch to get blurry reflections;  and without control of blurriness of reflections, environment mapping is a joke.  Could try it anyways...

 

[DanW58]

@Stan`  This is the brightest I can go while still seeing sunshine glow on the women's arms.

It is actually quite a bit brighter than before the gamma fiasco.  If that's not bright enough for you, here's the deal:  make human skin a bit darker, so that it doesn't saturate in the sunshine, and I'll give you more light.  Deal?

I'm trying to get the skybox to work...

 

Edited February 17, 2021 by DanW58

[DanW58]

Gentlemen,  I declare this shader finished.

I wanted to put the skybox fetch, but I don't know how to pass the info to the shader.  I copied the code from the water shader, and so it should work, but it is not getting the skybox and the rotation;  and my C++ sources are nowhere near compiling state;  I interrupted the engine work I was doing to work on shaders;  and I no longer remember where I was at.  In any case, the sky box was not going to help much, unless it has LOD's to bias into, to control blurriness of reflections.

Then there is the green-detection hack, which should have worked but this common_model shader does not seem to have jurisdiction over too much foliage, apparently.  I don't know what shader does the trees and stuff.

So I removed the green hack, and the skybox;  all I have is

• metals
• blue dresses
• human skin

And that's it.  It works.  It does have a few bugs I cannot resolve, like the fact that roofs often acquire glossy varnish as the skin detection hack gets confused by the similar color to skin.  Overall, though, it's darned good.  What makes it so good is the correctness.  Even down to the detail that specular highlights look brighter the smaller they are;  this is something I have NEVER seen anybody else take into account.  The Fresnel calculation is based on Schlick's approximation, which is a science level approximation;  not some cheap graphics hack.  The lack of skybox is not an issue at all, in my opinion.  The reflective items are small and curvy;  you don't have big mirrors here;  it's not like the water, where you need to see clouds reflected.  This is small objects like pots or helmets reflecting;  a sky-biased ambient light is plenty good for that, specially WITH self-occlusion and ground occlusion hacks.  And we got variations in dielectric constant and in shininess, albeit all ad-hoc values, like 1.5 index of refraction for skin, versus 1.0 for everything else.  Specular power I take over a lot of the time, increasing it for metals, decreasing it for non-metals, setting it to 17.0 for skin, and to 2.0 for blue dresses.

That's nowhere nearly as good as being able to specify these things via texture channels, of course;  but that's the next shader.  So I invite you all to try it, discuss it, give me feedback, so we can finalize this and move to the the big one.

You need to apply a patch first, then replace the shaders attached below.  The patch is attached to this post here:

And you may also have to add a line to your config for gpu_skinning for the ao to work, I'm not sure because according to Stan it should make no difference, but if that's not what made the difference for me seeing the ao's in game, I don't know what did.

model_common.fs model_common.vs terrain_common.fs

Edited February 17, 2021 by DanW58

[hyperion]

2 hours ago, DanW58 said:

blue dresses

Supposedly they are player_colour, maybe shouldn't be touched.

[DanW58]

2 minutes ago, hyperion said:

Supposedly they are player_colour, maybe shouldn't be touched.

Gone.  Took me about 20 seconds

It was hard to notice, anyways.

model_common.fs model_common.vs terrain_common.fs

[Stan`]

Might be worth showing you this. You asked me a few times what was my strange mode

It's Atlas the map editor! That's where you can test things like skyboxes and other funny settings. Such as HDR DOF and bloom Which have been messing with your screenshots . There once was an attempt to improve all this mess  but the patch was too big https://code.wildfiregames.com/D1486

 

39 minutes ago, DanW58 said:

Gone.  Took me about 20 seconds

It was hard to notice, anyways.

model_common.fs 11 kB · 0 downloads model_common.vs 4 kB · 0 downloads terrain_common.fs 4 kB · 0 downloads

Would be nice to have it as a patch on Phab

[DanW58]

12 minutes ago, Stan` said:

Might be worth showing you this. You asked me a few times what was my strange mode

It's Atlas the map editor! That's where you can test things like skyboxes and other funny settings. Such as HDR DOF and bloom Which have been messing with your screenshots . There once was an attempt to improve all this mess  but the patch was too big https://code.wildfiregames.com/D1486

 

Would be nice to have it as a patch on Phab

Cool project;  I many times asked for something of the sort from the Vegastrike devs.

Should take no time at all to come up with a patch...  I'll be back...

[Panther]

terrain_common.fs + model_common.vs only with a custom bloom as I personally don't like the metallic effect that gets created on the roofs.
 

Spoiler

 

 

Here some pictures using terrain_common.fs + model_common.vs + model_common.fs:

Spoiler

 

Edited February 17, 2021 by Panther
Attached some more pictures

[DanW58]

Thanks.  Just to be pedantic, the specular effect on the roofs is the opposite of metallic;  it is the addition of Fresnel specularity that results from detecting skin for Fresnel purposes, so what the roofs suffer from is more like a varnish, or clear epoxy coat.

But I don't like it either  ;-)  But OTOH I love to see gloss on human skins;  and can't distinguish roofs from skins.  Actually, I haven't looked at specular;  maybe there's a way...

By the way, sorry for the delay coming up with a patch;  I wanted to do an svn up first, and I got a nasty conflict I'm trying to resolve.  It will be in by tonight, though.

Edited February 17, 2021 by DanW58

[DanW58]

50 minutes ago, Panther said:

as I personally don't like the metallic effect that gets created on the roofs

I have good news, I was able to improve skin detection to exclude roofs.  In the pic below, white means is_skin(), black means it is not.  Just when I thought the shader was finished...  Tonight's patch will fix the roofs.

[DanW58]

The patch has arrived.  Roof brilliance artifact is about 90% removed.  Patch diffed from the /binaries folder.  I call it "prohibited" because the intent of this patch is to make the best of existing art assets, but is to be discouraged as a target shader for new art.

EDIT:  Rather than submitting a new patch to Phabricator, I updated my previous patch, D3555.

https://code.wildfiregames.com/D3555

 

 

Edited February 18, 2021 by DanW58

[DanW58]

Stan left some comments for me at the D3555 patch, mostly style things to fix, comments, blank space.

The one change I wasn't able to make is diffing from the root folder;  when I do the diff includes gazillions of C++ files I was working on in the sources folder coming to 4MB and change, which the Phabricator rejects.  There must be a way to avoid that, but I'm no svn expert.

For whatever's worth, here's the updated patch, diffd again from /binaries, but cleaned up and with the civic center removed;  wasn't important.  And I include the shaders too, for good measure.

EDIT, re-uploading;  fixed everything, diffed from root folder.  Updated D3555 patch again.

 

prohibited_shader.patch terrain_common.fs model_common.fs model_common.vs

Edited February 18, 2021 by DanW58

[DanW58]

I've kept revisiting and reuploading the patch to D3555.  Nothing too important.  I had leftover code for my attempt at using the skybox as environment map.  Spaces, indentations, dotting comments...

And I made one minor optimization that saved one call to sqrt().

Anyhow, if you want to have the latest, here it is, linked and attached.

https://code.wildfiregames.com/D3555

prohibited_shader.patch model_common.fs model_common.vs terrain_common.fs

[hyperion]

Just had a quick look at the patch uploaded to phab.

If you want ao to always range from 0-1 as mentioned in TODO 1 for any given map you might want to provide a script around imagemagick so you and other users can convert their assets to test this case as well. No need for the shader to take care of it.

 

ERROR: Failed to link program 'shaders/glsl/model_common.vs'+'shaders/glsl/model_common.fs':
error: fragment shader varying v_half not written by vertex shader

 

[DanW58]

1 hour ago, hyperion said:

If you want ao to always range from 0-1 as mentioned in TODO 1 for any given map you might want to provide a script around imagemagick so you and other users can convert their assets to test this case as well. No need for the shader to take care of it.

True, but there's an issue of quantization if you take an 8-bit deep texture, scale value up by 1.618, and then re-quantize it to 8-bits.  If you do that you will get quite noticeable banding artifacts in most assets.

The other two options are a) for the shader to do it, where there's scaling up but no further quantization...

or... b) to re-bake the AO.  If you bake AO's from Blender, there is a button you can check for Normalized output.  Re-baking would also give us a chance to correct errors with current bakings, besides the lack of normalization, namely that sometimes buildings contain assets like obelisks that in'game  deploy outwards, but themselves lack AO.  There's also a need to place props under the assets, just for the bakings, to avoid light hitting the model from below ground.  Last but not least, it would be good to switch to a baking tool that supports ray dot normal modulation;  they look much more natural and realistic.

Of the three choices, I'd favor re-baking, really;  but the shader doing it would be second-best.  To try to fix the textures would be my last choice by a long shot.

I'll fix the problem with the vertex shader and re-post.  Thanks.  I wonder why it doesn't report a problem on my machine...

Edited February 18, 2021 by DanW58

[DanW58]

This should fix the problem.

https://code.wildfiregames.com/D3555

 

model_common.fs model_common.vs terrain_common.fs prohibited_shader.patch

[hyperion]

ERROR: Failed to compile shader 'shaders/glsl/model_common.vs':
0:138(2): error: `v_half' undeclared
0:138(21): error: `sunVec' undeclared
0:138(21): error: operands to arithmetic operators must be numeric
0:138(11): error: no matching function for call to `normalize(error)'; candidates are:
0:138(11): error:    float normalize(float)
0:138(11): error:    vec2 normalize(vec2)
0:138(11): error:    vec3 normalize(vec3)
0:138(11): error:    vec4 normalize(vec4)

 

[Stan`]

Dan could you provide basic steps for correct AO baking in blender 2.9?

[DanW58]

Yes;  let me study it first;  I'm a 2.79 guy, but I have 2.9;  I'll take a look.

[DanW58]

Okay, @hyperion  fingers and toes crossed... knocking on wood...  this should fix the problem.

https://code.wildfiregames.com/D3555

 

prohibited_shader.patch model_common.fs model_common.vs terrain_common.fs

Edited February 19, 2021 by DanW58

[DanW58]

3 hours ago, Stan` said:

Dan could you provide basic steps for correct AO baking in blender 2.9?

2.9 is driving me nuts, as always.  I don't understand ANYTHING with this new interface.

I struggled to get set up with the model, the second UV layout, got a blank image loaded, setup evee for AO baking (I could not find AO interfaces for the other engines),  I got the output screen set up for PNG RGB 8 bit no compression.

Now, where is the goddam button to bake?

EDIT:  Found a tutorial on how to bake AO's in Blender.  As is the case with all Blender tutorials, it goes too fast to follow.  One thing is clear, though, it uses cycles.  I found the cycles interface;  tried to bake but it tells me there's "no active texture", whatever active texture is supposed to mean.  I think it may be the fact that I don't have a node network set up yet.  The guy in the video sets it up so fast it is a blur...

 

Edited February 19, 2021 by DanW58

[gameboy]

@Stan`

[Stan`]

To set an active texture you need to use the node editor. There add an image texture with your blank one and click on it to make it "active"

[hyperion]

18 hours ago, DanW58 said:

Okay, @hyperion  fingers and toes crossed... knocking on wood...  this should fix the problem.

Still get the same error.