[Flickering shadows！]

Thanks for the encouragement.

This version now works better.  Pretty good with Ptolemaic, but the problem with the Athenians was that the texturing there makes so little sense it's nearly impossible to work with.  Like the fountain in the civic center has a ring of carpeting around, looks like persian carpeting, but obviously it has a high specular value and I can't avoid detecting it as a metal.  Maybe polished stone was the intent.  The Greek columns, I'm not sure what kind of material they are, but they seem to have high specularity AS WELL AS being white in diffuse, so they detect as metallic and get unduly darkened.  On the plus side, the Athenian spearmen's bronze helmets and shields shine!  But I got better news;  even as I was writing this post, it occurred to me I could force sense into the texturing right in the shader, before trying to detect metallic intent, simply by clamping the specular color to (0,0,0) to ((1,1,1)-diffuse),  that way, texels for which the textures add to more than white are corrected down.  This "sense injection" is what the first 3 lines of the code below do.  The second part is "metallic intent" detection, and the third part is what it does to the diffuse and specular colors in order to implement the metallic look.

So, this is not advocated as a permanent mod;  it is more of a teaching tool or demo.  Which doesn't mean I don't suggest it be incorporated in A24;  I actually do;  it will make the game look much better.  Even if it's a hack, it's a hack that helps break the plastic look monotony by depicting a few items as clearly metallic; and this is a Good Thing (TM).  What it does mean is that I suggest it be in A24, but then removed by A25, by which time most of the texturing should be revisited and made sense of;  and metallic/non-metallic distinctions should be made clearly and deliberately in the art itself, not in a shader.

However, I have trepidations about throwing this into D3555.  Partly, I'd rather wait for feedback first;  partly I think I can improve this hack further...

Relevant glsl code in model_common.fs:

    vec3 ZERO = vec3(0.0);
    vec3 ONE  = vec3(1.0);
    specCol = clamp(specCol,ZERO,ONE-texdiffuse);
    
    float is_metal;
    is_metal = dot(texdiffuse,texdiffuse);
    is_metal *= clamp(1.0-100.0*(texdiffuse.g-texdiffuse.r),0.0,1.0);
    is_metal *= clamp(1.0-100.0*(texdiffuse.b-texdiffuse.g),0.0,1.0);
    is_metal *= max(max(specCol.r,specCol.g),specCol.b);
    is_metal -= (texdiffuse.b*0.75);
    is_metal = smoothstep(0.05,0.2,is_metal);

    vec3 metal_specCol = sqrt(texdiffuse);
    vec3 metal_diffCol = texdiffuse*texdiffuse;
    specCol = mix(specCol,metal_specCol,is_metal);
    texdiffuse = mix(texdiffuse,metal_diffCol,is_metal);

Also on the positive side, note how some of the vases are clearly painted terracotta (the big ones), but the smaller ones look metallic (bronze).  Unfortunately, the chickens detect as metallic, but the end result doesn't look metallic, anyways.  Who would be going around making chickens specular?  Beats me...  The skins of most human models also detect as metallic, but after metallic processing they just look tanned, which is a positive development, as people in the first Century presumably spent more time in the sun than we do, and survived in spite of vitamin D not being available at their pharmacies.

Anyways, this is a hack, as I said;  it's main purpose is to prove to artists that there is value in getting colors and materials right by physics and optics principles;  that everything is not a matter of taste when it comes to depicting things realistically and convincingly.  Metals and non-metals are worlds apart in how they behave optically, and you need to know what you are doing to make things look right, not just follow gut feelings.  What coding and testing this hack has revealed to me is that people have been producing specular textures left and right without an iota of understanding of material specularity.  Like making white things specular as well as white, or like making the most matte things like carpets and chickens have non-zero specularity, or like expressing dielectric specularities in a metallic way, via the specular texture.

I'm including all three shaders, as they need to be tested together, though only model_common.fs has changed since my last shaders upload.

 

model_common.fs model_common.vs terrain_common.fs

[Flickering shadows！]

It's not working yet, --the metal hack.

It worked for Ptolemaic;  but as soon as I tried it with the Greeks, it created all sorts of artifacts.

An update is imminent, but I'm too tired right now.  Tomorrow.

[Flickering shadows！]

I just added yet another shader hack.  And a hack it is.  Despicable.  Reprehensible.  But it kind of works.

I got tired of seeing things like yellow bracelets and headbands that I KNOW were meant to look like gold or bronze but look like yellow paint instead.  So I wrote a disgusting algorithm to decide whether metal was intended, and where it decides that metal was intended, it shamelessly changes the diffuse and specular colors to make the thing look metallic.  So, this is totally in lieu of fixing the art.

This is what the relevant fragment shader glsl code looks like:

    float is_metal =
specCol.r*specCol.r*specCol.g*specCol.g*texdiffuse.r/max(texdiffuse.r,texdiffuse.g)*texdiffuse.g/max(texdiffuse.g,texdiffuse.b);
    is_metal *= 4.0;
    specCol = mix(specCol,sqrt(texdiffuse),is_metal);
    texdiffuse = mix(texdiffuse,texdiffuse*texdiffuse,is_metal);

Check out the bronze wrist bands and copper head band on Mr Faro, and the gold headband on his Wife.

model_common.fs

[Flickering shadows！]

7 minutes ago, gameboy said:

Can you provide screenshots? Can I see the effect?

I did;  they are in my previous post, which is at the bottom of the previous page.

But I'll show them again here:

 

[Flickering shadows！]

On 08/02/2021 at 5:49 PM, wowgetoffyourcellphone said:

Dear god that's too dark. 

Hopefully it no longer looks too dark, even though it's the same brightness, after adjusting down the terrain.  See my second-last post for screenshots.

[Flickering shadows！]

On 08/02/2021 at 4:43 PM, hyperion said:

How the ancients used white in architecture it's unreasonable that the ground is "whiter" than the CC. While the ao might have improved I guess it also highlights the need to fix base colour.

To really give an answer whether I share @Stan` fear you claim please provide all your changes as a single patch against trunk or a mod to play with.

https://code.wildfiregames.com/D3555

It's diffed from the folder .../binaries/data/mods/public

[Flickering shadows！]

UPDATE:

I was accidentally applying this to the color after lighting and shadows, instead of to the color from the texture.  That is fixed now.

I also improved the algorithm to give it much less influence whenever or wherever the saturation is not close to zero.

As such, the most affected ground is the Acropolis map.  Alpine Valley and Belgian Bog are unchanged except for the gray stone paths.  Bactria is not affected at all.

 

terrain_common.fs

[Flickering shadows！]

Shader mod to compensate for bad art.

As the color of the ground progresses towards too bright AND unsaturated, this shader dims as well as increases contrast.  It also removes any specularity, and ignores specular texture if such an atrocity is specified.

This should move us towards better light level agreement between grounds and buildings, which is probably what causes the perception of correct ao level as being "too dark".  So it is intended to go together with the other mods.

EDIT:

I suggest that the method of baking ao's be revisited with the understanding that an ao texture MUST span from full black to full white.  If there is no will to redo ao's for all existing assets, a software solution could be to check for min and max values on load, and inform the shader of what factors to multiply ao by.  Reading the whole texture to find min and max may be costly, however.  A random sampling might be a good compromise.

 

 

terrain_common.fs

[Flickering shadows！]

The only other thing I changed in the shader is I made the default ao vec3(0.5), otherwise things with no ao would look too bright by comparison.

  #if /*(USE_INSTANCING || USE_GPU_SKINNING) &&*/ USE_AO
    vec3 ao = texture2D(aoTex, v_tex2).rrr;
    ao = mix(vec3(0.5), ao, effectSettings.w);
    ambColor *= ao;
  #endif

ALL of the xml files in binaries/data/mods/public/materials that possessed a,

<uniform name="effectSettings" value="1.0 50.0 0.0075 0.85"/> (or whatever number)

I changed to read

<uniform name="effectSettings" value="1.0 50.0 0.0075 1.0"/>

 

Here's a couple other pics.   NOTE, I'm not entirely happy with the ao as is;  it should be a bit more noticeable, and I think it is due to the kind of ao it is.  The type in which the rays are modulated by cos of incident angle is a lot more realistic looking;  but this is a lot better than nothing at all, or than the way it was (either invisible or not there at all).  The question for you is, is this too dark?  Because that was Stan's worry.

I think it IS darker, but that it is as it should be.  It looks more realistic, less arcadish, in my eyes.  I see no glaring problems;  I think it can easily go into A24.

Of course, it needs to also have ambient shadowing onto the ground, --VERY subtle, but it needs to be there--, and I have a solution to propose for that, for A25.

Here's a few other pics, and the shaders, and I'm going to bed, finally.

 

model_common.fs model_common.vs terrain_common.fs

[Flickering shadows！]

1 minute ago, vladislavbelov said:

You can always check your frame by capturing it with RenderDoc or something similiar.

My problem was stupid;  I changed those xml files, but I did not reboot the game;  I merely started a new one, which is ok for testing a shader.  Once I rebooted the game, the ao kicked in.

So, that's it, the picture above is with 1.0 in the shader, 1.0 in the xml's.  Whaddya think?

[Flickering shadows！]

BINGO!

Is that too dark?

Or does it look "too real"?

[Flickering shadows！]

But this is not the game;  is it?  There's no trees, no sun, no chickens;  this is a testing application, or what?

[Flickering shadows！]

Okay, yeah, it DOES work for you, though it is terribly, terribly light... but it is definitely there.

Hmmmm....

 

[Flickering shadows！]

8 minutes ago, Stan` said:

Maybe you broke it with your changes in shaders (just spitballing here)

Else could be something more pernicious, e.g the game Hwdetect.js detecting your machine is not fast enough to run them, and disabling it, but that would be strange, since it would disable GLSL in the option as well.

Nope;  can't be;  my changes to the shader were additive;  I did not change much at all, just added 3 lines of code to modulate ambient light.

Hwdetect can't be the culprit either, as I can confirm I have glsl running, as every time I make a mistake in the shader I get game assets disappearing or big error messages from the glsl jit compiler.

But besides, Gameboy is not getting ao either, and he seems to have a better machine than mine.

I think this is something serious.  It may be the case that nobody is getting ao;  and you never had a bug report because people don't know what ao looks like, or that it even exists.  I think this is a big bug.

Can you show me a screen shot from the game, no white building, no special test mode;  from the game itself, a screenshot with ao?  I'm starting to believe it just doesn't work at all.

[Flickering shadows！]

Alright, I changed both files, making the last term in settings 1.0.  In player_trans_parallax_spec.xml I even added the ao file as required for good measure;  and I have the gpu skinning line in config.  Still no sign of an ao anywhere.

Could it be I need an ao line in the config file?  I see nothing of the sort there...

[Flickering shadows！]

4 hours ago, Stan` said:

https://trac.wildfiregames.com/browser/ps/trunk/binaries/data/mods/public/art/materials

I can confirm, adding the gpuskinning line to userconfig made no difference.

Found the folder!  I don't know what I was looking at..

[Flickering shadows！]

4 hours ago, Stan` said:

The one above is player_trans_parallax_spec

<uniform name="effectSettings" value="1.0 50.0 0.0075 0.85"/>

should be

<uniform name="effectSettings" value="1.0 50.0 0.0075 0"/>

Since there is no AO. But that doesn't seem to work. Buildings still have self shadows.

This one is used by the left cc player_trans_ao_parallax_spec

<?xml version="1.0" encoding="utf-8"?>
<material>
  <alternative material="player_trans_ao_spec.xml" quality="8"/>
  <define name="USE_PLAYERCOLOR" value="1"/>
  <define name="USE_PARALLAX" value="1"/>
  <required_texture name="baseTex"/>
  <required_texture name="normTex" define="USE_NORMAL_MAP"/>
  <required_texture name="specTex" define="USE_SPECULAR_MAP"/>
  <required_texture name="aoTex" define="USE_AO"/>
  <shader effect="model"/>
  <uniform name="effectSettings" value="1.0 50.0 0.0075 0.6"/>
</material>

EDIT For the record

<uniform name="effectSettings" value="Normal Specular Parallax AO"/>

 

Stan, nowhere you told me what files are these, where they are;   or well, you mentioned a folder that doesn't exist in my svn;  though apparently it exists in the website.  I don't know what these xml files do, or where they are.  Is it really so difficult to get ao working?

I will try that skinning thing, but you tell me it won't make a difference;  then why mention it?

[Flickering shadows！]

@Stan`Could you give me the file(s) I need to be able to see ambient occlusion in-game?  I'm aching to work with it, even if it doesn't make it to A24.

[Flickering shadows！]

Interesting.  Well, frankly it still irks me that Cycles introduced all this horrendous amounts of noise that no other renderer ever did, and instead of being fed to the sharks they get crowned kings of renderland, and everybody goes dilligently to work on super-intelligent denoisers.  Why don't they remove all the useless random stuff from their code and call it a day, instead?

Also, ao's are tricky things to filter.  Some places they can be smooth;  some places need sharpness...

What about running Xnormal on Wine?

[Flickering shadows！]

2 minutes ago, Stan` said:

Since 2.5 I think you could bake with both internal and cycles.

If that's the case, I never noticed.

But like I said, the Cycles people are crazy about noisifying everything.

I was in that camp, at one time;  I thought random noise was God's lubricant.  I eventually found out what a terrible mistake that is.  Even using ray jitter randomizations in an Blender AO introduces noise;  and noise is NOT so easy to filter out.  Filters can drop the high end of the noise spectrum, but leave the "pink noise" (lower frequencies) still remaining, and it becomes a visual repellent. 

[Flickering shadows！]

I haven't baked anything since 2.74, and back then there was only one renderer for ao.

We could also try and write our own ao baker.

That text tools looks marvelous;  but it doesn't bake, does it?  But it looks it makes much better uv layouts than most tools out there.  I never found a tool good enough;  always did the unwrap by hand.

At least in the old days there was a Normalized check box in the Blender Bake panel.

If you click it, the brightest texel is scaled up to white, and the rest are all scaled by the same factor.

I bet this is the reason the ao's are so dark here.  That thing has to be clicked on, and then this situation where nothing is brighter than 0.63 cannot happen.

And you might ask, "Isn't that tweaking in the tool?".  Well, yes and no.  A fully unoccluded point on the surface of the model SHOULD be white.  If it isn't it's just got something to do with the number of rays that were accumulated;  some non-power-of two number.  So it is a science based tweak to Normalize an ao texture.

[Flickering shadows！]

Stan, did you set the flag for Normalized when you baked ao's in Blender?

That is a flag that should not even be optional;  it MUST be on.

[Flickering shadows！]

Yes, so maybe I should edit a letter to send to Blender, as a feature request in the name of wildfiregames.

I'll have it ready in an hour or so.

[Flickering shadows！]

I'm afraid Cycles would not work.  Several problems:

1) Cycles's devs are devotees of random;  they solve every problem by adding noise, and then give you decapitating filters to get rid of it.

2) A node network would never be as fast as hand-written code.

3) Neither the Cycles people nor the Blender people have the slightest clue about floating point numerical precision issues, and when you are adding 300 or 1000 rays the accumulator had better be double precision, or you get huge aliasing problems.

My buddy Klauss and I, at Vegastrike, once built a relationship with this Spanish guy that had an ao tool called Xnormal.  We helped him improve quality, such as by using a double for the accumulator, and helped him get rid  of a lot of bugs.  He had the option of vlight dot normal modulation or plain ao.  Not sure if it still exists, though;  that was 20 years ago.

EDIT:  (still there, but still Windows-only)

https://xnormal.net/?lang=en

[Flickering shadows！]

8 minutes ago, Stan` said:

Will take months to import and rebake everything. Maybe we could just apply a filter on the texture or something using Image Magic.

We can even write our own image manipulation programs;  it's not that difficult.  I'd even prefer to do that.  But baking ao that is angle of incidence modulated is not really convertible from standard Blender ao;  it needs to be redone.  We should get in touch with Blender devs and make a formal request.  I could write it out for you.

So I removed the instancing and skinning conditionals from the ao block in the shader, as it makes absolutely no sense to me that should have to either skin a cat or use instancing if I want to have ao;  that's absurd.  Here's the update:

model_common.fs