DanW58

But if you want specularities in the stones, you might want to consider making them darker, though; I'm just saying. Because you mention you'll have normal maps. Normal maps look best with specularities. If these stones are anything like cobble-stones, they'd be much darker in diffuse. But if they are some kind of matte white stones, then they are fine.

Excellent!

Ok, fine. Thanks. I'll wait for your new maps then, as I see nothing in the current set that breaks with the terrain shader as is. Just so you know, though, my brightness correction is not a dimming of the whole thing, it is a curve that only begins to kick in when the ground material goes above 75% white; so if you have nothing so bright, my terrain shader should be doing nothing to it, EXCEPT getting rid of the specular, which I'm working on to trying to bring back, but is made so difficult by the data in specular for Acropolis Bay being complete nonsense. Another problem is that the terrain_common.fs shader gets neither an eye vector nor a sky-box from the engine. All I can do, in terms of specularity, is reflect the sun. That's it. I don't know why are shaders denied so much info.

I wasn't talking to you, Stan. I know my friends from my enemies. What I did say to you earlier stands, though; the tiles texture is too bright. A shader should not be expected to fix this kind of problem. EDIT: Consider this: If floor tiles deserve 255,255,255, what color do you use for Greek houses, which are traditionally painted with the whitest paint known to man? In fact, if Greek houses were full white in diffuse, I would NOT complain to you. But floor tiles *full white* ?!?!?! And it's inconsistent with the other textures. Try scrolling through the textures in the folder with a picture browser, the moment you hit the tiles texture your eyes begin to tear from the brightness. It instantly ruins any map that uses it.

@Panther A *.pyromod is a renamed *.zip; you can rename it back to *.zip if needed. I'm having a hard time with bringing back specularity to terrains. Initially I tried testing with my favorite terrain, the Belgian Bog, but nothing I was doing had any effect; then discovered that Belgian Bog has specularity turned off via xml settings. Maybe that's one of the reasons it's so nice So then I turned to my pet-peeve map, which I knew had specularity, the Acropolis Bay. So, first I wanted to know what exactly am I dealing with, so I made the shader output raw specular color to the screen, and this is what I found: I have no idea why some patches have almost full specularity. It seems totally random. There's nothing particularly different about those areas to tell in the diffuse texture. I mean, if the diffuse intensity went down as the specular intensity went up, those areas would turn metallic; but the diffuse intensity is rather uniformely too high; too close to white, in fact. And there is no material that could possibly be white in diffuse AND in specular; this is complete trash. But then, as I brought back specular code that I had erradicated from the terrain shader with full prejudice, I noticed something peculiar: The alpha channel in the specular texture is used to alpha-blend specular color with diffuse color AFTER lighting, which I'm still trying to wrap my head around... Why not just pre-multiply this alpha channel by the specular RGB, and use the alpha channel for a more valuable purpose? I decided to try and see what this alpha channel contains, so I converted it to grayscale and put it on the screen: As you can see, we find no relief from specular absurdity in the alpha channel. I was hoping that maybe those bright patches had low alpha to compensate, but the alpha channel is not only conceptually wasteful, it is wasted in actual fact by making it 255 all across. How is a poor shader supposed to deal with garbage at the input? One map can be beautifully well done; another can be totally absurd. But I'm not sure who exactly I'm pointing a finger to, though I certainly am. Is this the map-maker's fault?, or is it the materials?, or the engine? One thing I know is that I looked at the textures in the materials library a week ago or so, and I found the texture that makes up the tiles (that we barely see in the second picture above, so bright they are). That texture, as I said to Stan, is way too white, and on top of that has content in the specular channel. What my so hated terrain shader has done is inject beauty into this acropolis bay absurdity by toning down those whites, removing specularity altogether, and increasing contrast at the high end, thus making the tiles noticeable AND good looking *** for the first time *** ... ... but what do I get for it if not hate? How is it even possible? Or is this just the case of one little tyrant's opinion that everybody else is afraid to confront? @wowgetoffyourcellphone Can you please tell me which map is so badly affected by my terrain shader, so that I know what is it exactly that I need to fix?

Okay, so, the case for a dielectric, transparent, glossy layer over a diffuse base begins similarly to the case of dielectric over specular metal. We have a sunray coming in, 'a', that hits our dielectric and splits into a reflected ray, 'b' (the name of which got cut off in the pic), and a refracted ray 'c'. We already have the math for all that, except for one detail: My calculation of reflected light by the diffuse base was based on ndotl (normal dot lightray vector, or, the cosine of the angle), but light refracts and changes angle, so I should consider the refracted lightray, versus the original lightray. The good news is that the new Fresnel function gives me the cosine of the refracted angle already, as it needs to compute it internally anyways; so I don't need a special calculation; I just need to use that output of the function. So, our concern here is what happens from the moment 'c' is about to hit the diffuse layer. When 'c' hits the diffuse base, it explodes in all directions evenly. At least that is the simplified model of diffuse optics. Rays shoot out in a semi-sphere of even distribution. Some of the rays, shown in green, make it out of the dielectric layer; and some of them, shown in dark red, reflect off the dielectric interface, heading downwards again. This is simplified because the rays escaping are actually split, with a portion reflecting; however the portion is miniscule and can be ignored here. I put gray lines between the green and red, forming a triangle. This is the escape cone, whose half-angle (radius), which I marked in the drawing as 'x', and therefore solid angle, are a function of the material's refractive index alone. The higher the RI, the narrower the escape cone gets. THIS is what the function in the previous posts was all about. But note, as an aside, that the distribution of rays coming out of the dielectric and into the air is not limited to the cone of escape, because they refract, expanding again to a semi-spherical distribution. With this we can calculate how much light will refract out, and how much light will reflect back in to produce yet another diffuse spherical distribution, and so on and so forth. So, given a 'c' ray, let's follow the saga in pseudo-code: void saga_of_a_photon( vec3 ray_c, vec3 normal, vec3 MatDiffuseRGB, vec3 RefractiveIndex vec3 RefractingOutRGBlight, vec3 ReflectingBackInRGBlight ) { vec3 ExplodingRGBlight = dot(ray_c,normal) * MatDiffuseRGB; //diffuse explosion RefractingOutRGBlight = ExplodingRGBlight * RefractiveEscapeFunction( RefractiveIndex ); ReflectingBackInRGBlight = ExplodingRGBlight - RefractingOutRGBlight; } Before you jump horrified that RefractiveIndex is a vec3 instead of a float, there's a reason for it: most materials' refractive index changes with wavelength, so it is not the same for R, G or B. In the latest water shader I have the index of refraction for water as a vec3, not that I'm going to boast of noticing the difference in the results, but every little bit helps. And you might ask, how am I going to specify these "anisochromatic" refractive indices with only one texture channel, the definitive answer is I don't know yet. But seriously, chromium oxide, whose properties are what causes iridescence in chrome-plated Harley exhaust pipes, has steeply changing refractive indexes. To model iridescence I will need to model them correctly. There is one problem that arises from the picture and the pseudocode. I was going to grab the middle of the 'c' ray as my starting point to define the repeating cycle to then solve as standard geometric series. But now I realize I cannot do that, because the angle that 'c' is coming at is input light vector-dependent, whereas the reflecting-back-in rays are not. In fact, I would say that the average angle of light reflecting back in is (90 degrees - x) / 2. Don't forget we need to compute dot(ray,normal) at each diffuse bounce. So, my repeating cycle really begins AFTER the first diffuse explosion. Time for a break; I'll work on the other shaders for a bit.

Thanks! Back from a nap and you guys solved all my problems. Well, a table is not an option with a shader. Actually, it is; I have seen people commit complex things, such as the Fresnel formula, to a little texture, and read it for an answer. But I don't like that; it is slow in its own way, ocupies a texture unit, and has a precision problem; but if many cases of complex functions crop up it might make sense to put them all into a single 16-bit texture and have lookups for several functions. But definitely f(x) = 0.6217/( x^2 + 1.207*x - 1.585 ) is a lot simpler to compute than my powerz abomination. And it works like a charm and a half, too! So, in glsl: float DiffuseEscapeFractionFromDielectric( float RI ) { float temp = (2.207 * RI) + (RI * RI) - 1.585; return 0.6217 / temp; } So, I'm going to make a drawing for diffuse case; so I'll be back in 2 hours or so.

Well, no, it's not 2-dimensional; it is just a linear function: solid_angle = f( refractive_index ). Inverting the Fresnel function would take me centuries; that's why I did it this way. I have some software that's much like Matlab, but never got around to learn it. Many years ago I was using a beautiful app called DataFit; but it is pay-for software, and I got no money. This is the data: refr. hemispheres index solid angle 1.0 1.0000 1.5 0.2499 2.0 0.1314 2.5 0.0819 3.0 0.0562 3.5 0.0410 4.0 0.0312 4.5 0.0247 5.0 0.0200 y = 0.514 / x^7 + 0.486 / x^2 does it, but something cheaper to compute would be nice. For example a trig function multiplied by a constant ... something along such lines ... What's 1/RI? 1.0 ... 0.2 1-that = 1-(1/RI) = 0 ... 0.8 cos( (1-(1/RI)) * pi/2 ) = ... Nah, doesn't work.

Hahaha, well, that's the problem; I don't know what the function is. I made the huge table in page 1 of the spreadsheet to find out the angle of semi-reflection for each index, put the results into a table in page 2; now the trick is inventing a function that fits the data. y = 0.514 / x^7 + 0.486 / x^2 works nicely; but maybe something not involving powers would be nicer, performance wise. FresnelTable.ods

I still don't understand. My input is in the range from 1 to 5. IOW, I'm trying to compute % of diffuse light escaping as a function of refractive index. What you're talking about is a trigonometry approximation; no? I think you're going from angle to solid angle; I need a solution from refractive index. There's a cosine function in glsl; I don't need taylor expansion for that.

I'm COMPLETELY lost, Nani. Please explain the whole thing, not just one part.

Managed to improve the function, while using integer powers now. Example Refractive Angle for Solid angle f(x)= 0.593/x^6.5 Material index 50% refl. (hemispheres) + 0.407/x^1.75 Error ======== ========== ======= =========== =============== Air 1.0000 90.00 1.0000 1.0000 0.0000 Water 1.3350 48.10 0.3322 0.3361 0.0040 Glass 1.5000 41.40 0.2499 0.2427 -0.0072 Cr2O3 2.0000 29.70 0.1314 0.1276 -0.0038 Diamond 2.5000 23.35 0.0819 0.0834 0.0015 Merc.Sulfide 3.0000 19.30 0.0562 0.0600 0.0038 Silicon 3.5000 16.47 0.0410 0.0456 0.0046 Germanium 4.0000 14.36 0.0312 0.0360 0.0048 Unobtanium1 4.5000 12.76 0.0247 0.0293 0.0046 Unobtanium2 5.0000 11.47 0.0200 0.0244 0.0044 Thanks; I had to submit this table I was working on, to look at your reply. I'll check that out. EDIT: I don't get what you use as x.

Okay, it's been long enough that might as well put this in a new post. I'm attaching the spreadsheet here so that people interested can look at it. Note that we are talking about light bouncing diffusely (spherically) and then trying to get out of the dielectric medium. So we are talking about going from a higher refraction index medium to a lower index of refraction medium. In this case, there is an angle at which reflection is mirror-like. If you've ever been underwater in a swimming pool with your eyes open, or with goggles on, you've probably noticed that looking up you see a circle through which you can see stuff above the water, but further away the surface of the water looks like mercury; totally opaque, and reflective like a mirror. Inside water, the angle of total reflection is 49 degrees from the vertical. The angle of 50% reflection is 48.1 degrees. At 45 degrees, see-through is 95%. So, it's not a very gradual transition; it is smooth but almost step-wise in its rapidity. So when we obtain our 50% reflectivity angle and use that to calculate solid angle and % of light that escapes, we don't need to worry about transition function feactoring in; our result will be pretty much exact. So here is the relevant results: First column is indices of refraction, second column is the 50% reflectivity angle, and the third column is an approximation function I found to match the results, which function involves two odd powers. If someone is good at matching data with functions and wants to contribute a better one, it will be much appreciated. The goal is an f(x) that approximates the solid angle (hemispheres) column with the refractive index as input; hopefully a polynomial, or something cheaper to compute than two powers plus two multiplications and one addition... Example Refractive Angle for Solid angle f(x)= 0.593/x^6.5 Material index 50% refl. (hemispheres) + 0.407/x^1.75 Error ======== ========== ======= =========== =============== Air 1.0000 90.00 1.0000 1.0000 0.0000 Water 1.3350 48.10 0.3322 0.3361 0.0040 Glass 1.5000 41.40 0.2499 0.2427 -0.0072 Cr2O3 2.0000 29.70 0.1314 0.1276 -0.0038 Diamond 2.5000 23.35 0.0819 0.0834 0.0015 Merc.Sulfide 3.0000 19.30 0.0562 0.0600 0.0038 Silicon 3.5000 16.47 0.0410 0.0456 0.0046 Germanium 4.0000 14.36 0.0312 0.0360 0.0048 Unobtanium1 4.5000 12.76 0.0247 0.0293 0.0046 Unobtanium2 5.0000 11.47 0.0200 0.0244 0.0044 FresnelTable.ods

Update: The third thing I said I did I didn't. I may have tried it and not liked it. It's just not there. I'll try bringing back some specularity as dielectric shimmer ...

I had to refresh my memory. What I did to terrains was three things: I got rid of the specular channel; I ignore it with full prejudice, since ground never has much specularity to speak of, unless it is man-made polished stone, or smooth ice; but even in those cases the specularity is dielectric rather than metallic. Even so, some grounds, particularly the Acropolis elevated ground, were so bright in diffuse they still saturated even after removing the specular channel. So what I do is detect unsaturated (gray-scale) material that is too bright, and dim it down by using squaring, color*color, which dims while increasing contrast. This did wonders for the Acropolis and the Badlands maps, both of which had very bright tiles around where you start the game. Belgian Bog did not really need fixing but it got a little bit of darkening and contrast enhancing of the stone paths, and they looked beautiful; I appreciated them for the first time. The third thing I did, and which I'm thinking perhaps it's not entirely correct, is I multiplied ambient lighting for terrains by 0.5, to compensate for the fact that we don't have terrain AO bakes. I'm thinking this is perhaps unwarranted because if we did have such AO bakes they would look about 80~90% white on average. So, yeah, I think I'll get rid of that 0.5 and report back. Thanks for the reminder. EDIT: And regarding specularity of grounds, now that I have Fresnel, I could bring the spec channel back with some arbitrary Fresnel and some arbitrary but very low spec power, to make it just a blurry shimmer. I'll experiment with that.

I just checked the video stats in Youtube; 13 views so far; average view time 2 minutes and 3 seconds. That's less than half the video. So most people click away before getting to half of the video. I frankly don't understand this community. This is a waste of my time like no other.

I made a little demo video, for the undecided ... The fish need alpha.

The Fresnel function worked the first time; no debugging needed. It's now incorporated in the metal_shader_set version1.3 pyromod's modified water_high.fs. It works a charm and a half. So, let's get back to the problem of multiple diffuse light bounces within a dielectric glossy coat: I was making a table that was failing due to using Schlick's Abomination, er, I mean Approximation, to try and visually educe a formula to relate how much diffuse light gets out of a dielectric at each bounce. To re-do the table using real Fresnel would involve more columns than fit in my poor man's screen; so I'm going to use LibreOffice Calc, and maybe I will figure out how to copy the relevant stuff to insert here ... Darn! To have "50% reflection ANGLE" as a function of refractive index involves a huge math operation on the Fresnel formula. I'm not sure this is even possible. I think my table is going to be a multiplication of refractive indexes and angles as a multi-page column, and resulting reflective coefficients in the last column, and then I can visually select input angle for 50% result.

I get this problem sometimes. If I've played a map before with allies, next time I select that same map it seems to remember I won the previous time.

@gameboy Maybe I will do better; but it's no reason to wait. Enjoy the present moment. I got disconnected from the Internet last night; something to do with an unpaid bill ... Unbelievable how obsessed with money most vendors are ... There may yet be a version 1.4; I'm not sure. I want to get rid of white skins. No, I'm not into white-bashing; I'm white myself; I mean completely white (or very nearly completely white) color in diffuse to try and represent human skin. Some civs have good looking skins, like Ptolemies for example; but many are so white they look like porcelain; like the blood has been sucked out of them. The problem is that my skin-detection hack is too encompassing, and lots of things detect as skin that shouldn't, including wheat fields, roofs and walls. So the problem is not fixing zombie skin; the problem is seeing zombies everywhere ...

Version 1.3 is here. Fixes issues with Fresnel refraction factors with light going into the water, and with light coming out: e.g., when the Sun is low, very little sunlight should be able to enter the water, most of it reflecting back up instead; therefore bottoms of water bodies should darken faster than by mere ray-dot-normal. The light reflecting off the bottom and coming back out again is modulated by fresnel refraction, except in this case it is calculated by eye-view vector angle, rather than the Sun's angle. Particularly noticeable in the second picture below: The shadows of the floating leaves are not too dark. Why? Because sunlight is attenuated by Fresnel, and so most of the light the bottom gets, with the Sun so low near the horizon, is bluish light from the sky, rather. The wet coast hack looks a bit better now. Modulating water's specularity linearly over a couple of yards is a noticeable artifact, really. I now modulate water's index of refraction, instead, as I fade in bottom darkness. The way it works is it darkens the bottom early (as if we are walking towards the water), but water's reflectivity (and foam) fade in more slowly. If you have any problems with the mod, try maxing out settings like "number of shaders" and most graphics quality options, EXCEPT sharpening and MSAA. I keep MSAA at (4x); good enough for me. And be sure to report here. And if you have nice screenshots to share, don't hesitate. metal_shader_set_v1.3.pyromod

Coming soon; next half hour; but the difference is very subtle. I notice because I know what to look for. Bottoms of water bodies were getting too much sunlight for sun positions close to the horizon. Water's fresnel reflectivity is high at shallow angles, and so sunlight should not be able to penetrate as much as it did. Now the amount of light the bottoms get is exactly right. I'll post soon; like 15 minutes.

Yep, solved already; just putting together the pyromod and screenshots. Did you get the previous version to work?

ROTFLMAO! That is EXACTLY what I just finished doing. I found that the original shader had no light computations onto the ocean bottom, or floor under the water. It is as if it was emissive. Needless to say the light was not modulated by the fresnel refraction factor. So I put that in, but then I forgot that on coming out of the water, there's another fresnel refraction factor to account for. I just added it. However, the two are not the same: One is calculated on the light vector angle to the water; the other is calculated on the view angle to the water. Right now I have the same fresnel refraction factor used twice. Have to fix that. And I also have an idea that might improve my wet coasts algorithm; a crazy idea, have to try it. So expect a version 1.3 by tomorrow morning.

DONE! The water shader now uses TRUE fresnel for reflection and refraction coefficients. This is the Fresnel formula from Wikipedia; no approximations; no hacks. And it works! Updated also the Phabricator water_patch: https://code.wildfiregames.com/D3603 metal_shader_set_v1.2.pyromod