Ambient Occlusion Baking Tutorial

[DanW58]

Stan asked me to write a tutorial on AO baking, due obviously to my criticisms about the way it is done at present.

However, the challenge for me was getting to do this in Blender 2.9, being stuck, as I've been, at 2.7x, unable to handle the frustrations with the new interface, and having to deal with Cycles and all that.

But with the help of people and video tutorials I managed, sort of.  There's probably things not needed, or better ways of doing stuff.

Anyways, let's begin:

I used the ptol_civic_centre.dae model for the test, since it is a model I became familiar with for different reasons.

So first step is I import it into Blender,  WITH units (yards... why not furlongs?).

Sorry;  this interface doesn't allow me to intersperse text and images;  does it?  No way to edit caption,  nothing.

Alright, I'll just attach all the images and figure out later how we can turn this into a tutorial.

Edited February 20, 2021 by DanW58

[DanW58]

Ok, in lieu of a properly formatted tutorial, here's a rough outline of what the image sequence is about:

After loading the civic center, I went to UV mode to slap a blank texture on it.  I don't know if this is necessary;  probably not;  it was the way it used to be in Blender.

The images are really a chronicle of my journey to getting to bake an AO like 80%;  the other 20% is tutorial.

One thing that IS tutorial material is when I place a big cube so that the top-side is at ground-level relative to the cc.

Then I raise the bottom of this cube to a reasonable depth;  we don't need to reach the center of Earth to block light...

Then I used Ctrl-R 15 to divide X and Y of this cube into 16 strips.  This is important to reduce possible errors by Blender if the occluder is so big the edges of the occluding polygon are outside the range of what Blender is looking at.  Very large surfaces are often sources of artifacts.

The box is only there for the baking, then it is thrown away;  deleted permanently for all I care.  It prevents light coming from underground affecting the AO.  Usually, buildings don't float in space;  except in SciFi;  and so they get most of the ambient light coming from above, down to the horizon, but not from below the horizon (except for light reflecting off the ground, but we can't compute that light for AO baking because the same building can be on different grounds, of different colors and albedos.

Then come a bunch of screens about setting up Cycles for AO baking, setting up a material, creating a texture, and all that.

Finally, a first AO test, very pixelated due to low setting for samples, since it's just a first test.

Then I increased samples to 256 and baked again.

What I wasn't liking at all was the way the UV layout was made, with almost no space at all between islands.  That is bad for filtering.  When you zoom-out, artifacts may start appearing along polygon edges.  Why?  Because as you zoom out, the GPU needs to use smaller LOD's of the texture, where each pixel may be 4 pixels of the full size texture, or 16, or 64 ...  So if two islands are too close to each other, and one is light, one is dark,  as you zoom out, some LOD pixels may be averaging many pixels across two unrelated islands, and end up with a grey edge on each polygon, instead staying light and dark respectively.  This is a well known problem, and it is the reason automatic tools for UV layout let you specify how much space you want to leave between islands, and why the AO baking panel has a "Margin" setting, for how many pixels of color extension you want around your islands.

So I selected all the islands, set the scaling and rotation mode to "Individual Centers" and scaled all the islands by 0.75.  Then baked my AO again.

After this I stopped seeing some artifacts, but other artifacts emerged:  color bleeding artifacts on a number of wall strips.  Thinking that it was due to polygon overlaps in the UV layout, I proceeded to first move all those polygons out of the texture space, then moving some of the big islands near the bottom left corner to make room for them, and I laid them out in their new home.

However, upon baking a new AO, most of the artifacts were gone, but not all.  I moved one of the islands for a strip again, and the problem remained.

I think the cause of the artifacts is that those strips are overlapping large polygons.  Fixing that would be a complete redesign.

I used to design pretty complex spaceships that were for the most part a single, manifold mesh.  (Example pics attached.  Note that the only interpenetrating objects are the two little gunmounts on the wing;  the rest is a single, continuous mesh.)  I never used interpenetrating objects, much less overlapping polygons.  To design a building like this in a single manifold, you basically start with a cube, and you subdivide it with CTRL-R (loop cut), Extrude, scale... Just those 3 operations you can follow to turn a cube into an object of just about any complexity.  It should not be necessary to have gazillions of interpenetrating cubes and overlapping polygons.

It is not only unnecessary and artifact-dangerous;  it is also wasteful of the most precious GPU resource, pixel render operations.  The inside of a building like this should be completely empty;  it should look just like the outside, but in negative.  There should not be gazillions of polygons crisscrossing within the invisible interior.  They may be hidden from your eye on the screen, but the GPU tries to paint all those polygons, and ends up aborting each pixel draw after confirming that something is already drawn in the pixel with a lesser Z (closer to the camera).  This is the worst thing you can do for graphics performance.

But this is a tutorial for another day.

Another thing that is missing from this tutorial is the obelisks, the lions, and Mr Faro and his wife.  I don't know where they are.  The AO can't be complete until they grace our building with their beautiful shadows.

Cheers!

Edited February 20, 2021 by DanW58

[Stan`]

Non manifold objects were made to reduce the polycount.  That's the reason why we do it. Eg the bottom face of the cube is useless since you never see it.

Else you get a lot of "useless" extra faces which artists were told are bad for performance.

It's possible faces are black because they are inside out.

You need to import all the "props" referenced in  the cc actor. If you want the obelisks and other statues. You can either do it manually or use my blender addon. Some buildings have dozens of props.

The reason they are separate is because each different diffuse need a separate mesh. It can also be because they need to adapt to the terrain in which case they need a separate root from the main mesh.

Another reason is to allow for consistency between buildings by reusing the same assets.

It's all a bit strange at first but one gets used to it.

Thank for making a tutorial. So basically it's AO without any extra options and a ground plane

[nani]

Nice, Will check that blender add-on. I always wanted to see how a generated map looks in cycles

[Stan`]

Doesn't load maps yet I didn't know there was an interest for it

[DanW58]

@Stan`

1 hour ago, Stan` said:

Thank for making a tutorial. So basically it's AO without any extra options and a ground plane

Right.  Well, I'm not sure what you mean by extra options, so I can't opine yet.

It's AO with a ground plane, for sure.

Increasing space between islands... VERY IMPORTANT.  The islands in this model were JAM-PACKED like sardines in a can.  NOT good!!!

NOTE:  The problem I discovered with existing AO's of them being too dark;  no part of the texture looking brighter than 0.63, didn't manifest for me.  There was no "Normalized" button like there used to be in older Blender versions;  but what came out was normalized anyways.  I'm not sure why the current AO's are so dark.  One thing I did was I set the material to 1,1,1 white (default is darker).  Not sure AO should even look at material, but I just throw that in.

Yeah, well, poly-count is overrated.  Even if indeed the Pyrogenesis engine is CPU bound, the days when CPU's were so slow that poly-count was all that mattered are LONG gone;  I'd say today you could add one level of subdivision to everything, and it would not make a difference at all.  Your devs should try this as an experiment.  Whereas overdraw is a GPU KILLER. I'd cut off all the junk inside the building and weld together everything that's left, and I bet you ANY MONEY (including Bitcoin) it will improve display time.

 

Edited February 20, 2021 by DanW58

[wowgetoffyourcellphone]

2 hours ago, DanW58 said:

Ok, in lieu of a properly formatted tutorial, here's a rough outline of what the image sequence is about:

You can certainly intersperse text and images on the forum. I do it all the time.

[DanW58]

I've no idea how...

[DanW58]

2 minutes ago, wowgetoffyourcellphone said:

You can certainly intersperse text and images on the forum. I do it all the time.

I've no money to offer for the knowledge, I'm afraid.    I therefor beg for it;  how do you do it?

I go to edit and I see a bunch of square icons representing the images.  Nowhere to place a cursor between any icons.

[wowgetoffyourcellphone]

41 minutes ago, DanW58 said:

I've no money to offer for the knowledge, I'm afraid.    I therefor beg for it;  how do you do it?

I go to edit and I see a bunch of square icons representing the images.  Nowhere to place a cursor between any icons.

For uploaded images you can click the plus symbol on the image thumbnail to place it into the visual text editor. From there you can move it around to wherever you need and type text before and after it.

[Stan`]

5 hours ago, DanW58 said:

 

It's AO with a ground plane, for sure.

I think I said it before but I'm really worried about how it will look with uneven terrain. Can you check?

5 hours ago, DanW58 said:

Increasing space between islands... VERY IMPORTANT.  The islands in this model were JAM-PACKED like sardines in a can.  NOT good!!!

Indeed the other tutorial recomends a margin of .1 for smart uv. I also dos "seams from islands then do a normal unwrap to remove some issues. Then I do a last pass where I try to remove as much blank space as possible by splitting objects.

5 hours ago, DanW58 said:

NOTE:  The problem I discovered with existing AO's of them being too dark;  no part of the texture looking brighter than 0.63, didn't manifest for me.  There was no "Normalized" button like there used to be in older Blender versions;  but what came out was normalized anyways.  I'm not sure why the current AO's are so dark.  One thing I did was I set the material to 1,1,1 white (default is darker).  Not sure AO should even look at material, but I just throw that in.

Oh I think I remember that option now. We didn't use it cause it make buildings look out of place with the current material values.

5 hours ago, DanW58 said:

Yeah, well, poly-count is overrated.  Even if indeed the Pyrogenesis engine is CPU bound, the days when CPU's were so slow that poly-count was all that mattered are LONG gone;  I'd say today you could add one level of subdivision to everything, and it would not make a difference at all.  Your devs should try this as an experiment.  Whereas overdraw is a GPU KILLER. I'd cut off all the junk inside the building and weld together everything that's left, and I bet you ANY MONEY (including Bitcoin) it will improve display time.

Well if you have data to backup this claim sure I can't take your word for it. Remember thought that we support crappy intel drivers on windows and relatively old hardware.

Strong subdivision might increase overdraw.

So while what you say might be true for a GTX 780 (i took relatively old good hardware as an example) most people play the game on intel HD3000 -4000 (see https://feedback.wildfiregames.com/result for intel.vs nvidia amd repartition.

 

[DanW58]

4 hours ago, Stan` said:

I think I said it before but I'm really worried about how it will look with uneven terrain. Can you check?

darn!  I just understood what you're talking about.  The wall extensions.  They will be exposed, but seem shadowed.

Here's a simple solution:  Instead of at ground level, place the ground box down to the end of the wall extensions;  and either make it larger or perhaps make it circular and add a wall or rim at the edge that goes up to ground level.  The wall's downwards extensions will get progressively darker as they go down, but by very little.   I'll make a prop that can be imported and used for this.

Here we go:

Mesh attached at the bottom.  UGLOFAO stands for Universal Ground-Like Occluder For Ambient Occlusion.

(I should work for JPL...)

 

4 hours ago, Stan` said:

Oh I think I remember that option now. We didn't use it cause it make buildings look out of place with the current material values.

This is a perferct example why I'm such a non-believer in "tweaking" things.  If it should be right, but doesn't look right, there's a problem to investigate. But if you guys didn't know what Normalized was for, or why an AO should always be normalized, that explains things.  But generally, tweaks, like the gains set to arbitrary numbers in the xml files, will sooner or later come back to bite you.  Not only that;  tweaks also reproduce like mice.  You tweak one thing today, and tomorrow you have to tweak a dozen other things so they don't disagree with the tweaking of the first...  Let's go to physics and optics from now on.  Taste arguments can't resolve;  but physics and optics are science.

4 hours ago, Stan` said:

Strong subdivision might increase overdraw

What's "strong subdivision"?  Never heard the term.

UGLOFAO.dae

Edited February 20, 2021 by DanW58

[Stan`]

15 minutes ago, DanW58 said:

Here's a simple solution:  Instead of at ground level, place the ground box down to the end of the wall extensions;  and either make it larger or perhaps make it circular and add a wall or rim at the edge that goes up to ground level.  The wall's downwards extensions will get progressively darker as they go down, but by very little.   I'll make a prop that can be imported and used for this.

Here we go:

Sweet I was afraid it would not work.

15 minutes ago, DanW58 said:

 

(I should work for JPL...)

What is that ?

16 minutes ago, DanW58 said:

What's "strong subdivision"?  Never heard the term.

Just meant a lot of.  (also we have a 32768 limit of vertices per mesh) mostly say below 10k but sometimes artists happen

[DanW58]

1 hour ago, Stan` said:

What is that ?

JPL = Jet Propulsion Laboratory;  the Production department of NASA.  They have a habit of making super-long acronyms for spacecraft, etc.

 

1 hour ago, Stan` said:

Just meant a lot of.  (also we have a 32768 limit of vertices per mesh) mostly say below 10k but sometimes artists happen

Gottcha.  Yeah, that's why I said "one level of division" would probably not affect it.  Two is a different story.

The 32k limit per mesh is actually quite common;  just the fact that a 16-bit index can only address 64k objects, minus one bit for special things probably.  If you wanted to get rid of the limit, all indexes would have to be 32-bits, which is quite a jump.  And you can't have 20-bit.  Nothing to do with performance, anyhow.