Sign in to follow this  
Followers 0
fabio

[Fixed] - Shadows Misrender - Mesa Drivers Bug

29 posts in this topic

With some mesa drivers there is a wrong shadow behind some objects (example below) that goes away when disabling shadows in 0 A.D.. This is a known mesa clamping bug affecting r300, r300g, softpipe, llvmpipe, swrast mesa drivers and possibly others.

Other than the original shadow bug this thread discussed also two other graphic related problems. Overall status update:

Example:

post-8891-0-28785300-1351854341_thumb.jp

Edited by fabio

Share this post


Link to post
Share on other sites

Does the unit's icon (in the bottom/center) really look like in the screenshot? (That looks rather ugly - it shouldn't be like that...)

When you have the shadow problem, could you press "g" to disable the GUI and "`" to open the console and enter "?renderer.displayFrustum=true" ? That should make it render the shadow map in the top left corner, which might help show something about the problem.

Share this post


Link to post
Share on other sites

AHA, nice FOW bug...

And yes, the center icons all look blurry because they are being scaled up from 64x64 to something like 100x100. The only solution is to use bigger icon sheets.

Share this post


Link to post
Share on other sites

It's not just blurred, it's got patches of black and blue that really shouldn't be there.

Share this post


Link to post
Share on other sites
Does the unit's icon (in the bottom/center) really look like in the screenshot? (That looks rather ugly - it shouldn't be like that...)
Yes, some images are even uglier, e.g.:

http://imagebin.org/117060

When you have the shadow problem, could you press "g" to disable the GUI and "`" to open the console and enter "?renderer.displayFrustum=true" ? That should make it render the shadow map in the top left corner, which might help show something about the problem.

I get this:

http://imagebin.org/117061

Note: I pressed F9 to open the console.

Edited by fabio

Share this post


Link to post
Share on other sites
Yes, some images are even uglier
Hmm, looks kind of like a bug in the driver's S3TC handling, but I'm not sure why it only affects that texture. Maybe because it's scaled up, and/or has no mipmaps? Do you see similar texture issues anywhere else, e.g. if you do shift+D and disable "restrict camera" and zoom in so other textures get scaled up? Does it change if you edit binaries/data/mods/public/art/textures/ui/textures.xml to say mipmap="true"?
I get this:
That looks like the shadow map is getting generated correctly, and the horse's shadow is faint but clearly separated from the edge of the shadow map, but in the main display the dark edge cuts through the middle of it. So it seems the problem is somewhere between the generation of the shadow map and the application of it. I'm afraid I don't know enough about the rendering code or OpenGL or potential driver bugs etc to understand the problem more, though :victory:

(By the way, another useful console command is "?gameView.lockCullCamera=true", which locks the position of the camera used for shadow generation and lets you move around and see it from other angles. Probably won't help much here, though.)

Share this post


Link to post
Share on other sites
Hmm, looks kind of like a bug in the driver's S3TC handling, but I'm not sure why it only affects that texture. Maybe because it's scaled up, and/or has no mipmaps? Do you see similar texture issues anywhere else, e.g. if you do shift+D and disable "restrict camera" and zoom in so other textures get scaled up?
I looked at details of many units and buildings but I don't see other artifacts (this model has a grey face: http://imagebin.org/117106 but I think it's not an artifact).
Does it change if you edit binaries/data/mods/public/art/textures/ui/textures.xml to say mipmap="true"?
It's still the same.
That looks like the shadow map is getting generated correctly, and the horse's shadow is faint but clearly separated from the edge of the shadow map, but in the main display the dark edge cuts through the middle of it. So it seems the problem is somewhere between the generation of the shadow map and the application of it. I'm afraid I don't know enough about the rendering code or OpenGL or potential driver bugs etc to understand the problem more, though :victory:

(By the way, another useful console command is "?gameView.lockCullCamera=true", which locks the position of the camera used for shadow generation and lets you move around and see it from other angles. Probably won't help much here, though.)

I tried but it doesn't get any new info apparently.

I noticed also that (using a mesa driver compiled with debug info) sometimes (I can't find a way to reliably reproduce it) I get this warning:

Mesa warning: glDraw[Range]Elements(start 0, end 24, count 36, type 0x1403, indices=0xa7ebec88)
end is out of bounds (max=23) Element Buffer 0 (size 0)
This should probably be fixed in the application.

This looks like a 0ad bug and similar to an old nexuiz bug:

http://bugs.freedesktop.org/show_bug.cgi?id=22743

Edited by fabio

Share this post


Link to post
Share on other sites
I looked at details of many units and buildings but I don't see other artifacts (this model has a grey face: http://imagebin.org/117106 but I think it's not an artifact).
Hmm, looks like lots of artifacts to me :victory:. Should look like this - there's a big quality difference in the face, the grass, the back of the shield, etc. Just to check it's not a bug in the texture compression code, could you upload the file ~/.cache/0ad/art/textures/terrain/types/grass/grass1_spring.dds.*.dds ?
I noticed also that (using a mesa driver compiled with debug info) sometimes (I can't find a way to reliably reproduce it) I get this warning:

Mesa warning: glDraw[Range]Elements(start 0, end 24, count 36, type 0x1403, indices=0xa7ebec88)
end is out of bounds (max=23) Element Buffer 0 (size 0)
This should probably be fixed in the application.

Can you reproduce it often enough that you could catch it in a debugger? (Put a breakpoint on _mesa_warning, I think, then do a "bt full" if it hits it.)

Share this post


Link to post
Share on other sites

Hmm, looks like lots of artifacts to me :victory:. Should look like this - there's a big quality difference in the face, the grass, the back of the shield, etc. Just to check it's not a bug in the texture compression code, could you upload the file ~/.cache/0ad/art/textures/terrain/types/grass/grass1_spring.dds.*.dds ?

I never noticed they were artifacts. I uploaded the file here:

http://uploadbin.net/5fb586b421e7102e98449...365fa586346.dds

Can you reproduce it often enough that you could catch it in a debugger? (Put a breakpoint on _mesa_warning, I think, then do a "bt full" if it hits it.)

I'll try.

Share this post


Link to post
Share on other sites
Can you reproduce it often enough that you could catch it in a debugger?
Actually there's no need, I reproduced it myself - it fails when the vertex buffer allocator happens to allocate the final index of a buffer (which is why it seems random), since we pass the wrong end value to glDrawRangeElements. Fixed.

Share this post


Link to post
Share on other sites
I never noticed they were artifacts. I uploaded the file here:
Oh, interesting, it looks like the artifacts are introduced by the texture compression process, not by the rendering.

In that case:

If you delete ~/.cache/0ad/ then run the game again, does it still have those artifacts?

Are you using the default bundled copy of NVTT, not a different version (installed into /usr/include/nvtt etc)? What's the output of the "Building NVTT..." section when running update-workspaces.sh? What OS and CPU architecture are you on?

Share this post


Link to post
Share on other sites
Oh, interesting, it looks like the artifacts are introduced by the texture compression process, not by the rendering.

In that case:

If you delete ~/.cache/0ad/ then run the game again, does it still have those artifacts?

Yes.

Are you using the default bundled copy of NVTT, not a different version (installed into /usr/include/nvtt etc)? What's the output of the "Building NVTT..." section when running update-workspaces.sh?

Yes:

Building NVTT...

-- Setting optimal options
-- Processor: i686
-- Compiler Flags: -march=i686
-- Looking for OpenGL - found
-- Looking for GLUT - found
-- Looking for DirectX - not found
-- Looking for GLEW - not found
-- Looking for Cg - not found
-- Looking for CUDA - not found
-- Looking for Maya - not found
-- Looking for JPEG - found
-- Looking for PNG - found
-- Looking for TIFF - found
-- Looking for OpenEXR - not found
-- Use thread library: -lpthread
-- Configuring done
-- Generating done
-- Build files have been written to: /home/fabio/sorgenti/0ad/libraries/nvtt/src/build
[ 16%] Built target nvcore
[ 30%] Built target nvmath
[ 32%] Built target posh
[ 58%] Built target nvimage
[ 72%] Built target squish
[100%] Built target nvtt

What OS and CPU architecture are you on?

32 bit Ubuntu 10.10 (maverick) on a Core Duo cpu:

$ cat /proc/cpuinfo 
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 14
model name : Genuine Intel(R) CPU T2600 @ 2.16GHz
stepping : 8
cpu MHz : 2167.000
cache size : 2048 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc arch_perfmon bts aperfmperf pni monitor vmx est tm2 xtpr pdcm
bogomips : 4327.52
clflush size : 64
cache_alignment : 64
address sizes : 32 bits physical, 32 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 14
model name : Genuine Intel(R) CPU T2600 @ 2.16GHz
stepping : 8
cpu MHz : 2167.000
cache size : 2048 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
apicid : 1
initial apicid : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc arch_perfmon bts aperfmperf pni monitor vmx est tm2 xtpr pdcm
bogomips : 4327.55
clflush size : 64
cache_alignment : 64
address sizes : 32 bits physical, 32 bits virtual
power management:

Share this post


Link to post
Share on other sites

Aha, I can seemingly reproduce the texture compression if I compile with GCC 4.4.4, but not with 4.3.4 or 4.5.0. (This is on Gentoo amd64). So it looks like a GCC bug. But since Ubuntu defaults to GCC 4.4 I guess it's a bug we have to find a workaround for :victory:

Share this post


Link to post
Share on other sites

Hmm, actually (after painful investigation) it's a bug in NVTT, which peculiarly seems to be triggered only by that specific version of GCC, as far as I can tell.

I'll patch our bundled copy of NVTT tomorrow, which should fix this texture thing for you.

(The shadow issue is completely unrelated, though.)

Share this post


Link to post
Share on other sites

OK, the nvtt bug is fixed.

Is it true that 0ad tarballs will include only precompressed textures? Does providing precompressed textures avoid the "gray textures for some seconds the first time the game is run"? Does providing precompressed textures remove the dependency on nvtt?

Share this post


Link to post
Share on other sites
Is it true that 0ad tarballs will include only precompressed textures?
Yes. (And it's only precompressed textures because including both versions would substantially increase the download size, and most users will never want the uncompressed ones, and it should be easy enough to find the uncompressed ones via SVN.)
Does providing precompressed textures avoid the "gray textures for some seconds the first time the game is run"?
Yes.
Does providing precompressed textures remove the dependency on nvtt?
No. We want to support mods well, which means the game still needs to handle dynamic conversion of textures even if the standard bundled ones are preconverted.

Share this post


Link to post
Share on other sites
Does providing precompressed textures remove the dependency on nvtt?
No. We want to support mods well, which means the game still needs to handle dynamic conversion of textures even if the standard bundled ones are preconverted.

This may be a problem for linux distribution, then. The point of the disabled S3TC and the external libtxc_dxtn is that they included patented algorithm and the it isn't included by default. If nvtt will include the same libtxc_dxtn patented algorithm it still couldn't be distributed by linux distributions. At that point it could make sense to entirely drop nvtt (using it only for generating the compressed textures from the PNGs to distribute in tarballs) and suggest users install libtxc_dxtn.

Edited by fabio

Share this post


Link to post
Share on other sites

NVTT FAQ:

NVIDIA has a license of the S3TC patent that covers all our products, including our Texture Tools. You don't have to obtain a license of the S3TC patent to use any of NVIDIA's products
which I believe means Linux distros providing NVTT packages would be covered.

Most distros already seem to provide packages for DevIL, which includes a S3TC compression algorithm (which would be affected by the same patents as NVTT) and isn't covered by any form of patent license, so they should be no less willing to provide NVTT. Debian apparently dislikes only actively-enforced patents, and I don't believe the S3TC patent is enforced in this context (only in GPU implementations).

If the patent is a real problem, dropping NVTT wouldn't help the game anyway, since we have our own S3TC decompression algorithm in the engine (to decompress the textures at runtime when the user doesn't have S3TC support). We'd have to remove that too, and then the game would totally fail to run for most Linux users.

Share this post


Link to post
Share on other sites
If the patent is a real problem, dropping NVTT wouldn't help the game anyway, since we have our own S3TC decompression algorithm in the engine (to decompress the textures at runtime when the user doesn't have S3TC support). We'd have to remove that too, and then the game would totally fail to run for most Linux users.

OK, note however that decompression is not patented, only compression (indeed libtxc_dxtn.so only add compression, while decompression works by default in mesa when forcing S3TC).

Again on the nvtt bug, it seems it's fixed, however the grass and also other textures (the helmet and the shield) in my system are less bright than in your, don't know if this is another problem: your screen - my screen

Edited by fabio

Share this post


Link to post
Share on other sites

The patent covers both encoding and decoding.

I believe Mesa doesn't include a decoder. main/texcompress_s3tc.c just imports fetch_ext_rgb_dxt1 etc implemented by libtxc_dxtn, for when it needs software decoding. force_s3tc_enable works because all GPUs support S3TC decompression natively and Mesa simply has to pass the compressed texture data blindly to the hardware.

I don't see any colour differences in those screenshots - the matching areas (adjusting for rotation) look identical when I toggle between them, and seem to have very similar RGB values, so I'm not sure what problem you're seeing.

Share this post


Link to post
Share on other sites

Got a likely explanation/fix in that bug now. Is anyone able to confirm/deny this shadow problem on Mesa drivers other than r300g/swrast/softpipe/llvmpipe?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0