View Issue Details

IDProjectCategoryView StatusLast Update
0004660The Dark ModGraphicspublic18.05.2019 08:13
ReporterstgatilovAssigned Tostgatilov 
PrioritynormalSeveritynormalReproducibilitysometimes
Status suspendedResolutionopen 
Product VersionTDM 2.06 
Target VersionTDM 2.08Fixed in Version 
Summary0004660: Soft shadows: sometimes light leaks near edges (in complete darkness)
DescriptionSometimes surfaces become brighter near edges because of soft shadows. I'm speaking about the case when all surfaces are dark around the edge, and everything should be in shadow.

NOTE: this is not about the famous halo effect.
Steps To ReproduceSee screenshot from "Patently Dangerous".
It is in the outside area near the beginning of the mission.
TagsNo tags attached.

Relationships

related to 0004851 resolvedcabalistic Soft shadows with multisampling produces sparklies near wall edges 

Activities

stgatilov

stgatilov

11.11.2017 16:48

developer  

PatentlyDangerous_SoftShadows_EdgeLeak.jpg (411,795 bytes)
stgatilov

stgatilov

11.11.2017 16:53

developer   ~0009575

I don't understand where the algorithm manages to find unshadowed pixels near the edges. It seems to me that all the pixels surrounding the edge are in shadow, and I don't see any bright pixels without soft shadows.


I have found a workaround, which fixes the problem completely in this case. I'm pretty sure it should work well in other cases. It is: increase polygon offset for shadow volumes rendering.

The default settings are:
  r_shadowPolygonOffset -1
  r_shadowPolygonFactor 0
The issue does not happen with either this:
  r_shadowPolygonOffset -10
  r_shadowPolygonFactor 0
or with this:
  r_shadowPolygonOffset -1
  r_shadowPolygonFactor 1
stgatilov

stgatilov

11.11.2017 16:55

developer   ~0009576

Duzenko, could you guess where unshadowed pixels come from?
Is there any easy debug-way to see the contents of stencil buffer on screen?
stgatilov

stgatilov

11.11.2017 16:55

developer  

Quicksave_0.save (4,816,783 bytes)
stgatilov

stgatilov

11.11.2017 16:56

developer  

Quicksave_0.jpg (135,018 bytes)
Quicksave_0.jpg (135,018 bytes)
duzenko

duzenko

12.11.2017 19:42

developer   ~0009585

You can visualize the stencil texture via interaction.fs provided you have a non-zero in r_softshadowsquality.
Base value sampled from u_stencilTexture is 128. 129 is one shadow-deep, etc.
duzenko

duzenko

13.11.2017 11:43

developer   ~0009597

Last edited: 13.11.2017 11:43

View 2 revisions

Added debug stencil visualizer in the end of StencilSoftShadow in interaction.fs.
Ran out of time with it but it looks like a driver bug.
If sampling from base texcoord then it returns a correct value.
If sampling from base texcoord + const offset then it returns noise for some pixels.

stgatilov

stgatilov

13.11.2017 15:29

developer   ~0009598

What if stencil buffer really contains white pixels there, but we do not see them, because those pixels are not rendered with this face?
I also tried showing stencil buffer contents, and they are black for unshifted case and contains white for shifted case. I suspect that stencil buffer there is really wrong, but we do not see it with such a way of debug visualization.
stgatilov

stgatilov

13.11.2017 15:39

developer  

stencil.png (370,560 bytes)
stgatilov

stgatilov

13.11.2017 15:40

developer   ~0009599

Added stencil.png, where you can see white lines somewhere.
Those lines are blurred and generate the artifacts.
If you increase polygon offset for shadow volumes (as suggested above), those lines disappear.

P.S. To be honest, I wonder how I got this picture. On previous attempt, I did not saw the lines there...
stgatilov

stgatilov

13.11.2017 16:14

developer   ~0009601

Ok, I see those white lines in stencil buffer only when r_useFBO is disabled. I also have 2xAA in effect in this case. When I enable r_useFBO, these lines disappear. But the issue appears regardless of r_useFBO.
nbohr1more

nbohr1more

13.11.2017 16:24

developer   ~0009602

Hmm.

I wonder if GL_EXT_polygon_offset_clamp could be used to enhance r_shadowPolygonOffset ?

https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_polygon_offset_clamp.txt

https://image.slidesharecdn.com/siggraphkilgardnvidiaopenglin2017-170731201008/95/nvidia-opengl-46-in-2017-40-638.jpg?cb=1502125537
stgatilov

stgatilov

13.11.2017 16:49

developer   ~0009603

Is there any problem in using "r_shadowPolygonFactor 1" without any clamping?
nbohr1more

nbohr1more

13.11.2017 17:05

developer   ~0009604

Last edited: 13.11.2017 17:11

View 2 revisions

I don't think so.
Shadow Polygon Offset 1 probably good enough in 99.9999% of cases.

I just remembered seeing GL_EXT_polygon_offset_clamp awhile back and thought I would ping Duzenko to see if it looks useful.

We manually adjusted regular surface offset factors in 2.03 so I don't see why we can't change shadow polygon offsets in 2.06.

Does the change cause any adverse effects when soft shadows aren't enabled? (I doubt it).

duzenko

duzenko

13.11.2017 19:22

developer   ~0009605

Last edited: 13.11.2017 19:22

View 2 revisions

So you suspect stencil might not have the default value for pixels not on the interaction faces? The only way to test that is to draw a full screen quad. I will try that when I have the time.
Thanks Nbohr1more for bringing depth test in the conversation. When I think about it the noise did look a lot like depth fighting.

duzenko

duzenko

13.11.2017 20:21

developer   ~0009606

I had to fix the no-primary-fbo soft-shadow operation first on Intel so did not have the time for fullscreen quads tonight but in the process I noticed this thing.
I start with primary fbo off and SS on. Glitch is there.
I turn the primary fbo on -> glitch goes away.
I turn off the P-FBO off -> glitch does not come back.
How does it go for you?
Also, please test that I did not break the non-Intel path while fixing on Intel.
nbohr1more

nbohr1more

14.11.2017 02:00

developer   ~0009609

The latest build still renders just fine on Nvidia.

That said, I will relay that the first note on "Closemouthed Shadows" has some sorta rendering issue with a self-shadow that doesn't exist when you enable f_useFBO.
stgatilov

stgatilov

20.11.2017 05:33

developer   ~0009642

VanishedOne wrote me about two places where he saw light leaking with soft shadows only, and in both places setting "r_shadowPolygonFactor 1" fixed the issue.

Maybe we simply change r_shadowPolygonFactor to 1 by default and see how the beta continues?
nbohr1more

nbohr1more

20.11.2017 05:52

developer   ~0009643

I noticed hairline polygon cracks in the shadows for the first guard in "The Thieves" when setting r_shadowPolygonFactor 1 without soft shadows but I haven't noticed any widespread problems with the setting.

I guess, ideally, this would hard code to 0 if soft shadows are off?

At the very least, it should be set to cvar_archive so that you can save your preferred setting from session to session.
stgatilov

stgatilov

20.11.2017 16:14

developer   ~0009644

All these cvars which must behave differently depending on other cvars is too complicated, isn't it?
If setting cvar to 1 is not an option, then it would be better to remove the cvar, and compute it on-demand depending on soft shadows cvars. But this reduces game tweakability...
Alternatively, we can add value -1, which means something like "auto" or "default", and make if default. And in this case effectively set 0 or 1 depending on soft shadows presence.
nbohr1more

nbohr1more

20.11.2017 16:50

developer   ~0009645

I like your last idea "-1 == auto"

I'm not terribly inclined to object to a default of 1 either as long as the cvar can be archived and we inform users about the caveats of the new default with hard shadows.
stgatilov

stgatilov

21.11.2017 17:14

developer   ~0009663

After some consideration, this idea is weird. Value -1 is quite valid for polygon offset factor: it is just the same as value 1, but with offsetting to the opposite side.
One the other hand, we can set default value of r_shadowPolygonFactor to be "default", and then check for it everywhere. But it is rather awkward to set string value to a float variable =)
stgatilov

stgatilov

21.11.2017 17:47

developer   ~0009664

Ok, I have checked the Thieves mission and I understand what you mean.

It seems that those artifacts are caused by using offsetfactor. This is because offset is different for adjacent faces of the mesh (unless they are parallel), so a crack appears between them.
For instance, it is described here:
  https://www.opengl.org/discussion_boards/archive/index.php/t-147627.html

To be honest, I do not feel that depth fighing the real issue here. Maybe we should simply leave things as they are? at least until we understand where this light comes from...
nbohr1more

nbohr1more

21.11.2017 18:18

developer   ~0009665

Last edited: 21.11.2017 18:19

View 3 revisions

Easy workaround for 2.06

Set the cvars to CVAR_ARCHIVE in RenderSystem_init.cpp

idCVar r_shadowPolygonOffset( "r_shadowPolygonOffset", "-1", CVAR_RENDERER | CVAR_FLOAT | CVAR_ARCHIVE, "bias value added to depth test for stencil shadow drawing" );
idCVar r_shadowPolygonFactor( "r_shadowPolygonFactor", "0", CVAR_RENDERER | CVAR_FLOAT | CVAR_ARCHIVE, "scale value for stencil shadow drawing" );

Let user know they can cure some shadow leakage via r_shadowPolygonFactor 1.

Move bug to 2.07 for further investigation.

duzenko

duzenko

22.11.2017 11:46

developer   ~0009673

Last edited: 22.11.2017 11:50

View 2 revisions

I tried disabled culling for stencil visualization but failed.
What works for me is r_showshadows 1 and then step inside the building corner shadow. (also r_singlelight 87 helps here)
Screenshot attached
What I can see here is a shadow surface flickering on top of the wall, classic depth fighting case.
One way to fix it is to merge both corner walls in one surface so it casts a single shadow.
I'm not sure what the flickering surface is - is it SV front cap or ???

duzenko

duzenko

23.11.2017 10:14

developer   ~0009680

Current version appears to be able to render the back faces for stencil debugging.
I set r_lightallbackfaces as exe run parameters in VS debugging options or alternatively you can quick save/load after toggling it on the fly.
Attached screenshot with stencil visualization on both front and back faces.
duzenko

duzenko

23.11.2017 10:15

developer  

Untitled.jpg (169,030 bytes)
Untitled.jpg (169,030 bytes)
stgatilov

stgatilov

23.11.2017 17:19

developer   ~0009685

Last edited: 23.11.2017 17:22

View 4 revisions

If you disable r_useFBO, then you will see the same issue without any flickering. Stencil buffer contains zero inside the wall. At least that is what I see when I use your debug code with X shift and without FBO.

It seems this effect shows every time you have a wall, and some static box object at zero distance from it (i.e. sharing a plane). In such case there is a whole vertical line which makes a reflex angle between the wall and the object. If you place a light in front of the object, so the the line is in shadow, then you get this light leaking.

I have tried to position a candle near several static objects close to walls, and I can reproduce it in all cases.
EDIT: Ok, not in all cases. But in many cases the issue shows up.

stgatilov

stgatilov

23.11.2017 17:25

developer  

PatentlyDangerous_LeakAgain.jpg (485,274 bytes)
stgatilov

stgatilov

23.11.2017 17:28

developer   ~0009686

Added a new image (LeakAgain) near the fountain.
In this case the issue happens only with r_useFBO 0.
I suppose that there is something wrong with rendering stencil in such cases, but it leads to different results in FBO and non-FBO cases. Like if some buffer is not cleared between frames or something like that.
stgatilov

stgatilov

24.11.2017 06:05

developer   ~0009687

The problem disappears if I dmap the map with shadowOptLevel set to SO_NONE instead of SO_MERGE_SURFACES. Strangely enough, this is true even if R_CreateShadowVolume is called with SO_STATIC, so the key difference is not in R_CreateShadowVolume.
nbohr1more

nbohr1more

26.04.2018 03:04

developer   ~0010402

Halo reduction shaders 15171, 15172, 15173 have significantly reduced this problem but it still persists.

r_shadowPolygonFactor 1

is still the best option
nbohr1more

nbohr1more

10.05.2018 01:58

developer   ~0010456

Last edited: 10.05.2018 03:10

View 5 revisions

I think this is a pretty easy workaround:

draw_stencil.cpp

// If softshadows is enabled and r_shadowPolygonFactor is 0 change it to 1
// otherwise carry on using the cvars as originally intended

if ( r_softShadowsQuality.GetBool() && !r_shadowPolygonFactor.GetBool() ) {
        qglPolygonOffset ( 1.0f, -r_shadowPolygonOffset.GetFloat() );
        qglEnable (GL_POLYGON_OFFSET_FILL);
    } else {
 if ( r_shadowPolygonFactor.GetFloat() || r_shadowPolygonOffset.GetFloat() )
        qglPolygonOffset( r_shadowPolygonFactor.GetFloat(), -r_shadowPolygonOffset.GetFloat() );
        qglEnable( GL_POLYGON_OFFSET_FILL );
    }


Edit: seems to work as expected

(edited version attached)

nbohr1more

nbohr1more

10.05.2018 03:10

developer  

4660_draw_stencil.cpp (8,529 bytes)
stgatilov

stgatilov

10.05.2018 16:23

developer   ~0010459

We still know that "r_shadowPolygonFactor 1" adds issues to stencil shadows. So this workaround is not perfect.
Besides, no one knows why the problem exists, and how this workaround hides it.
nbohr1more

nbohr1more

10.05.2018 16:31

developer   ~0010460

In the above, we aren't really changing the cvar though.

We are changing it's affect on the renderer so the cvar remains at 0
but the renderer acts as if it's 1.

It's basically a blacklist for a value-combo we know to be broken.
"Never let Polygon Factor = 0 if soft shadows is on".

I think this would reduce a lot of confusion for end users in 2.06
and we could do a deeper dive in 2.07?
nbohr1more

nbohr1more

11.12.2018 03:06

developer   ~0010865

In Patently Dangerous 2.07\SVN this problem barely exists now.
It is also cured in Shadow Map mode.

I suggest we move this to 2.08 or remove the release version
 target and suspend it.
stgatilov

stgatilov

11.12.2018 04:10

developer   ~0010867

I don't mind.

The problem seems to happen rarely, and we did not manage to fully understand why it happens despite much time. The anti-halo patch diminishes the problem, and shadow maps are not affected at all.
STiFU

STiFU

20.01.2019 14:12

developer   ~0011445

Is this the same issue?
http://forums.thedarkmod.com/topic/19774-beta-testing-207/?p=433190

Issue History

Date Modified Username Field Change
11.11.2017 16:48 stgatilov New Issue
11.11.2017 16:48 stgatilov Status new => assigned
11.11.2017 16:48 stgatilov Assigned To => stgatilov
11.11.2017 16:48 stgatilov File Added: PatentlyDangerous_SoftShadows_EdgeLeak.jpg
11.11.2017 16:53 stgatilov Note Added: 0009575
11.11.2017 16:55 stgatilov Note Added: 0009576
11.11.2017 16:55 stgatilov File Added: Quicksave_0.save
11.11.2017 16:56 stgatilov File Added: Quicksave_0.jpg
12.11.2017 19:42 duzenko Note Added: 0009585
13.11.2017 11:43 duzenko Note Added: 0009597
13.11.2017 11:43 duzenko Note Edited: 0009597 View Revisions
13.11.2017 15:29 stgatilov Note Added: 0009598
13.11.2017 15:39 stgatilov File Added: stencil.png
13.11.2017 15:40 stgatilov Note Added: 0009599
13.11.2017 16:14 stgatilov Note Added: 0009601
13.11.2017 16:24 nbohr1more Note Added: 0009602
13.11.2017 16:49 stgatilov Note Added: 0009603
13.11.2017 17:05 nbohr1more Note Added: 0009604
13.11.2017 17:11 nbohr1more Note Edited: 0009604 View Revisions
13.11.2017 19:22 duzenko Note Added: 0009605
13.11.2017 19:22 duzenko Note Edited: 0009605 View Revisions
13.11.2017 20:21 duzenko Note Added: 0009606
14.11.2017 02:00 nbohr1more Note Added: 0009609
20.11.2017 05:33 stgatilov Note Added: 0009642
20.11.2017 05:52 nbohr1more Note Added: 0009643
20.11.2017 16:14 stgatilov Note Added: 0009644
20.11.2017 16:50 nbohr1more Note Added: 0009645
21.11.2017 17:14 stgatilov Note Added: 0009663
21.11.2017 17:47 stgatilov Note Added: 0009664
21.11.2017 18:18 nbohr1more Note Added: 0009665
21.11.2017 18:19 nbohr1more Note Edited: 0009665 View Revisions
21.11.2017 18:19 nbohr1more Note Edited: 0009665 View Revisions
22.11.2017 04:28 stgatilov Target Version => TDM 2.07
22.11.2017 11:46 duzenko Note Added: 0009673
22.11.2017 11:46 duzenko File Added: Untitled.jpg
22.11.2017 11:50 duzenko Note Edited: 0009673 View Revisions
23.11.2017 10:14 duzenko Note Added: 0009680
23.11.2017 10:15 duzenko File Deleted: Untitled.jpg
23.11.2017 10:15 duzenko File Added: Untitled.jpg
23.11.2017 17:19 stgatilov Note Added: 0009685
23.11.2017 17:20 stgatilov Note Edited: 0009685 View Revisions
23.11.2017 17:20 stgatilov Note Edited: 0009685 View Revisions
23.11.2017 17:22 stgatilov Note Edited: 0009685 View Revisions
23.11.2017 17:25 stgatilov File Added: PatentlyDangerous_LeakAgain.jpg
23.11.2017 17:28 stgatilov Note Added: 0009686
24.11.2017 06:05 stgatilov Note Added: 0009687
26.04.2018 03:04 nbohr1more Note Added: 0010402
10.05.2018 01:58 nbohr1more Note Added: 0010456
10.05.2018 01:59 nbohr1more Status assigned => feedback
10.05.2018 02:08 nbohr1more Note Edited: 0010456 View Revisions
10.05.2018 02:36 nbohr1more Note Edited: 0010456 View Revisions
10.05.2018 02:36 nbohr1more Note Edited: 0010456 View Revisions
10.05.2018 03:10 nbohr1more File Added: 4660_draw_stencil.cpp
10.05.2018 03:10 nbohr1more Note Edited: 0010456 View Revisions
10.05.2018 16:23 stgatilov Note Added: 0010459
10.05.2018 16:23 stgatilov Status feedback => assigned
10.05.2018 16:31 nbohr1more Note Added: 0010460
25.06.2018 06:11 stgatilov Relationship added related to 0004851
11.12.2018 03:06 nbohr1more Note Added: 0010865
11.12.2018 03:06 nbohr1more Status assigned => feedback
11.12.2018 04:10 stgatilov Note Added: 0010867
11.12.2018 04:10 stgatilov Status feedback => assigned
11.12.2018 04:21 nbohr1more Target Version TDM 2.07 => TDM 2.08
20.01.2019 14:12 STiFU Note Added: 0011445
18.05.2019 08:13 stgatilov Status assigned => suspended