View Issue Details

IDProjectCategoryView StatusLast Update
0005815The Dark ModCodingpublic01.01.2024 11:46
Reporterstgatilov Assigned Tostgatilov  
PrioritynormalSeveritymajorReproducibilitysometimes
Status resolvedResolutionfixed 
Product VersionTDM 2.08 
Target VersionTDM 2.10Fixed in VersionTDM 2.10 
Summary0005815: Projected lights behavior has changed since 2.08
DescriptionIt turned out that merging some code form D3BFG changed behavior of projected lights, breaking them on some maps.

The difference resides in R_DeriveLightData function, which builds "light projection matrix", light frustum and polytope.
In D3BFG, it calls R_ComputeSpotLightProjectionMatrix (as well as R_ComputeParallelLightProjectionMatrix and R_ComputePointLightProjectionMatrix).
In Doom 3, it calls R_SetLightProject.
Their behavior is quite different.

As explained below, the behavior became different in 2.08 and 2.09.
Steps To ReproducePerhaps the best way to test is to use test_5815_spotlights FM in assets SVN.
Additional InformationForum discussion:
  https://forums.thedarkmod.com/index.php?/topic/21117-light-frustums-different-in-dr-and-tdm/#comment-465848
TagsNo tags attached.

Relationships

related to 0003727 new idLight::GetBounds() should handle projected lights correctly 
related to 0004888 resolvedstgatilov Terrible precision in light frustums 
related to 0005816 resolvedduzenko Volumetric lights (aka "god rays") 
related to 0005529 resolvedduzenko Projected lights can't use appropriate falloff images 
related to 0005876 resolvedstgatilov Stripped bright line on the boundary of light volume 

Activities

stgatilov

stgatilov

28.11.2021 11:45

administrator   ~0014571

Copying analysis of Doom 3 and BFG code for spotlights from the forums thread...
stgatilov

stgatilov

28.11.2021 11:46

administrator   ~0014572

Last edited: 28.11.2021 11:46

Original Doom 3 behavior

1) The inputs are the same as in the previous analysis: R, U, T are unit vectors from spawnargs, and W, H, D (aka width, height, and depth) are the lengths of spawnargs.
2) It looks like "z" is the perspective divisor here, but it is applied only to "x" and "y". The falloff coordinate "w" should not undergo perspective division.
3) You can see decomposition of matrix into three transforms, which produce homogeneous coordinates for "x" and "y" in range [0..1] both. Computation of the falloff coordinate "w" is totally independent of this transformation.
4) Falloff parameter at a point is determined by its projection to "start"-"end" line segment: it is 0 at start, 1 at end, 0.5 in the middle, etc. In fact, falloff parameter changes linearly from 0 to 1 while going from S to E. Note that it does not depend on the frustum at all: you can e.g. have frustum direction and falloff direction different.
5) Frustum half-angles are same as in BFG case: length(right) / length(target) and length(up) / length(target).
6) If R/U/T triple is not orthogonal, then things go equally uncertain as in BFG case. Except that here T is made orthogonal to R/U plane, and becomes N. At the same time, size D is adjusted too: we take length of "target"'s projection along N instead of full length of "target". This perhaps gives some sense to the whole stuff when R and U are orthogonal but T is not.

According to p.6, this code supports T not being orthogonal to R/U plane in some sense. In fact, only the component of "target" orthogonal to R/U plane matters, the component along R/U plane is dropped away. Note that you cannot create a non-orthogonal light frustum, so this "support" does not offer any additional flexibility.

The case of non-orthogonal R and U still looks like unsupported/unintended.

Here is how this light can be set up:
1) Specify X/Y coordinates on frustum "screen" by choosing orthogonal "right" and "up" vectors.
2) Specify direction of light frustum by setting "target" vector. Only the orientation (which side of the plane "target" looks into) matters. If you set non-orthogonal "target", then it is automatically replaced by its projection onto X/Y plane normal in the engine.
3) Set lengths of "right" and "up" in such way that (target +/- up +/- right) vectors look through frustum corners. Here "target" is the orthogonal vector already "fixed" by the engine if you set it non-orthogonal.
4) Choose "start" and "end" points completely independent of all the previous parameters. Falloff parameter ranges from 0.0 to 1.0 as you move point from "start" to "end". Gradient of "falloff" parameter goes along "end" - "start" everywhere.
stgatilov

stgatilov

28.11.2021 11:49

administrator   ~0014573

Original Doom 3 BFG behavior

1) The inputs are the same as usual: R, U, T are unit vectors from spawnargs, and W, H, D (aka width, height, and depth) are the lengths of spawnargs.
2) Coordinate "w" is divisor for "x" and "y", but not for "z".
3) You can see decomposition of matrix into three transforms, which produce homogeneous coordinates for "x" and "y" in range [0..1] both. Computation of the falloff coordinate "w" is totally independent of this transformation.
4) Coordinate "z" (falloff) must be scaled down by (n+f), which is achieved by returning 1 / (n+f) from R_ComputeSpotLightProjectionMatrix, then multiplying matrix[2] row by it.
5) "start"/"end" spawnargs define linear gradient of falloff parameter, but only their component along "target" vector matters, the two other components don't affect anything.
6) Let n = dot(target, start), f = dot(target, end). Falloff = 0 is at distance n, and falloff = 1 at distance (f + n). Quite surprisingly, "end" vector does not define the end of falloff texture, but ("start" + "end") does.
7) Frustum half-angles are: length(right) / length(target) and length(up) / length(target). Multiplying all three vectors by same coefficient does not change anything, only their length ratios matter.
8) If R/U/T triple is not orthogonal, then the matrix which I called "rotation" is no longer orthogonal. Strictly speaking, it maps a parallelepiped with face normals R, U, T respectively into axis-aligned box. The vectors are not axes of the local coordinate system (I guess they are called "cobasis"), they are not normals of frustum planes.

Judging from point 8, I don't think non-orthogonal R/U/T were ever intended.
However, if there is no special tools, drawing/specifying three exactly orthogonal vectors is very hard. If mapper sets almost orthogonal vectors, then the transformation will work almost as if they were orthogonal...

I'd say the spawnargs should be set as follows:

1) Set frustum direction into "target" vector, choose length arbitrarily.
2) Specify X/Y coordinates on frustum "screen" by choosing orthogonal "right" and "up" vectors.
3) Set lengths of "right" and "up" in such way that (target +/- up +/- right) vectors look through frustum corners.
4) Choose "start" and "end" so that falloff is zero at "start", and unit at "start" + "end".
stgatilov

stgatilov

28.11.2021 11:51

administrator   ~0014574

Also the important thing is that D3BFG matrix can be used as ordinary OpenGL projection matrix.
I.e. divide x, y, and Z by perspecitve divisor w.
In such case z/w (aka depth) ranges from 0 to 1 as distance along target vector goes from "near" to "far" + "near".
This allows to use this matrix in frontend culling using e.g. idRenderMatrix::ProjectedBounds method.
stgatilov

stgatilov

28.11.2021 12:12

administrator   ~0014575

Here is the list of related commits so far.

Fixing lighting outside light polytope (https://bugs.thedarkmod.com/view.php?id=5529#c14557):
  r9640 projected lights shining backwards
  r9645 Extracted common code for handling light projection/falloff into single place (common include).
  r9646 Ensure there is no light outside of the frustum (i.e. when falloff or projection UV are outside texture bounds).

Changes for related issue 0004888 (large rounding errors in light polytope):
  r9658 Added new function R_PolytopeSurfaceFrustumLike for precise light volume polytope construction.
  r9659 Use the new R_PolytopeSurfaceFrustumLike function to create frustum windings and triagnles.
  r9663 Removed frustumTrisExact, hopefully set frustumTri orientation correctly
  r9664 Added good check and fixed winding orientations in R_PolytopeSurfaceFrustumLike.
  r9665 Removed old debug code for frustum triangles, based on plane distances.

Fixing r_showLights to visually debug the issue:
  r9653 temp fix for r_showLights on compatibility profiles
  r9654 Fundamental fix for r_showLights on all profiles
  r9662 Render BFG-style frustum under r_showLights & 8.



Some preliminary tweaking of 1/f vs 1/(n+f):
  r9636 BFG lights: changed the projection matrix to D3/DR convention with the end point Volumetric lights: bugfix
  r9657 Reverted R_ComputeSpotLightProjectionMatrix to work as in D3BFG.

Finally, the main changes about light projection matrix:
  r9660 Preliminary refactoring of R_DeriveLightData.
!!! r9661 Implemented D3-style light projection/falloff behavior under cvar.
stgatilov

stgatilov

28.11.2021 12:23

administrator   ~0014577

I have added test FM: test_5815_spotlights
It contains various (hopefully all) cases of weird light frustums, including even the case when light polytope is unbounded in Doom 3.
Looking through releases, I managed to learn which version has which behavior.

TDM 2.00, 2.04, 2.05, 2.06, 2.07:
  The lit voxels are exactly the same as on the current SVN with "spotlightBehavior 0".
  The only exception is the light with unbounded volume (yep, it is truly unbounded/infinite in 2.00).
  The floor is lit under "nonortho" + ceiling sometimes too (the bug described in https://bugs.thedarkmod.com/view.php?id=5529#c14557)

TDM 2.08, 2.09:
  Exactly the same voxels are lit as on: SVN + "spotlightBehavior 1" with zScale = 1/far
  Floor and ceiling is lit both for "weirdstartend" and "onortho".

The BFG code got into TDM by 2.05, due to commit by anonreclaimer.
However, it was disabled by cvar until 2.08 (probably someone noticed the problem and disabled it):
  idCVar r_useAnonreclaimer( "r_useBfgPortalCulling", "0", CVAR_RENDERER | CVAR_BOOL | CVAR_ARCHIVE, "test anonreclaimer culling patch" );

Finally, it was enabled for 2.08:
  Revision: 8100
  Author: duzenko
  Date: 29 march 2019 г. 23:57:44
  Message: Forced anonreclaimer's culling codepath
stgatilov

stgatilov

01.12.2021 15:59

administrator   ~0014580

Hopefully, the final fix here:
  r9666. Changed r_spotlightBehavior cases, description, and default behavior.

The behavior of TDM 2.07 and before is restored as default.
Other behaviors are available only for debugging purposes under r_spotlightBehavior cvar.
stgatilov

stgatilov

01.01.2024 11:46

administrator   ~0016307

In svn rev 10608 (0005172), I have switched some light culling back from BFG frustum to Doom 3 frustum.
I think I'll eradicate BFG-style frustum for lights.
Having two different frustums is awful, and I doubt BFG code provides noticeable performance boost.

Issue History

Date Modified Username Field Change
18.11.2021 17:45 stgatilov New Issue
18.11.2021 17:45 stgatilov Status new => assigned
18.11.2021 17:45 stgatilov Assigned To => stgatilov
18.11.2021 17:47 stgatilov Relationship added related to 0004888
18.11.2021 17:49 stgatilov Relationship added related to 0003727
19.11.2021 04:39 stgatilov Additional Information Updated
19.11.2021 04:40 stgatilov Relationship added related to 0005816
21.11.2021 06:40 stgatilov Relationship added related to 0005529
28.11.2021 11:38 stgatilov Summary Projected lights behavior is difference since 2.05 => Projected lights behavior has changed
28.11.2021 11:42 stgatilov Product Version TDM 2.05 => TDM 2.08
28.11.2021 11:42 stgatilov Summary Projected lights behavior has changed => Projected lights behavior has changed since 2.08
28.11.2021 11:42 stgatilov Description Updated
28.11.2021 11:45 stgatilov Note Added: 0014571
28.11.2021 11:46 stgatilov Note Added: 0014572
28.11.2021 11:46 stgatilov File Added: DeriveLightData_Spot_D3.pdf
28.11.2021 11:46 stgatilov Note Edited: 0014572
28.11.2021 11:49 stgatilov Note Added: 0014573
28.11.2021 11:49 stgatilov File Added: DeriveLightData_Spot_BFG_v2.pdf
28.11.2021 11:51 stgatilov Note Added: 0014574
28.11.2021 12:12 stgatilov Note Added: 0014575
28.11.2021 12:23 stgatilov Note Added: 0014577
28.11.2021 12:26 stgatilov Description Updated
28.11.2021 12:26 stgatilov Steps to Reproduce Updated
01.12.2021 15:59 stgatilov Note Added: 0014580
01.12.2021 15:59 stgatilov Status assigned => feedback
11.01.2022 05:02 stgatilov Relationship added related to 0005876
30.01.2022 21:13 nbohr1more Status feedback => resolved
30.01.2022 21:13 nbohr1more Resolution open => fixed
30.01.2022 21:13 nbohr1more Fixed in Version => TDM 2.10
01.01.2024 11:46 stgatilov Note Added: 0016307