View Issue Details
ID  Project  Category  View Status  Date Submitted  Last Update 

0005815  The Dark Mod  Coding  public  18.11.2021 17:45  30.01.2022 21:13 
Reporter  stgatilov  Assigned To  stgatilov  
Priority  normal  Severity  major  Reproducibility  sometimes 
Status  resolved  Resolution  fixed  
Product Version  TDM 2.08  
Target Version  TDM 2.10  Fixed in Version  TDM 2.10  
Summary  0005815: Projected lights behavior has changed since 2.08  
Description  It turned out that merging some code form D3BFG changed behavior of projected lights, breaking them on some maps. The difference resides in R_DeriveLightData function, which builds "light projection matrix", light frustum and polytope. In D3BFG, it calls R_ComputeSpotLightProjectionMatrix (as well as R_ComputeParallelLightProjectionMatrix and R_ComputePointLightProjectionMatrix). In Doom 3, it calls R_SetLightProject. Their behavior is quite different. As explained below, the behavior became different in 2.08 and 2.09.  
Steps To Reproduce  Perhaps the best way to test is to use test_5815_spotlights FM in assets SVN.  
Additional Information  Forum discussion: https://forums.thedarkmod.com/index.php?/topic/21117lightfrustumsdifferentindrandtdm/#comment465848  
Tags  No tags attached.  
related to  0003727  new  idLight::GetBounds() should handle projected lights correctly  
related to  0004888  resolved  stgatilov  Terrible precision in light frustums 
related to  0005816  resolved  duzenko  Volumetric lights (aka "god rays") 
related to  0005529  resolved  duzenko  Projected lights can't use appropriate falloff images 
related to  0005876  resolved  stgatilov  Stripped bright line on the boundary of light volume 
Copying analysis of Doom 3 and BFG code for spotlights from the forums thread...  
Original Doom 3 behavior 1) The inputs are the same as in the previous analysis: R, U, T are unit vectors from spawnargs, and W, H, D (aka width, height, and depth) are the lengths of spawnargs. 2) It looks like "z" is the perspective divisor here, but it is applied only to "x" and "y". The falloff coordinate "w" should not undergo perspective division. 3) You can see decomposition of matrix into three transforms, which produce homogeneous coordinates for "x" and "y" in range [0..1] both. Computation of the falloff coordinate "w" is totally independent of this transformation. 4) Falloff parameter at a point is determined by its projection to "start""end" line segment: it is 0 at start, 1 at end, 0.5 in the middle, etc. In fact, falloff parameter changes linearly from 0 to 1 while going from S to E. Note that it does not depend on the frustum at all: you can e.g. have frustum direction and falloff direction different. 5) Frustum halfangles are same as in BFG case: length(right) / length(target) and length(up) / length(target). 6) If R/U/T triple is not orthogonal, then things go equally uncertain as in BFG case. Except that here T is made orthogonal to R/U plane, and becomes N. At the same time, size D is adjusted too: we take length of "target"'s projection along N instead of full length of "target". This perhaps gives some sense to the whole stuff when R and U are orthogonal but T is not. According to p.6, this code supports T not being orthogonal to R/U plane in some sense. In fact, only the component of "target" orthogonal to R/U plane matters, the component along R/U plane is dropped away. Note that you cannot create a nonorthogonal light frustum, so this "support" does not offer any additional flexibility. The case of nonorthogonal R and U still looks like unsupported/unintended. Here is how this light can be set up: 1) Specify X/Y coordinates on frustum "screen" by choosing orthogonal "right" and "up" vectors. 2) Specify direction of light frustum by setting "target" vector. Only the orientation (which side of the plane "target" looks into) matters. If you set nonorthogonal "target", then it is automatically replaced by its projection onto X/Y plane normal in the engine. 3) Set lengths of "right" and "up" in such way that (target +/ up +/ right) vectors look through frustum corners. Here "target" is the orthogonal vector already "fixed" by the engine if you set it nonorthogonal. 4) Choose "start" and "end" points completely independent of all the previous parameters. Falloff parameter ranges from 0.0 to 1.0 as you move point from "start" to "end". Gradient of "falloff" parameter goes along "end"  "start" everywhere. 

Original Doom 3 BFG behavior 1) The inputs are the same as usual: R, U, T are unit vectors from spawnargs, and W, H, D (aka width, height, and depth) are the lengths of spawnargs. 2) Coordinate "w" is divisor for "x" and "y", but not for "z". 3) You can see decomposition of matrix into three transforms, which produce homogeneous coordinates for "x" and "y" in range [0..1] both. Computation of the falloff coordinate "w" is totally independent of this transformation. 4) Coordinate "z" (falloff) must be scaled down by (n+f), which is achieved by returning 1 / (n+f) from R_ComputeSpotLightProjectionMatrix, then multiplying matrix[2] row by it. 5) "start"/"end" spawnargs define linear gradient of falloff parameter, but only their component along "target" vector matters, the two other components don't affect anything. 6) Let n = dot(target, start), f = dot(target, end). Falloff = 0 is at distance n, and falloff = 1 at distance (f + n). Quite surprisingly, "end" vector does not define the end of falloff texture, but ("start" + "end") does. 7) Frustum halfangles are: length(right) / length(target) and length(up) / length(target). Multiplying all three vectors by same coefficient does not change anything, only their length ratios matter. 8) If R/U/T triple is not orthogonal, then the matrix which I called "rotation" is no longer orthogonal. Strictly speaking, it maps a parallelepiped with face normals R, U, T respectively into axisaligned box. The vectors are not axes of the local coordinate system (I guess they are called "cobasis"), they are not normals of frustum planes. Judging from point 8, I don't think nonorthogonal R/U/T were ever intended. However, if there is no special tools, drawing/specifying three exactly orthogonal vectors is very hard. If mapper sets almost orthogonal vectors, then the transformation will work almost as if they were orthogonal... I'd say the spawnargs should be set as follows: 1) Set frustum direction into "target" vector, choose length arbitrarily. 2) Specify X/Y coordinates on frustum "screen" by choosing orthogonal "right" and "up" vectors. 3) Set lengths of "right" and "up" in such way that (target +/ up +/ right) vectors look through frustum corners. 4) Choose "start" and "end" so that falloff is zero at "start", and unit at "start" + "end". 

Also the important thing is that D3BFG matrix can be used as ordinary OpenGL projection matrix. I.e. divide x, y, and Z by perspecitve divisor w. In such case z/w (aka depth) ranges from 0 to 1 as distance along target vector goes from "near" to "far" + "near". This allows to use this matrix in frontend culling using e.g. idRenderMatrix::ProjectedBounds method. 

Here is the list of related commits so far. Fixing lighting outside light polytope (https://bugs.thedarkmod.com/view.php?id=5529#c14557): r9640 projected lights shining backwards r9645 Extracted common code for handling light projection/falloff into single place (common include). r9646 Ensure there is no light outside of the frustum (i.e. when falloff or projection UV are outside texture bounds). Changes for related issue 0004888 (large rounding errors in light polytope): r9658 Added new function R_PolytopeSurfaceFrustumLike for precise light volume polytope construction. r9659 Use the new R_PolytopeSurfaceFrustumLike function to create frustum windings and triagnles. r9663 Removed frustumTrisExact, hopefully set frustumTri orientation correctly r9664 Added good check and fixed winding orientations in R_PolytopeSurfaceFrustumLike. r9665 Removed old debug code for frustum triangles, based on plane distances. Fixing r_showLights to visually debug the issue: r9653 temp fix for r_showLights on compatibility profiles r9654 Fundamental fix for r_showLights on all profiles r9662 Render BFGstyle frustum under r_showLights & 8. Some preliminary tweaking of 1/f vs 1/(n+f): r9636 BFG lights: changed the projection matrix to D3/DR convention with the end point Volumetric lights: bugfix r9657 Reverted R_ComputeSpotLightProjectionMatrix to work as in D3BFG. Finally, the main changes about light projection matrix: r9660 Preliminary refactoring of R_DeriveLightData. !!! r9661 Implemented D3style light projection/falloff behavior under cvar. 

I have added test FM: test_5815_spotlights It contains various (hopefully all) cases of weird light frustums, including even the case when light polytope is unbounded in Doom 3. Looking through releases, I managed to learn which version has which behavior. TDM 2.00, 2.04, 2.05, 2.06, 2.07: The lit voxels are exactly the same as on the current SVN with "spotlightBehavior 0". The only exception is the light with unbounded volume (yep, it is truly unbounded/infinite in 2.00). The floor is lit under "nonortho" + ceiling sometimes too (the bug described in https://bugs.thedarkmod.com/view.php?id=5529#c14557) TDM 2.08, 2.09: Exactly the same voxels are lit as on: SVN + "spotlightBehavior 1" with zScale = 1/far Floor and ceiling is lit both for "weirdstartend" and "onortho". The BFG code got into TDM by 2.05, due to commit by anonreclaimer. However, it was disabled by cvar until 2.08 (probably someone noticed the problem and disabled it): idCVar r_useAnonreclaimer( "r_useBfgPortalCulling", "0", CVAR_RENDERER  CVAR_BOOL  CVAR_ARCHIVE, "test anonreclaimer culling patch" ); Finally, it was enabled for 2.08: Revision: 8100 Author: duzenko Date: 29 march 2019 г. 23:57:44 Message: Forced anonreclaimer's culling codepath 

Hopefully, the final fix here: r9666. Changed r_spotlightBehavior cases, description, and default behavior. The behavior of TDM 2.07 and before is restored as default. Other behaviors are available only for debugging purposes under r_spotlightBehavior cvar. 

Date Modified  Username  Field  Change 

18.11.2021 17:45  stgatilov  New Issue  
18.11.2021 17:45  stgatilov  Status  new => assigned 
18.11.2021 17:45  stgatilov  Assigned To  => stgatilov 
18.11.2021 17:47  stgatilov  Relationship added  related to 0004888 
18.11.2021 17:49  stgatilov  Relationship added  related to 0003727 
19.11.2021 04:39  stgatilov  Additional Information Updated  
19.11.2021 04:40  stgatilov  Relationship added  related to 0005816 
21.11.2021 06:40  stgatilov  Relationship added  related to 0005529 
28.11.2021 11:38  stgatilov  Summary  Projected lights behavior is difference since 2.05 => Projected lights behavior has changed 
28.11.2021 11:42  stgatilov  Product Version  TDM 2.05 => TDM 2.08 
28.11.2021 11:42  stgatilov  Summary  Projected lights behavior has changed => Projected lights behavior has changed since 2.08 
28.11.2021 11:42  stgatilov  Description Updated  
28.11.2021 11:45  stgatilov  Note Added: 0014571  
28.11.2021 11:46  stgatilov  Note Added: 0014572  
28.11.2021 11:46  stgatilov  File Added: DeriveLightData_Spot_D3.pdf  
28.11.2021 11:46  stgatilov  Note Edited: 0014572  
28.11.2021 11:49  stgatilov  Note Added: 0014573  
28.11.2021 11:49  stgatilov  File Added: DeriveLightData_Spot_BFG_v2.pdf  
28.11.2021 11:51  stgatilov  Note Added: 0014574  
28.11.2021 12:12  stgatilov  Note Added: 0014575  
28.11.2021 12:23  stgatilov  Note Added: 0014577  
28.11.2021 12:26  stgatilov  Description Updated  
28.11.2021 12:26  stgatilov  Steps to Reproduce Updated  
01.12.2021 15:59  stgatilov  Note Added: 0014580  
01.12.2021 15:59  stgatilov  Status  assigned => feedback 
11.01.2022 05:02  stgatilov  Relationship added  related to 0005876 
30.01.2022 21:13  nbohr1more  Status  feedback => resolved 
30.01.2022 21:13  nbohr1more  Resolution  open => fixed 
30.01.2022 21:13  nbohr1more  Fixed in Version  => TDM 2.10 