View Issue Details

IDProjectCategoryView StatusLast Update
0006271The Dark ModGraphicspublic15.03.2024 10:50
Reporterstgatilov Assigned Tostgatilov  
PriorityurgentSeveritynormalReproducibilityN/A
Status resolvedResolutionfixed 
Product VersionTDM 2.11 
Target VersionTDM 2.12Fixed in VersionTDM 2.12 
Summary0006271: Merge old and new backend
DescriptionThe plan is to get a single code path again:
  1) it would use uniforms like the old backend
  2) it would be split into classes and methods like the new backend.
Additional Informationhttps://forums.thedarkmod.com/index.php?/topic/20949-on-bindless-textures-multidraws-and-improvements-to-the-new-render-backend/page/3/#comment-469307
https://forums.thedarkmod.com/index.php?/topic/21761-212-planning/&do=findComment&comment=482780
TagsNo tags attached.

Relationships

related to 0005285 closedcabalistic Remove old backend's depth/interaction/stencil stages 
related to 0006228 resolved Fog has banding in "Marsh of Rahena" 

Activities

stgatilov

stgatilov

17.03.2023 19:28

administrator   ~0015974

I started with some cleanup, removing dead experiments and totally obsolete code:
  r10296 Deleted manylight interaction code: both in old and new backend.
  r10297 Deleted INATTR_POSITION macro in shaders.

Then I converted the new backend to using uniforms, like in the old one.
The new backend source is so clean, and the DrawBatchExecutor's abstraction was so good, that it took me a few days to backport everything to single uniforms!
  r10298 DepthStage of new backend converted to single uniforms.
  r10299 Added DepthBoundsTest::Update method.
  r10300 Converted stencil shadow stage in new backend to single uniforms.
  r10301 Converted shadow map stage from new backend to single uniforms.
  r10302 Converted interaction stage of new backend to single uniforms.

Finally, the underlying tech of new backend is deleted:
  r10303 Deleted DrawBatchExecutor.
stgatilov

stgatilov

17.03.2023 19:31

administrator   ~0015975

What I plan next:
  1) Refactor shadow maps stage so that it can work in non-singlepass mode (which is default now).
  2) Try to delete as much duplicate code from the old backend as I can.
  3) Further refactor single-pass shadow maps so that they handle noselfshadow properly, then make them default.
  4) Try to remove some usages of backEnd global variable in the new backend: the individual stages should get all his data is parameters.
  5) See if I can convert more code form old backend into the new architecture.
stgatilov

stgatilov

18.03.2023 10:28

administrator   ~0015976

Last edited: 19.03.2023 10:45

Some more work with cleaning code of shadow map state:
  r10304 Deleted some commented code in shadow map stage.
  r10305 Don't forward declare nested enum.
  r10306 Support alpha-tested shadow mapping under r_shadowMapAlphaTested cvar.
  r10307 One more Linux build fix.
stgatilov

stgatilov

19.03.2023 10:47

administrator   ~0015977

I switched non-singlepass shadow maps to the new code too:
  r10308 Refactoring ShadowMapStage for future single-light support.
  r10309 Now ShadowMapStage code works even in "r_shadowMapSinglePass 0" mode.

And then supported local/global shadows division:
  r10310 Refactoring around InteractionStage + RenderBackend.
  r10311 Now single-pass shadow maps respect noselfshadow setting.

Now singlepass shadow maps should have no downside compared to non-singlepass ones.
stgatilov

stgatilov

25.03.2023 12:03

administrator   ~0015979

Deleting some obsolete experimental stuff:
  r10313 Deleted frob shader in old backend.
  r10318 Deleted FrobUniforms struct (not used already).
  r10312 Deleted RB_LogComment: Tracy is the way to go now.
  r10314 Deleted "new fog" experimental shader.

Also, refactored auto-translated shaders into simple readable GLSL:
  r10315 Cleaning up bumpyEnvironment shader.
  r10316 Cleaning environment shader.
  r10317 Cleaning cubeMap and oldStage shaders.
  r10319 Cleaning soft particle shader.
stgatilov

stgatilov

02.04.2023 09:57

administrator   ~0015988

Deleted the old backend!
  r10320 Deleted all the code from old backend which is replaced by new backend.
This is the main commit in this whole issue =)

Note that dev16785-10319 is the last release which can run the "old backend" code.
However, this only covers depth prepass, and interactions + shadows.
There are still "render passes" and "fog/blend lights" which exists exclusively in the old code.
stgatilov

stgatilov

02.04.2023 10:01

administrator   ~0015989

Last edited: 02.04.2023 10:04

Some cleanup:
  r10321 Some cleaning in rendering backend.
  r10323 Deleted R_FindGLSLProgram function.

Then "render passes" refactoring:
  r10325 Refactored "render passes" part of rendering into new backend. This is enabled by default, see cvar r_useNewRenderPasses.
  r10326 Environment mapping math moved back world space.

Note that "min" and "max" support is necessary in order to fix player damage overlay, which otherwise gets used with negative alpha:
  r10324 Added support for min and max functions in material expressions.
  r16787 Ensure alpha does not go below zero in textures/darkmod/decals/tunnel material.
The old code occasionally skipped rendering with negative alpha (depends heavily on other settings), which does not make sense.
This might be a breaking change, so we'll have to keep an eye on weird things during 2.12 beta.
The best approach would be fixing usages with negative alpha, but if this turns out to be too hard, we'll have to return the hacky skip conditions.
stgatilov

stgatilov

02.04.2023 10:04

administrator   ~0015990

More refactoring and fixes:
  r10327 Moved the function to read bumpmap into new tdm_compression.glsl.
  r10328 Restored forgotten frob outline under "r_useNewRenderPasses 1" mode.
  r10329 Support textures/particles/blacksmokepuff particles in the new code for soft particles.
  r10330 Added methods for checking material condition and getting material color.
  r10331 Moved R_ReloadGLSLPrograms_f to GLSLProgramManager, moved shadow map cvars to InteractionStage.
  r10332 Moved poisson disk sampling generation to InteractionStage.

Note that support for "textures/particles/blacksmokepuff" is also breaking change: such particles have been skipped for ages (since soft particles became default).

Also fixed a very annoying assert for wrong depth bounds in the new backend:
  r10333 Skip stencil shadow surfaces with empty scissor or depth bounds.
stgatilov

stgatilov

22.04.2023 15:35

administrator   ~0016001

I noticed that my new code copies _currentRender always, while the old code only copied it if there is at least one postprocessing surface.
So I fixed the performance regression and improved it: now new code does the copy only if there is at least one processing surfaces ACTUALLY USING IT.
  r10362 Fixed performance regression in r_useNewShaderPasses.
  r10363 Don't copy color texture if NeedCurrentRenderTexture returns false.

Then goes some refactoring:
  r10357 Deleted some code which was for experimental multilight rendering in old backend.
  r10359 Added methods GetTextureMatrix to drawSurf and viewLight.
  r10360 Renamed RenderPassesStage to SurfacePassesStage.
  r10361 Compile fix.
  r10365 Renamed "passes" to "surface passes" in shaders too.
The idea is that "surface passes" use surfaces but not lights, "light passes" uses lights but only barely uses surfaces, and "interaction" uses both lights and surfaces.

Then I converted blend and fog lights too:
  r10364 Added LightPassesStage for fog+blend lights in new backend, enabled by default.
  r10366 Disable blend lights with bad blending function.
  r10367 Copied fog math from fog image initialization code into the fog shader.
  r10369 Smoothen transition from "very fogged" to "totally fogged" in fog shader.
The new code for blend + fog lights works under "r_useNewRenderPasses 2", which is default now.

Speaking of blend lights: now they reuse bits of interaction shader code, so they process any kind of lights exactly as intended.
As for fog lights, I converted fog math from baked images to shader functions (which added a tiny bit of visual difference).

This also allows to understand the math of Doom 3 fog:
  1. it is exponential fog, with fog opacity jumping from 0.991 to 1.0 at the distance specified by color alpha value
  2. one plane of fog light frustum is intended to have soft transition, although this math is complicated, and does not work well anyway.

Finally, I reduced color banding on difficult fog scene (March of Rahena) by adding dithering:
    r10370 Added blue noise dithering to tonemapping shader on 64-bit color precision.
This only helps in 64-bit color mode, since otherwise we don't have necessary precision at the moment of tonemapping.

I guess I could apply dithering at the end of fog shader: then it would work on 32-bit precision too, but would only work for fog =)
I don't like the idea of adding dithering everywhere just to make obsolete 8-bit colors work...
stgatilov

stgatilov

04.06.2023 11:00

administrator   ~0016010

Last edited: 04.06.2023 20:18

More changes, fixes and optimizations:
  r10372 Optimized soft particles fragment shader for high-overdraw cases.
  r10376 Tonemapping moved to its own mini-stage.
  r10378 Reshuffled renderer files across directories.
  r10379 Removing usages of glsl.h.
  r10380 Fixed link error in tonemap shader.
  r10393. Fixed bug with volumetric light shadows in single-pass shadow maps mode.
stgatilov

stgatilov

15.03.2024 10:50

administrator   ~0016570

Now that 2.12 is released and 2.13 development is started, I deleted the old code:
  r10677 Deleted r_useNewRenderPasses cvar and code + shaders which were used on legacy path.
  r10678 Moved gaussianBlur and testImageCube shaders to the files where they are actually used.
  r10679 oldStage shader renamed to renderTools and heavily reduced.
  r10680 More obsolete code deleted.
  r10681 Finished cleaning up glsl.h / draw_glsl.cpp.
  r10682 Followup to previous commit.
  r10683 draw_glsl.cpp renamed to glsl.cpp in order to match glsl.h.

Issue History

Date Modified Username Field Change
12.03.2023 16:31 stgatilov New Issue
12.03.2023 16:31 stgatilov Status new => assigned
12.03.2023 16:31 stgatilov Assigned To => stgatilov
12.03.2023 16:31 stgatilov Relationship added related to 0005285
12.03.2023 16:31 stgatilov Additional Information Updated
17.03.2023 19:28 stgatilov Note Added: 0015974
17.03.2023 19:31 stgatilov Note Added: 0015975
18.03.2023 10:28 stgatilov Note Added: 0015976
19.03.2023 10:45 stgatilov Note Edited: 0015976
19.03.2023 10:47 stgatilov Note Added: 0015977
25.03.2023 12:03 stgatilov Note Added: 0015979
02.04.2023 09:57 stgatilov Note Added: 0015988
02.04.2023 10:01 stgatilov Note Added: 0015989
02.04.2023 10:01 stgatilov Note Edited: 0015989
02.04.2023 10:04 stgatilov Note Added: 0015990
02.04.2023 10:04 stgatilov Note Edited: 0015989
22.04.2023 15:35 stgatilov Note Added: 0016001
04.06.2023 11:00 stgatilov Note Added: 0016010
04.06.2023 20:18 stgatilov Note Edited: 0016010
05.12.2023 01:18 nbohr1more Status assigned => feedback
05.12.2023 01:54 nbohr1more Relationship added related to 0006228
05.12.2023 02:10 nbohr1more Status feedback => resolved
05.12.2023 02:10 nbohr1more Resolution open => fixed
05.12.2023 02:10 nbohr1more Fixed in Version => TDM 2.12
15.03.2024 10:50 stgatilov Note Added: 0016570