View Issue Details

IDProjectCategoryView StatusLast Update
0005668The Dark ModCodingpublic01.08.2021 16:50
ReporterMirceaKitsune Assigned Toduzenko  
PrioritynormalSeveritynormalReproducibilityalways
Status resolvedResolutionfixed 
Platformx64OSLinux openSUSEOS VersionRelease
Product VersionTDM 2.09 
Target VersionTDM 2.10Fixed in VersionTDM 2.10 
Summary0005668: hide_distance won't work for lights
DescriptionMappers frequently use the spawnargs dist_check_period / hide_distance on entities to improve performance by making them disappear after a certain distance, for instance "dist_check_period 0.5" with "hide_distance 1000" on a func_static using high-poly models. Unfortunately this doesn't appear to work for lights, including both builtin light entities (eg: lamps, candles) but even the base light class ignores it: A light will still show when the player is beyond the specified distance. I don't know if this used to work before or was simply never implemented as I noticed it during the last couple of days when I tried using it.

This is a major limitation which prevents us from properly optimizing the performance of a map. I have a few short range lights that cast shadows which could greatly benefit from this. DarkRadiant even documents those spawnargs on lights like other entities, indicating they should work too as far as mappers understand it. Could this be fixed before the 2.10 release please? Personally I don't even care about fancy features like fading or turning off just shadows with distance, I'd be happy just to have it work as it does on any other entity, by derendering the light when the view is far enough then showing it again once the player gets close.

On a relevant note I'd like to remind that with the current functionality of the lighting system, each light creates more triangles for every shadow caster in its area, something I discovered just recently myself. With lights increasing polygon count as they do, distance hiding is an essential ability to have for light sources too.
Steps To ReproducePlace any type of light on your map. Set the spawnargs dist_check_period and hide_distance to your desired value. Recompile the map and spawn on it. You should notice the light never turns off after you get way beyond the specified distance limit.
Additional InformationThe same is true for func_emitter: The spawnarg will not make particles disappear when the player is hide_distance units away from the entity.
Tagsperformance

Relationships

duplicate of 0000563 closedSteveL Add ability to turn off shadow-casting for distant lights (optimization) 
related to 0004303 closedSteveL hide_distance not hiding entities (without an LOD model set) 
related to 0005478 new Extend basic LOD system spawnargs to support other entities 

Activities

nbohr1more

nbohr1more

15.07.2021 23:11

developer   ~0014157

Particle hide_distance should work.

Thread about it breaking in the past:

https://forums.thedarkmod.com/index.php?/topic/17283-hide_distance-broken-again/
MirceaKitsune

MirceaKitsune

15.07.2021 23:27

reporter   ~0014158

Interesting: I tried it in the latest development snapshot on a func_emitter and the particles clearly didn't stop after that distance. It works on func_static though including patches with particle emitting textures (those stop showing accordingly).
MirceaKitsune

MirceaKitsune

23.07.2021 14:53

reporter   ~0014179

FYI: hide_distance also won't work on func_rotating entities. Just noticed as I was trying to hide a spinning gear with distance.
stgatilov

stgatilov

26.07.2021 04:29

administrator   ~0014186

As far as I understand, the current state is:
  r9492. Fixed hide_distance for light entities (street lights, etc)
  r9479. Fixed hide_distance for emitters
And it is not yet justified that more changes are necessary...

Note that:
1) Particle emitters is only about entities, particle deform systems cannot be hidden (although you can maybe hide the entity it belongs to).
2) Only geometry of light entity is hidden with r9492. The light source itself continues working.
MirceaKitsune

MirceaKitsune

26.07.2021 14:23

reporter   ~0014187

Yes, I understand a recent revision does get hide_distance working for light entities. However that only refers to their models, not the light source itself. What I was hoping for is that the light could be entirely disabled to avoid its pixel / shadow calculations. I would be grateful if that part could be done as well.
duzenko

duzenko

26.07.2021 15:57

developer   ~0014188

As per forum discussion, a repro case that demonstrates a performance issue with distant light is recommended here
MirceaKitsune

MirceaKitsune

26.07.2021 16:40

reporter   ~0014189

I can't test a feature that doesn't exist per say. What I can say, which is pretty close for now, is that in my FM (not yet public) I have areas where high-poly meshes combined with lights are causing notable FPS drops: They go away when you remove or lessen the lights. I might test that again just to reconfirm and be extra safe, will be going soon but perhaps tonight I'll take some time for it.
MirceaKitsune

MirceaKitsune

26.07.2021 20:41

reporter   ~0014190

Okay... what I can provide in terms of a performance comparison for now. This is a small closed area so it may not be the ideal scenario, but distant lights do the same thing and have the same effect just at a different scale.

I have that basement / jail cell where I get a pretty big performance drop due to torches hitting some high-poly models. I took a screenshot of the area in its initial state... I had 6 water arrows placed on the map so I used them to turn off 6 of the torches then took another screenshot. First image shows roughly 800000 tris, the second shows it dropping to nearly 500000 after the torched go out. Performance noticeably improves a little as this happens though not radically much.
empire_2021-07-26_23.33.51.jpg (256,364 bytes)   
empire_2021-07-26_23.33.51.jpg (256,364 bytes)   
stgatilov

stgatilov

27.07.2021 04:22

administrator   ~0014191

And do you have moments when this basement is seen from a distance far enough that disabling these lights won't freak out player?
Even if this is useful for someone, I'm not sure it should work under "hide_distance" parameter.

P.S. To be honest, I personally dislike the LOD thing.
I always notice the popping. To me it's the last-resort feature if proper FPS cannot be maintained even on good hardware.
MirceaKitsune

MirceaKitsune

27.07.2021 13:58

reporter   ~0014196

The popping is why I like to enable it at a good distance. There's also a lod_fadein and lod_fadeout which can do it smoothly: Not pretentious enough to hope that's going to work for lights too though that would be nice.
MirceaKitsune

MirceaKitsune

27.07.2021 22:03

reporter   ~0014201

I spent the day creating a miniature city from scratch just to offer the required test case for this, I hope it was worth it and will help in some form. It's probably not perfect, no test case is for something as fiddly as map design is, but I tried getting as close as I could; It's meant to replicate a city area done in my typical style, containing what you can expect to be the top level of detail an author would use (myself included): Not insanely overboard but pretty detailed in what you can see at once.

It differs from Peter's test with the trees in that it uses the default prefabs (some of which already have a hide_distance) containing many smaller models and skins, which should better replicate what you get when you detail a FM in practice. If I was to add an objective this could technically be turned into a super tiny FM you can play... if anyone feels like expanding it you have my permission, I don't plan on it unless it's to improve the testing process. The map is free to do what you want with and its darkradiant file is included, prefabs are grouped so you can easily select and manipulate them.

The player starts at a position from which they see the vast majority of lights and objects. There's a pulpit with two levers in front: The left one disables electrical lights the right one torches: It's the easiest way I could think of to toggle them and see the difference with and without lights. Most of the far away lights should be at a distance of over 1000 to the player, which should replicate what would happen if you'd use a reasonable "hide_distance 1000" on a light.

The numbers with r_showprimitives indicate exactly what I've been seeing in my normal FM: Once you have enough meshes on your map and enough lights around them (even separated enough so they don't overlap much) it gets quite a lot for the engine to manage. The tris with this dedicated test case as you toggle the levers are:

- All lights enabled: 1.300.000
- Only torches or electric electric: 1.000.000
- All lights disabled (except for candles): 670.000

So turning off all important light sources on the map literally halves the total triangle count. Obviously we wouldn't be downright halving it in a practical scenario, but this should show there's a significant difference: If even close to a quarter of the total triangles processed get removed thanks to light hiding, it's worth considering in my book.
disttest.pk4 (387,913 bytes)
MirceaKitsune

MirceaKitsune

28.07.2021 12:42

reporter   ~0014206

Just updated my test case to ensure it's as representative as possible to what we're looking at. I namely removed all lights near the starting position of the player, leaving only those that are at least 512 units away which would be the minimal sane distance to start hiding some lights like torches and candles. I also repositioned them to avoid even that bit of overlap between electrical lights where the edges were touching slightly. Overall I left in only 3 street lamps and 2 torches in between them, disabling even the prefab candles for consistency: This should be as accurate as it gets to a realistic scenario.

How the numbers changed: With all 5 lights enabled we have 810.000 tris, with them turned off it's 480.000. Once more it's almost halved, more than a third of the tris calculated are gone. And with this updated version it's representative of the exact effect the player would get with such distance hiding, nothing generic or estimative: All lights are +512 units away, with "hide_distance 500" it would be the exact same thing as toggling the lever from behind the pulpit.

Extra info: I'm on the latest development version available in the installer (9507). I'm using shadow maps not stencil shadows, they're the fastest option as of the newest dev snapshots. OS is Linux Manjaro on KDE, amdgpu module.
disttest_onlydistant.pk4 (387,454 bytes)
MirceaKitsune

MirceaKitsune

28.07.2021 13:17

reporter   ~0014207

A little extra test, since I know what some are going to say next and maybe rightfully so: "But Mircea, you still have those two torches overlapping the electrical lights, many mappers won't do that". Also the street lamps are long range and we wouldn't be crazy to turn those off at a distance of 512 (I'd probably have them at 1024 - 2048), only torches would be disabled at 512.

So let's try the updated map without the torches. Not gonna post another version just for that, simply delete the two torches and load the map. This way we have only 3 electrical lights that don't touch.

For the first debate: Disabling the electric lights alone and having no more lights (except ambient and fog) still takes you from 639655 tris and 3552 draws to 476101 tris and 2474 draws, removing roughly a third for both.

On the second issue: Between street lamps + torches and only street lamps, we seem to get 811.636 tris and 4525 draws versus 639.655 tris and 3552 draws. So even if we were to hide just the torches at +512, we get about a quarter of the tris and draws trimmed down. I definitely think this says something honestly.
MirceaKitsune

MirceaKitsune

28.07.2021 16:15

reporter   ~0014208

I think I found a way to settle this. Managed to put together a script / addon which emulates this functionality. The advantage is you can use it in any FM: Drop the pk4 directly inside your TDM folder and it will automatically work. Two versions are provided: One turns the light off entirely, the other only disables its shadows... don't use both at once but pick one.

Just remember this is a testing script and will break your run! For one it doesn't know when a light is meant to be turned off, thus will enable even lights that started or were toggled off once they're in range... meanwhile the nowshadows version enables self lighting on torch models which looks wrong. They also seem to cause infinite loop crashes on occasion, though it should take long enough for them to occur in order to obtain a result.
light_dist.pk4 (860 bytes)
duzenko

duzenko

29.07.2021 18:40

developer   ~0014211

Last edited: 29.07.2021 18:42

Sorry, but this particular test is not good enough for me
Normal fps = 51. Lights off - 64 fps. There's just nothing to optimize here.
I'd need the lights at least halve the fps number to make this worth it because that's just the base line as we can't simply hide all lights. Much of it will still make it through the filter.
stgatilov

stgatilov

30.07.2021 03:27

administrator   ~0014212

I think you should look at exact times of game tics, frontend, and backend (on CPU and GPU) to make conclusions.
Did you already use Tracy? =)
duzenko

duzenko

30.07.2021 07:07

developer   ~0014214

Last edited: 30.07.2021 08:00

I only just realized there was a lever controlling the lights and that I forgot to disable shadows in my previous test
I can now see a 2x fps difference with lights on and off, even more so with sky and fog off
I suppose it merits a further investigation now
MirceaKitsune

MirceaKitsune

30.07.2021 16:36

reporter   ~0014221

Just found something really funny: hide_distance already works for torch lights in the latest dev snapshot, including turning off the light source as desired! It only doesn't work on electric and standalone light entities. So we actually have this half implemented now, if anything it's an inconsistency... it just needs to be fixed for the remaining light types and that's it!
duzenko

duzenko

01.08.2021 14:01

developer   ~0014230

 Completed: At revision: 9528

Issue History

Date Modified Username Field Change
13.07.2021 10:40 MirceaKitsune New Issue
13.07.2021 10:40 MirceaKitsune Tag Attached: performance
15.07.2021 23:11 nbohr1more Note Added: 0014157
15.07.2021 23:13 nbohr1more Relationship added related to 0004303
15.07.2021 23:18 nbohr1more Relationship added related to 0005478
15.07.2021 23:19 nbohr1more Relationship added duplicate of 0000563
15.07.2021 23:27 MirceaKitsune Note Added: 0014158
23.07.2021 14:53 MirceaKitsune Note Added: 0014179
26.07.2021 04:25 stgatilov Assigned To => duzenko
26.07.2021 04:25 stgatilov Status new => assigned
26.07.2021 04:26 stgatilov Target Version => TDM 2.10
26.07.2021 04:29 stgatilov Note Added: 0014186
26.07.2021 14:23 MirceaKitsune Note Added: 0014187
26.07.2021 15:57 duzenko Note Added: 0014188
26.07.2021 16:40 MirceaKitsune Note Added: 0014189
26.07.2021 20:41 MirceaKitsune Note Added: 0014190
26.07.2021 20:41 MirceaKitsune File Added: empire_2021-07-26_23.32.28.jpg
26.07.2021 20:41 MirceaKitsune File Added: empire_2021-07-26_23.33.51.jpg
27.07.2021 04:22 stgatilov Note Added: 0014191
27.07.2021 13:58 MirceaKitsune Note Added: 0014196
27.07.2021 22:03 MirceaKitsune Note Added: 0014201
27.07.2021 22:03 MirceaKitsune File Added: disttest.pk4
28.07.2021 12:42 MirceaKitsune Note Added: 0014206
28.07.2021 12:42 MirceaKitsune File Added: disttest_onlydistant.pk4
28.07.2021 12:42 MirceaKitsune File Added: disttest_2021-07-28_15.20.14.jpg
28.07.2021 12:42 MirceaKitsune File Added: disttest_2021-07-28_15.20.36.jpg
28.07.2021 13:17 MirceaKitsune Note Added: 0014207
28.07.2021 13:17 MirceaKitsune File Added: disttest_2021-07-28_15.49.40.jpg
28.07.2021 13:17 MirceaKitsune File Added: disttest_2021-07-28_15.49.52.jpg
28.07.2021 16:15 MirceaKitsune Note Added: 0014208
28.07.2021 16:15 MirceaKitsune File Added: light_dist.pk4
28.07.2021 16:15 MirceaKitsune File Added: light_dist_noshadows.pk4
29.07.2021 18:40 duzenko Note Added: 0014211
29.07.2021 18:41 duzenko Note Edited: 0014211
29.07.2021 18:42 duzenko Note Edited: 0014211
30.07.2021 03:27 stgatilov Note Added: 0014212
30.07.2021 07:07 duzenko Note Added: 0014214
30.07.2021 08:00 duzenko Note Edited: 0014214
30.07.2021 16:36 MirceaKitsune Note Added: 0014221
01.08.2021 14:01 duzenko Status assigned => closed
01.08.2021 14:01 duzenko Resolution open => fixed
01.08.2021 14:01 duzenko Note Added: 0014230
01.08.2021 16:50 stgatilov Status closed => resolved
01.08.2021 16:50 stgatilov Fixed in Version => TDM 2.10