View Issue Details

IDProjectCategoryView StatusLast Update
0006296The Dark ModGraphicspublic18.06.2023 17:30
Reporterstgatilov Assigned Tostgatilov  
PrioritynormalSeverityminorReproducibilityhave not tried
Status resolvedResolutionfixed 
Target VersionTDM 2.12Fixed in VersionTDM 2.12 
Summary0006296: CreateLightDefInteractions optimization
DescriptionIn missions with large number of lights and entities, CreateLightDefInteractions function takes too much time and noticeably dump down FPS.
This function finds which light-entity interactions should exist for the current frame.
It is also responsible for the majority of TLB misses in our code (the other major culprit was idClip which is already fixed).
Steps To ReproduceMissions Iris and Scroll of Remembrance are good examples.
TagsNo tags attached.

Relationships

related to 0005459 closedstgatilov Optimize idRenderWorldLocal::CreateLightDefInteractions for ambient_world light 

Activities

stgatilov

stgatilov

28.05.2023 16:22

administrator   ~0016006

Last edited: 28.05.2023 16:22

Initially I thought about full reimplementation of entity/light storage for better cache locality.
The idea was to store everything (render entities, render lights, interactions, area-entity refs, area-light refs) on per-area basis on contiguous chunk of memory.
But this implied that we need to relocate entities, lights, and interactions around as they move in the world, which is a huge mess to code.
I started coding this, when I realized that I don't want to add this shit to the codebase =)

Instead, I tried to do specific optimizations with existing data structures.
Basically, I tried to change CreateLightDefInteractions so that touching idRenderEntityLocal is not needed in most cases.
A major trick here is to duplicate some member of idRenderEntityLocal in a separate array in render world.
For instance, we can store an array "bool entityIsInView[]" in the world, so that we can randomly access this array with low memory footprint.

As I did these changes, it soon became clear that random access over areaRefs is a bottleneck which stops any improvements.
Even if we don't access idRenderEntityLocal-s, we still have to access 48-byte areaReference_t structure for every entity.
These structs are placed in linked list in random order, so traversing them is slow.
To fix it, I still had to completely change the data structures of area-entity references (and same for lights), but at least I did not have to mess with entities/lights themselves.

After I improved speed of iteration over all entities in an area, all the other improvements were rather straightforward.
Most of them intend to avoid touching idRenderEntityLocal, some of them intend to reduce interaction table size.
As a result, I reduced this function from 7.5% down to 2% on Iris.
I think it is almost as fast as I can get it.

The list of commits (there are tons of comments in them too):
  r10384 Changed data structures for area-entity and area-light connections for faster iteration.
  r10385 Store bitset in render world, which duplicate information of whether entity is in current view or not.
  r10386 When considering a noshadows light, skip areas which are not in view.
  r10387 Moved rarely failed checks in CreateLightDefInteractions behind interaction table caching.
  r10388 Pass indices of light + entity instead of pointers to idInteractionTable::Find.
  r10389 Do fast light-entity culling test before checking interaction table.
  r10390 Don't call R_SetEntityDefViewEntity if entity is in view.
stgatilov

stgatilov

31.05.2023 16:44

administrator   ~0016008

r10391 Fixed lightref adding bug in FloodLightThroughArea_r (introduced in svn rev 10384).
stgatilov

stgatilov

18.06.2023 17:30

administrator   ~0016023

The optimization of non-shadowing lights (svn rev 10386) was applied only to lights where shadows are disabled in material.
But a lot of lights are marked with noshadows sparnarg, and the optimization did not exlpoit this opportunity.

This is fixed now:
  r10415 Apply non-shadowing lights optimization even if shadows are disabled by spawnarg on light entity.

See also:
  https://forums.thedarkmod.com/index.php?/topic/22143-latest-svn-visportal-shadow-issue/

Issue History

Date Modified Username Field Change
28.05.2023 09:47 stgatilov New Issue
28.05.2023 09:47 stgatilov Status new => assigned
28.05.2023 09:47 stgatilov Assigned To => stgatilov
28.05.2023 16:22 stgatilov Note Added: 0016006
28.05.2023 16:22 stgatilov Note Edited: 0016006
28.05.2023 21:37 stgatilov Status assigned => resolved
28.05.2023 21:37 stgatilov Resolution open => fixed
28.05.2023 21:37 stgatilov Fixed in Version => TDM 2.12
28.05.2023 23:51 stgatilov Relationship added related to 0005459
31.05.2023 16:44 stgatilov Note Added: 0016008
18.06.2023 17:30 stgatilov Note Added: 0016023