View Issue Details

IDProjectCategoryView StatusLast Update
0004957The Dark ModFeature proposalpublic06.01.2021 05:12
Reporterstgatilov Assigned Tostgatilov  
PrioritynormalSeverityfeatureReproducibilityN/A
Status resolvedResolutionfixed 
Product VersionTDM 2.07 
Target VersionTDM 2.08Fixed in VersionTDM 2.08 
Summary0004957: Snow and rain particles stopping on brushes
DescriptionOriginal post by Goldwell:
  http://forums.thedarkmod.com/topic/19793-208-major-changes/#entry432359

Currently rain and snow particles fail through everything on their way. It is possible to make them stop/hit at desired place, but it is tedious.

This issue is to investigate the possibility of better rain/snow system which would automatically stop its particles on hitting brushes (and maybe even models).
Additional InformationMore recent discussion on dev forums:
  https://forums.thedarkmod.com/index.php?/topic/20235-proposal-for-particle-collision/
Basically, there is a lengthy design document about intended way of dealing with the problem.
Tagsparticle

Relationships

related to 0005136 resolvedstgatilov Wrong bounds of particle system 
related to 0005137 resolvedstgatilov Dmap optimization breaks rain patches 
related to 0005138 resolvedstgatilov Refactor particle systems code 
related to 0005139 resolvedstgatilov Renew particle declarations for rain 
related to 0005437 resolvedstgatilov Particles: static collision with "linear" mapLayout 
related to 0005486 resolvedstgatilov dmap: close vertices due to numeric errors, and bloomed sparklies 

Activities

stgatilov

stgatilov

12.01.2019 09:42

administrator   ~0011325

I have no idea yet how particle system works in the code.
But I'm pretty sure that all particles are modeled completely on CPU. Then it should be straightforward to add collision detection under some new spawnarg.

One issue to keep in mind is performance.
Collision detection is not free, and doing it for every particle every frame may be costly (and may be not --- to be found). That's why there are ideas like 1) using only brushes for collision, and 2) computing hit location once or once per 10 frames.
STiFU

STiFU

12.01.2019 14:52

developer   ~0011327

Theoretically, it should be possible to calculate particle lifetimes offline such that no collisions with brushes occur. I imagine something like a lookup table that maps particle spawn position to particle lifetime, which could be calculated during dmap.
stgatilov

stgatilov

12.01.2019 15:25

administrator   ~0011328

Yes, this is third optimization (even more serious one): precompute particle path depending on point in emitter area.
One graphical way to do that is to render all solid objects from above into one bitmap with depth info, then depth at each pixels says how long the particle will go.

Note that the generic solution with always checking collision produces less artefacts. It always works good, even in presence of moving objects. Every optimization applied adds artefacts in some case: rain goes through crates if only brushes are taken into account, or rain going through AI when they are moving, etc.
That's why I suggest starting with slow but correct solution, profile, and then apply optimization in order of increasing severity.
STiFU

STiFU

12.01.2019 16:26

developer   ~0011331

Right, good idea. There is also the option to do an offline-online hybrid, use offline data only for distant particles and accurate online calculations for close ones.
VanishedOne

VanishedOne

12.01.2019 20:52

reporter   ~0011339

From what I remember of SteveL's posts on particles, 'smoke' particles in the 'world particle system' have their positions stored so that e.g. smoke trails don't follow their emitters around, but particles spawned by func_emitters don't have individual state (though the emitter does get a bounds check on map load).

http://forums.thedarkmod.com/topic/17034-trailing-particles/
http://forums.thedarkmod.com/topic/16710-particle-request-candle-smoke/page-5#entry360281

I'm not sure where particles emitted from patches come into this (http://wiki.thedarkmod.com/index.php?title=World_Particle_System indicates they're another subsystem again), though visportalling has a known effect: http://forums.thedarkmod.com/topic/11314-keeping-your-house-dry/
stgatilov

stgatilov

01.02.2020 04:57

administrator   ~0012161

Here is the chronology of events.

Duzenko did the following commits while working on the feature:
  8420, 8423, 8433, 8436
They include:
  test code to call idRenderWorld::Trace for every particle every frame --- purely experimental
  command to generate heightmap by material name (why was it implemented as a cvar?) --- now runParticle tool does the job
  new deform DFRM_RAIN and copy/pasted code --- not a good idea to leave such stuff lying around
I have reverted all of these changes in rev 8549.

Then there was a proposal in dev forums:
  https://forums.thedarkmod.com/index.php?/topic/20235-proposal-for-particle-collision/
I made a bunch of preparational commits. I guess it includes all my commits from rev 8534 to 8548.
Here are the most important ones:
  rev 8537:
    Now idImage can store image data on CPU side. This depends on "residency", which can be "graphics", "cpu" or "both".
    Also discussed on dev forums: https://forums.thedarkmod.com/index.php?/topic/20237-idimage-store-cpu-side-data/
    Rev 8546 is worth mentioning: don't reload missing texture if it is queried with nondefault filtering/clamping settings.
  rev 8540:
    Implemented idRenderWorld::TraceAll as a faster and more flexible substitute for preexisting idRenderWorld::Trace.
    I could not live without flexibility, and performance boost is also very welcome given the number of traces done.
    In rev 8541, I removed idRenderWorld::Trace code and made it call the new idRenderWorld::TraceAll instead.
    Commit 8543 adds SIMD optimization to ray-surface intersection code (it was present in Doom 3, but we did not reimplement that for x64 until now)

All the main changes come in commit 8547.
They implement enough stuff to make working rain.
stgatilov

stgatilov

01.02.2020 07:05

administrator   ~0012163

Here is what is currently implemented:

"mapLayout texture {W} {H}" --- specify texture width/height for cutoff map.
This is required for all the other features. Also it allows to specify resolution for collisionStatic precomputation.
For example: if you set take a 3000 x 3000 patch which emits particles with 1000 x 1000 mapLayout resolution, then there will be one sample precomputed per every 3 units.
"mapLayour linear" is now parsed, but not yet supported anywhere in the code.

"cutoffTimeMap {path/to/tga/image}" --- manually specify texture with particle lifetimes.
Every texel stores a ratio in range from 0.0 to 1.0. Ratio 0.0 means that particle dies immediately, 1.0 means that it lives for its whole time, 0.5 means that it dies after half of its lifetime.
Note that normally mapper should use collisionStatic keyword, and NOT use this one.
But it might be helpful for some effects if you draw this image manually.
The ratio is encoded into color as ((B/256 + G)/256 + R)/256. If you draw your own map, just use grayscale image.

"collisionStatic 1" --- enables static collisions on this type of particles.
In order to make it work, you have to run "runParticle {path/to/map/file}" after dmapping the map.
Then cutoff textures will be created in directory textures/_prt_gen/, which are loaded by game code and used in a way similar to cutoffTimeMap.

"collisionStaticWorldOnly 1" --- only collide with world geometry when precomputing collisionStatic images.
This makes precomputation much faster, since it removes all models from collision detection.
This should not be used normally. Can only be useful if normal precomputation takes too much time on your map.
stgatilov

stgatilov

01.02.2020 07:29

administrator   ~0012164

Last edited: 01.02.2020 07:31

The new compiler tool "runParticle" precomputes the textures for collisionStatic particle systems.

It parses .map file, then creates temporary idRenderWorld from it. The render world reads .proc file and takes "local models" from there, i.e. all surfaces produced from brushes and patches. Then we iterate over all map entities, and add some of them into our render world as blockers. The exact criterion here is quite complicated since we want to exclude everything which may move or disappear. There is a spawnarg to override this decision. Note that we have to load .def-files here in order to obtain all spawnargs including inherited ones, but we do NOT precache any media when doing so.

Now we iterate over all map entities (including world) again to find particle emitters. Only particle-emitting surfaces are supported now. Also, only brushes and patches are supported as collisionStatic emitters. For every such entity, we load its static model (in fact, it was already loaded by render world from .proc file when it loaded map), and inspect its surfaces. If you find particle emitter with collisionStatic, then we generate a cutoff map. Also, there is a way to exclude particle emitter from collisionStatic computation via spawnarg, but obviously it won't work on world geometry.

When computing cutoff texture, we inspect particle stage settings and fail if we see something unsupported. Note that "texture" layout has severe restrictions on which particles it supports. Then iterate through texels of the future map, for each texel find a triangle which uses it and cast a ray through each texel. The hit is recorded into image buffer, which is finally saved into TGA image.
A few thing to note here:
1) Emitting surface should have texture coordinates [0..1] x [0..1]. Coordinates out of this range are not supported, and using smaller range would be a waste of storage.
2) Each texel must belong to exactly one triangle (well, except for boundary effects). If half of quadratic patch uses [0..1] x [0..1] texture area, and the other half uses the same area again, then the tool will break.
3) Since dmap splits a large patch into many surfaces (one surface per area), the tool actually computes and saves only a rectangular part of the whole map. It takes bounding box of vertices' texcoords, and saves the minimum subregion of the texture which covers it.
As for ray tracing, there are some builtin rules which cannot be overriden: ignore particle emitters, ignore dynamic models (e.g. md5), hit only solid & water contents.

When the game is running and a particle-emitting surface with collisionStatic flag is processed by the renderer, some special things happen.
First of all, it loads the texture as CPU-resident idImage. The path is hardcoded and contains the following info: entity name, surface index, and particle stage index. Note that there is nowhere to store this texture, so it is reloaded every frame. I.e. "globalImages->ImageFromFile" is called every frame, but internally it finds the loaded image and returns it without doing anything. Of course, the tweak with bounding box of vertices' texcoords is replicated here exactly as in the precomputing tool.
Then this image is sampled on CPU side with nearest filtering and clamping. This is the obvious part. If there is cutoffTimeMap in particle declaration, then it is used instead.

Overriding spawnargs are:
  particle_collision_static_blocker {0 or 1} --- force-exclude or force-include this entity as blocker in particle collision precomputation
  particle_collision_static_emitter 0 --- exclude all particle-emitting surfaces of this entity from particle collision precomputation

stgatilov

stgatilov

02.02.2020 16:05

administrator   ~0012171

A bunch of minor post-fixes:

r8548 Fixed crash if cutoff texture is not found.
r8550 Check mapLayout when cutoffTimeMap is set explicitly. If it is not "texture" with matching dimensions, then the map is dropped.
r8551 runParticle now complains about "mapLayout linear" and about "worldAxis".
r8554 Ignore dynamic models properly in runParticle tool.
stgatilov

stgatilov

03.02.2020 17:23

administrator   ~0012176

Last edited: 03.02.2020 17:44

More commits: 8561 and 8562.
Now collisionStatic images are preloaded when idRenderWorld reads .proc file (i.e. at map load time).
Also it fixes compatiblity with com_smp, although explicitly set cutoffTimeMap still does not work (see 0005141)

UPDATE: And commits 8563 and 8564 fixing the mess I recently made with latest commits =(

stgatilov

stgatilov

01.06.2020 17:30

administrator   ~0012583

A minor update due to Goldwell's feedback here:
  https://forums.thedarkmod.com/index.php?/topic/20291-beta-testing-208/&do=findComment&comment=447857

Hence, two changes:
svn rev 8746: now mapper can set "particle_collision_static_blocker 2" to force all surfaces of the entity to be blockers, regardless of their material or dynamic model
svn rev 8747: surfaces with "deform turbulent" material are no longer ignored as blockers --- this deform only changes texcoords.

Issue History

Date Modified Username Field Change
12.01.2019 09:39 stgatilov New Issue
12.01.2019 09:42 stgatilov Note Added: 0011325
12.01.2019 14:52 STiFU Note Added: 0011327
12.01.2019 15:25 stgatilov Note Added: 0011328
12.01.2019 16:26 STiFU Note Added: 0011331
12.01.2019 20:52 VanishedOne Note Added: 0011339
25.01.2020 13:38 stgatilov Assigned To => stgatilov
25.01.2020 13:38 stgatilov Status new => assigned
25.01.2020 13:39 stgatilov Additional Information Updated
01.02.2020 04:57 stgatilov Note Added: 0012161
01.02.2020 04:59 stgatilov Tag Attached: particle
01.02.2020 05:11 stgatilov Relationship added related to 0005136
01.02.2020 05:36 stgatilov Relationship added related to 0005137
01.02.2020 05:36 stgatilov Relationship added related to 0005138
01.02.2020 05:43 stgatilov Relationship added related to 0005139
01.02.2020 07:05 stgatilov Note Added: 0012163
01.02.2020 07:29 stgatilov Note Added: 0012164
01.02.2020 07:31 stgatilov Note Edited: 0012164
02.02.2020 16:05 stgatilov Note Added: 0012171
03.02.2020 17:23 stgatilov Note Added: 0012176
03.02.2020 17:44 stgatilov Note Edited: 0012176
03.02.2020 17:44 stgatilov Status assigned => resolved
03.02.2020 17:44 stgatilov Resolution open => fixed
03.02.2020 17:44 stgatilov Fixed in Version => TDM 2.08
01.06.2020 17:30 stgatilov Note Added: 0012583
30.11.2020 10:07 stgatilov Relationship added related to 0005437
06.01.2021 05:12 stgatilov Relationship added related to 0005486