View Issue Details

IDProjectCategoryView StatusLast Update
0003878The Dark ModFeature proposalpublic01.02.2020 05:03
ReporterSteveL Assigned ToSteveL  
PrioritynormalSeveritynormalReproducibilityN/A
Status resolvedResolutionfixed 
Product VersionTDM 2.03 
Target VersionTDM 2.03Fixed in VersionTDM 2.03 
Summary0003878: Soft particles
DescriptionParticles are 2d images representing 3d translucent volumes. They show hard lines where they intersect solid objects, which breaks the illusion of a 3d volume. Implement soft particles that (1) have no hard intersection lines, and (2) hide their depth by fading out partly *in front of* the solid objects.
Additional InformationDetails + demo of first draft: http://forums.thedarkmod.com/topic/15178-tdm-engine-development-page/page__view__findpost__p__356102
Tagsparticle
Attached Files
particle_report.ods (23,967 bytes)
soft_particle_impl.gif (41,630 bytes)   
soft_particle_impl.gif (41,630 bytes)   

Relationships

related to 0003877 resolvedSteveL Allow shaders to access scene depth (co-author: revelator) 
child of 0003684 new Investigate GPL Renderer Improvements 

Activities

SteveL

SteveL

16.10.2014 23:02

reporter   ~0007072

Needs to use depth capture 0003877. Also needs a new material keyword "ignoreDepth" which turns off depth testing for a material stage, so allowing it to be drawn and blended over the top of solid surfaces in front of it. That'll be added with 0003877.
SteveL

SteveL

25.10.2014 23:08

reporter   ~0007091

There are 300-odd particles defined in TDM and FMs, but between them they use just 97 materials. It's the materials that will need the soft particle shader adding, not the particles themselves. Not all of those 97 materials are in particles that are actually used, and on top of that some of them will be used only for small particles like dust motes where softening won't be needed. Others might want to keep hard edges, like moonbeams.

I've parsed all .map, .def, and .prt files to make a list and compile usage stats (attached). TODO: decide which are in scope.
SteveL

SteveL

28.10.2014 20:14

reporter   ~0007095

Last edited: 28.10.2014 20:14

Attempt # 1 committed (to a test map only) in svn rev #14044.

Trying a one-size-fits-all solution for all particles that won't need any parameters setting and won't need to know the particle size.

The fadeout is determined by the ratio between particle depth and scene depth.

When the ratio is >1.2, i.e. the scene depth is more than 20% further away than the particle, the particle is rendered at full thickness (transparency not increased).
When the ratio is less than 0.8, i.e. the scene is more than 20% closer than the particle, the particle is completely hidden (full transparency).
Between those limits, the particle is faded proportionately (linear).

So the particle is more faded when there are objects immediately behind it. That makes sense visually as there's less "volume" for the smoke etc to occupy.
It also overdraws the geometry in front of it for up to 20% of the intervening depth, so the eye doesn't get a fix on where it is exactly. That also give the illusion of a rounded volume in front of the player instead of a flat particle cutting ito the floor or wall.

Diagram added to try to make this explanation clearer!

SteveL

SteveL

28.10.2014 20:18

reporter   ~0007096

Video of the result from AluminumHaste here: http://forums.thedarkmod.com/topic/15178-tdm-engine-development-page/page__view__findpost__p__356941
SteveL

SteveL

28.10.2014 23:36

reporter   ~0007098

Last edited: 28.10.2014 23:38

The shader is going to have to know the size of the particle after all :-/

The scheme above works well for fixing the intersections on all particles I've tried so far, but it causes particles to be seen through walls if the player can get far enough away (5 times the distance between a particle in a room and the outer surface of the room's wall).

Particles bleed through walls anyway of course if the wall is thinner than the particle dispersion and size, but the soft shader would cause particles that don't make it through the wall to be seen from a distance.

If the max overdraw were clamped to the particle size, this shouldn't happen. The only time that particles would be seen through walls would be in situations where they can already stick out through walls.

Question is how to let the shader know the particle size. The same material is often uses by various particles of different sizes. Is there an unused shaderparm?

SteveL

SteveL

29.10.2014 11:45

reporter   ~0007100

Last edited: 30.10.2014 07:41

There are unused shaderparms. As far as I can see in the code, particles use only parms 0..3 and 8. None of the standard particle materials use any other parm (or parm8 for that matter) and in any case there'd be no way to make use of those parms even if they did reference them.

Correction: parms 4,5 and 8 are used, just not in the material files. They're used in idRenderModelPrt::InstantiateDynamicModel to decide whether to make a new particle.

Passing the particle size in to the shader is probably inevitable, so that the shader knows the size of the volume that it's imitating. I've been trying to think of ways that we could make use of hardware z-culling to achieve the right effect, but we'd need to do it for whole tris not tiles or pixels, so the culling options are too small-scale.

The existing view frustum culling works for particles, but this time on too large a scale. It culls only whole emitters.

SteveL

SteveL

31.10.2014 20:50

reporter   ~0007105

Last edited: 01.11.2014 00:01

We won't use a shaderparm. That would apply to the whole func emitter whereas we want to use the radius of individual quads in the different stages.

To achieve this we need to store the quad size in the modelSurface_t generated by the front end, and pass it to the drawSurf_t that gets drawn by the back end. drawSurf_t already have a bitset "dsFlags" to allow special case logic in the back end. Only 1 bit of it is used so far, so we can use the next bit to say it's a soft particle being drawn.

Adding a size member to these structs won't inflate the data passed to the GPU. They aren't passed in their entirety, they just store the info needed by the backend to draw a surface correctly.

Particles already have a slight softening effect built in to the renderer: the "modelDepthhack" which moves the near clip plane to allow a small amount of overdraw. Without it their intersections are even harder/uglier. This needs disabling for soft particles as it moves the near clip plane and invalidates the depth calculations. We can use the same flag to do this.

SteveL

SteveL

31.10.2014 20:52

reporter   ~0007106

NB the diagram above *no longer applies*. The fadeout and overdraw depths will now be calculated from the particle radius, not the ratio of particle depth to screen depth. I'll replace it when I get back from hols.
tels

tels

10.11.2014 18:59

reporter   ~0007118

Just for reference:

http://wiki.thedarkmod.com/index.php?title=List_of_shaderParm_variables
SteveL

SteveL

18.11.2014 22:00

reporter   ~0007144

Last edited: 23.11.2014 15:28

Implemented plan:

Implemented in the engine so it'll work for all existing maps, and we won't have to amend or duplicate material files or use spawnargs etc. It can be toggled with cVar r_useSoftParticles.

The particle is modelled as a 3d volume by allowing it to draw in front of solid geometry, up to a (default) depth of 0.4 times its size. "Size" means the max diameter of the individual quads that make up a particle stage, not the whole particle. I'll call that size (0.4 times diameter) the particle "radius".

The engine passes the radius to the backend as a property of the drawSurf. NB One drawSurf = the collection of quads making up one stage of the particle.

The particle's visibility is adjusted by how far behind or in front of the background geometry it is. If the particle is more than its radius behind a surface, nothing is seen. If it's in front of the background by its full radius, it'll show its full brightness/colour. Between those limits, it's faded linerarly. The fade is calculated per pixel in the GPU, so large particle quads like dust clouds appear to fade out as they approach the floor or wall.

That scheme suits particles that represent smoke and vapour etc. A cloud of smoke is half as visible with a wall standing in the middle of it as it is above the wall, where you get to see the full depth of the cloud. There's an adjustment for additive blend surfaces: those are mostly used for lamp glares, and they shouldn't lose any brightness when a wall is immediately behind. For them, the fading range is half as deep, and they are at full brightness at 0 depth.

By default:
- the radius is 0.4 times the particle stage's max quad size
- only view-aligned particle stages will be softened. They are the ones that always represent 3d volumes.
- Only particles with radius > 1 will be softened.
- modelDepthHack (old method of causing a particle to overdraw geometry) is disabled for any view-aligned stage.

The mapper can override these defaults for any particle stage by specifying a new particle stage keyword, softeningRadius. That's a floating point number that accepts these values:
-1.0 : No automatic softening. Use old method of drawinng, including modelDepthhack if specified
 0.0 : Don't soften, and suppress modelDepthHack too. Useful for suppressing depthHack for a single stage even if softening isn't wanted, which will help people stop secondary stages poking through walls.
Any +ve number: Soften the particle stage using the specified radius.

Only view-aligned particles, i.e. those that turn to face the player, are softened by default. Those are meant to be "round" in shape and the engine can tell how big they are supposed to be as they look the same size from any direction.

Most non-view-aligned particles are meant to be fixed in place and flat, like water ripples or electrical discharges between electrodes, so these don't get softened by default.

One example where I plan to commit an override with the new feature: particle tdm_dustfog_02 is for ground mist. It has a very large radius, but is floor-aligned, so the engine can't know whether it's meant to be thick or not and won't soften it by default. But it represents a thin layer of fog and looks much better at the edges with a small softeningRadius of 5.

SteveL

SteveL

23.11.2014 15:36

reporter   ~0007166

Committed at rev 6171 (code), rev 14081 (game files)

glprogs/soft_particle.vfp
particles/tdm_fog.prt // for the softeningRadius on ground fog tdm_dustfog_02

framework/DeclParticle.cpp
framework/DeclParticle.h
renderer/Material.cpp
renderer/Model_local.h
renderer/Model_prt.cpp
renderer/RenderSystem_init.cpp
renderer/draw_arb2.cpp
renderer/draw_common.cpp
renderer/tr_light.cpp
renderer/tr_local.h
SteveL

SteveL

28.11.2014 20:16

reporter   ~0007175

Reopened for vertexcolor issue. See this post: http://forums.thedarkmod.com/topic/16710-particle-request-candle-smoke/page__view__findpost__p__359465

The soft particle fragment program applies vertexcolor whether it's been specified in the material or not. It's required for several important effects in the particle system, notably fade in/fade out, but we don't want particles to look different with softp turned on or off, so best check for it.
SteveL

SteveL

30.11.2014 01:10

reporter   ~0007185

There are only 6 materials referenced in particle defs that don't have vertex color set, and 4 of them are obviously unused test cases. But one of them is sparks and might be used and we don't want them to look duller. I'll commit the "fix" so that the soft particle program doesn't enforce vertex coloring.
SteveL

SteveL

30.11.2014 01:13

reporter   ~0007186

glprogs\soft_particle.vfp
Completed: At revision: 14094

renderer\draw_common.cpp
Completed: At revision: 6259
SteveL

SteveL

14.12.2014 23:17

reporter   ~0007259

Bugfix required:

This bug [drawn arrow closing VPs and cuasig decals to disappear) revealed 2 problems with the soft particle implementation:

1. Unlit decals added during gameplay (like these blood splats) were setting the soft particle flag unnecessarily because of an incorrect default value in a function.
2. Soft particles overrides one of the two z-buffer hacks in the game: modelDepthHack (=the DepthHack keyword used in particle decls), which is the engine's original way of slightly softening particles by making them draw over solids in front of them by up to DepthHack units. There's another depth hack in use: WeaponDepthHack, which makes the player's weapon models always draw in front of anything in the scene. The soft particle code prevents modelDepthHack fro softened particles, but it doesn't touch WeaponDepthHack. In error, it stopped both depth hacks being undone at the end of the drawcall, so if a bit of a weapon got drawn with soft particle code, the hacked z-buffer would remain in place until some other surface reset it.
SteveL

SteveL

14.12.2014 23:20

reporter   ~0007260

renderer\tr_local.h
renderer\draw_common.cpp
Completed: At revision: 6408
SteveL

SteveL

24.12.2014 23:46

reporter   ~0007274

Reverted these changes for vertexcolor:
//
glprogs\soft_particle.vfp
Completed: At revision: 14094

renderer\draw_common.cpp
Completed: At revision: 6259
//

The above changes applied a vertex color of 1.0 instead of applying the fade color determined by the engine if the particle material did not specify vertexcolor. This was incorrect -- the correct behaviour in the absence of vertexcoloring is to apply the color settings found in the shader stage's RGB settings, or the _color shaderparm.

Also need to block soft particles being applied for surfaces that have any light interacting stages. Those surfaces represent solids, so don't want softening at the edges.
SteveL

SteveL

25.12.2014 13:49

reporter   ~0007275

== Game code ==
0003878 soft particles fix. (1) Light-interacting surfaces should not be softened. (2) use rgb settings from material file in the absence of vertexColor.

Modified: darkmod_src\renderer\draw_common.cpp
Modified: darkmod_src\renderer\tr_light.cpp
Modified: darkmod_src\renderer\tr_local.h
Completed: At revision: 6432

== Shader prog ==
0003878 soft particles fix. Revert rv14094. The engine will now always provide a vertex color, using the same rules as for non-particle surfaces.

Modified: darkmod\glprogs\soft_particle.vfp
Completed: At revision: 14185

Issue History

Date Modified Username Field Change
16.10.2014 22:54 SteveL New Issue
16.10.2014 22:54 SteveL Status new => assigned
16.10.2014 22:54 SteveL Assigned To => SteveL
16.10.2014 23:02 SteveL Note Added: 0007072
16.10.2014 23:03 SteveL Relationship added related to 0003877
25.10.2014 23:08 SteveL Note Added: 0007091
25.10.2014 23:08 SteveL File Added: particle_report.ods
28.10.2014 20:14 SteveL Note Added: 0007095
28.10.2014 20:14 SteveL Note Edited: 0007095
28.10.2014 20:15 SteveL File Added: soft_particle_impl.gif
28.10.2014 20:18 SteveL Note Added: 0007096
28.10.2014 23:36 SteveL Note Added: 0007098
28.10.2014 23:38 SteveL Note Edited: 0007098
29.10.2014 11:45 SteveL Note Added: 0007100
30.10.2014 07:34 SteveL Note Edited: 0007100
30.10.2014 07:41 SteveL Note Edited: 0007100
31.10.2014 20:50 SteveL Note Added: 0007105
31.10.2014 20:52 SteveL Note Added: 0007106
01.11.2014 00:01 SteveL Note Edited: 0007105
10.11.2014 18:59 tels Note Added: 0007118
18.11.2014 22:00 SteveL Note Added: 0007144
19.11.2014 08:09 SteveL Note Edited: 0007144
23.11.2014 15:28 SteveL Note Edited: 0007144
23.11.2014 15:36 SteveL Note Added: 0007166
23.11.2014 15:36 SteveL Status assigned => resolved
23.11.2014 15:36 SteveL Fixed in Version => TDM 2.03
23.11.2014 15:36 SteveL Resolution open => fixed
28.11.2014 20:16 SteveL Note Added: 0007175
28.11.2014 20:16 SteveL Status resolved => feedback
28.11.2014 20:16 SteveL Resolution fixed => reopened
28.11.2014 20:16 SteveL Status feedback => assigned
30.11.2014 01:10 SteveL Note Added: 0007185
30.11.2014 01:13 SteveL Note Added: 0007186
30.11.2014 01:13 SteveL Status assigned => resolved
30.11.2014 01:13 SteveL Resolution reopened => fixed
14.12.2014 23:17 SteveL Note Added: 0007259
14.12.2014 23:20 SteveL Note Added: 0007260
21.12.2014 14:01 SteveL Relationship added child of 0003684
24.12.2014 23:46 SteveL Note Added: 0007274
24.12.2014 23:46 SteveL Status resolved => assigned
25.12.2014 13:49 SteveL Note Added: 0007275
25.12.2014 13:49 SteveL Status assigned => resolved
01.02.2020 05:03 stgatilov Tag Attached: particle