View Issue Details

IDProjectCategoryView StatusLast Update
0003135The Dark ModCodingpublic15.03.2021 15:45
Reportertels Assigned To 
PrioritynormalSeverityfeatureReproducibilityN/A
Status newResolutionopen 
Target VersionTDM 2.11 
Summary0003135: Add per-pixel material definitions ("material maps")
DescriptionCurrently every surface can only have one material associated with it - be it stone, wood, glass etc.

This means combined surfaces (think woodbeams + plaster) can only be wood, or plaster, but not both.

This has drawbacks on reactions to the material. If f.i. the player shoots an arrow at a wooden beam, the arrow would break if the surface is tagged "stone". OTOH, if the surface is tagged "wood", the arrow would also stick in plaster.
Additional InformationAn 1,2, 4 etc bit image, with color-coded pixels (1, 2, etc) would need to be specified in the material shader. The shader would also need to supply a list of materials that are associated with each number (1=stone, 2=wood etc).
TagsNo tags attached.

Activities

nbohr1more

nbohr1more

22.05.2012 02:44

developer   ~0004605

Per vertex materials with interpolation would probably be more feasible? If brushes could have vertex blending you could rewrite the material parser to pair surfaces with vertex colors and have audio cross-fade work like vertex blending.

Mind, this is still a lot of CPU side management and memory overhead but perhaps not so much more than tracking all that clip texture geometry with surface attributes.

On further consideration for per-pixel mapping, if this were not applied to all areas but only special use cases, it might be pretty nifty too.
tels

tels

23.05.2012 16:30

reporter   ~0004617

The way it works in Quake 4 (and I understand it) is that if there is no material map, the default mateterial is used (straw, stone, wood) but if there is a material map, it is consulted at the pixel that was hit.

Just like specular maps etc the material map does not need to be of the same resolution than the underlying texture.

This system is quite simple to implement.

Whatever you wrote with vertex interpolation etc. does not make any sense to me. A triangle has 3 vertexes, but it can have an arbitrarily complex texture painted on - think "leather armor with metal belt buckle and bits of skin exposed". That would be 3 materials alone that could appear on a single triangle.

Currently, all these different surfaces need to be sep. into different meshes just so the game can decide wether something collided with metal, leather or skin.
nbohr1more

nbohr1more

23.05.2012 22:51

developer   ~0004618

Yes, there would be cases where a single texture represents many materials but "if vertex blending ever becomes more wide-spread" then interpolating a value between vertices would be a good second tier solution too. The texture artist could "paint" a wood texture onto an otherwise stone surface with stone texture and blend the two therefore the same interpolation needed to resolve the gradient in one case (vertex blending) might possibly work to communicate values in the other case (surface blending). Yeah, until such time that vertex blending is used in any other mission than NHAT 3/3 then this is a pretty moot point.

user81

28.03.2018 10:30

  ~0010233

@nbohr1more, can this be closed?
nbohr1more

nbohr1more

28.03.2018 18:45

developer   ~0010319

This is still an open ticket but it seems to be an impractical solution.

You would need to transform the image map onto the collision data which
is akin to texturing on a GPU.

CPU's are very slow at such things.

If someone were to move our physics engine to the GPU, or if they made a compute shader specifically for this purpose then this might be a more feasible concept.

Suspended.
duzenko

duzenko

11.04.2019 19:39

developer   ~0011723

Last edited: 11.04.2019 19:40

View 2 revisions

The 'collision point' position never hits the GPU. I think we have everything we need to calculate this on CPU. Right, CPU texturing is slow but we only need to fetch one sample per collision and that can't be a problem.

Can we have a test map?

VanishedOne

VanishedOne

12.04.2019 19:31

reporter   ~0011726

Last edited: 13.04.2019 08:15

View 2 revisions

I think we'll need a test material to go in the test map. Maybe we could lift one from Quake 4 for testing purposes?

Q4 uses the keyword materialImage for this: https://modwiki.dhewm3.org/Texturing#q4-hit-maps

We'd need to assign colours to material types. This is the Q4 list (it loads them from an external file), which has partial overlap with TDM's:

duzenko

duzenko

13.04.2019 08:14

developer  

ppm.txt (1,714 bytes)   
materialType none
{
    description "Akin to a void, no decals are left on this surface and no effects are spawned"
    rgb 0,0,0
}

materialType solidmetal
{
    description "Heavy duty metal - such as girders"
    rgb 102,102,102
}

materialType flesh
{
    description "Exposed skin"
    rgb 204,102,102
}

materialType hollowmetal
{
    description "Thin metal plate - such as air ducts or computer covering"
    rgb 70,70,70
}

materialType dirt
{
    description "Dirt"
    rgb 140,100,60
}

materialType sand
{
    description "Fine ground rock normally found in deserts"
    rgb 255,255,160
}

materialType gravel
{
    description "Loosely ground rough pebbles"
    rgb 200,170,95
}

materialType glass
{
    description "Translucent"
    rgb 255,255,255
}

materialType concrete
{
    description "Solid rock like material used in construction"
    rgb 170,170,170
}

materialType water
{
    description ""
    rgb 140,170,220
}

materialType gelatinousfluid
{
    description ""
    rgb 120,40,140
}

materialType oil
{
    description ""
    rgb 240,30,40
}

materialType bpglass
{
    description "Bulletproof glass"
    rgb 150,40,80
}

materialType rock
{
    description "Naturally exposed rock"
    rgb 90,70,64
}

materialType rubber
{
    description ""
    rgb 40,40,40
}

materialType plastic
{
    description ""
    rgb 130,200,120
}

materialType armor
{
    description "kevlar"
    rgb 100,90,170
}

materialType electronics
{
    description ""
    rgb 250,170,90
}

materialType monstermetal
{
    description " Monster metal... metal for monsters... "
    rgb 60,52,157
} 
ppm.txt (1,714 bytes)   
duzenko

duzenko

13.04.2019 08:15

developer   ~0011727

Extracted comment content to the attached file
duzenko

duzenko

13.04.2019 08:17

developer   ~0011728

Last edited: 13.04.2019 08:17

View 2 revisions

What I would like here is a e.g. wall with a per-pixel material. Then in a test map player shoots an arrow at that wall and it should either stick or break

VanishedOne

VanishedOne

13.04.2019 15:57

reporter   ~0011730

Okay, let's try modifying the metal_pattern_overwood material. Currently it has the wood surface type (even though the textures have /metal/ in their paths), which means arrows stick in the metal plates.

I've made a basic hitmap for it. It's black and white; change it to whatever colours you want.

This is the material def. currently used:

textures/darkmod/door/metal/metal_pattern_overwood
{

    qer_editorimage textures/darkmod/door/metal/metal_pattern_overwood_ed
    wood

    bumpmap textures/darkmod/door/metal/metal_pattern_overwood_local
    diffusemap textures/darkmod/door/metal/metal_pattern_overwood
    specularmap textures/darkmod/door/metal/metal_pattern_overwood_s
    {
        if ( parm11 > 0 )
        blend gl_dst_color, gl_one
        map _white
        rgb 0.40 * parm11
    }
    {
        if ( parm11 > 0 )
        blend add
        map textures/darkmod/door/metal/metal_pattern_overwood
        rgb 0.15 * parm11
    }

    // TDM Ambient Method Related
    {
        if (global5 == 1)
        blend add
        map textures/darkmod/door/metal/metal_pattern_overwood
        scale 1, 1
        red global2
        green global3
        blue global4
    }
    {
        if (global5 == 2)
        blend add
        program ambientEnvironment.vfp
        vertexParm 0 1, 1, 1, 1 // UV Scales for Diffuse and Bump
        vertexParm 1 1, 1, 1, 1 // (X,Y) UV Scale for specular
        vertexParm 2 global2, global3, global4, 1
                                                                
        fragmentMap 0 cubeMap env/gen1
        fragmentMap 1 textures/darkmod/door/metal/metal_pattern_overwood_local // Bump
        fragmentMap 2 textures/darkmod/door/metal/metal_pattern_overwood // Diffuse
        fragmentMap 3 textures/darkmod/door/metal/metal_pattern_overwood_s // Specular
    }
}

If you put that in a .mtr file in /materials/ you can override the existing one and edit it as required.
VanishedOne

VanishedOne

13.04.2019 15:57

reporter  

VanishedOne

VanishedOne

13.04.2019 15:58

reporter  

hitmaptest.map (5,667 bytes)
duzenko

duzenko

13.04.2019 17:14

developer   ~0011731

Added support of materialImage keyword in revision: 8169
My problem now is: I can't see where the game makes decision if an arrow should break or stick depending on the material in the source code. I hope to God it's not scripted somewhere?
VanishedOne

VanishedOne

13.04.2019 17:51

reporter   ~0011734

Last edited: 13.04.2019 17:52

View 2 revisions

Try finding what handles the active_surfaces spawnarg in the projectile defs. It's set to umpteen surftypes for broadheads but only wood for rope arrows, so I'd guess it has something to do with it.

duzenko

duzenko

14.04.2019 07:54

developer   ~0011735

Found it.
Apparently our collision manager operates its own separate data set, that only includes vertex positions. What's worse, it deals with polygons rather than triangles, so there's no simple way to map collision point to a particular 3D model triangle.
To traverse the materialImage texture we need vertex texture coordinates.
It turns out my statement "we have everything we need to calculate this" was wishful thinking.
We either need to add texture coords to collision manager data or try to find the model triangle based on its distance to collision point.
nbohr1more

nbohr1more

14.04.2019 08:12

developer   ~0011736

Last edited: 14.04.2019 08:13

View 4 revisions

It's a similar issue as to why we cannot use the projection images (textures) directly in the AI light scan.

If the game side had access to this data, AI would more accurately know
when dead bodies are illuminated etc.

We might even be able to make the lightgem calc itself cheaper ala the tdm_lg_weak algorithm.

stgatilov

stgatilov

14.04.2019 13:03

administrator   ~0011737

Last edited: 14.04.2019 13:07

View 3 revisions

I think using textures is an entirely wrong way of doing this.
The issue is completely unrelated to graphics. Even collision models are sometimes different from render models.

The correct way would be to set groups of triangles/vertices in collision meshes. I won't be surprised if groups are already supported for collision models and in mesh formats.

The proper workflow is:
1) Collision manager detects an arrow hit.
2) We take contact point: it hits either face, edge, or vertex of the mesh. Just choose some nearby vertex (or face) and check which group it belongs to.
3) Lookup that group in some {group->material} mapping, and handle hit physics/sounds based on the result.

UPDATE: But in fact what I am writing here is already how things work: a portion of object of every material is put into its own surface...

duzenko

duzenko

14.04.2019 14:54

developer   ~0011738

Started a forum thread
http://forums.thedarkmod.com/topic/19959-per-pixel-material-definitions-material-maps/
VanishedOne

VanishedOne

14.04.2019 15:03

reporter   ~0011739

That would work for e.g. Epi's microscope (which uses a "metal" material but has the microscope sitting on what's visually a wooden block; a moveable version would need a .cm added anyway). And I suppose the principle could be extended to vertex blended terrain meshes by checking whether the vertex colour or the inverse vertex colour would be dominant at the collision point. But I'm not clear on how it could work for something like textures/darkmod/door/metal/metal_pattern_overwood -- that's a material applied to big, rectangular, often flat surfaces. Ditto the woodbeams + plaster example in the bug description.
nbohr1more

nbohr1more

14.12.2020 16:50

developer   ~0013123

Duzenko got a prototype working in: rev 8171
Judith

Judith

14.12.2020 16:59

reporter   ~0013125

Is there a description somewhere on what problem is this trying to solve? I'm trying to find a case in terms of a content workflow, and I honestly have no idea.
stgatilov

stgatilov

14.12.2020 17:10

administrator   ~0013127

As far as I understand, it is something like this.

You have an object which has multiple types of materials (metal, wood, etc.).
Normally, you should split the object so that it has several surfaces: one with wooden material, one with metallic material, etc.
Then hitting different points of the object would hit difference surfaces, hence difference materials, hence give different results (sound/stuck).

The issue suggests to have one surface in the object, but add one more texture over it, which has wood/metal/etc. instead of "colors".

To be honest, the idea of use textures (which are usually graphics-only data) to drive gameplay feels like a hack to me, which better be avoided.
But I think there was never serious discussion about it, so the issue is still here.
stgatilov

stgatilov

14.12.2020 17:13

administrator   ~0013128

By the way, we have CPU-only textures now (they were needed for particle collisions).
So it is probably possible to do this given that the object has no custom collision model (i.e. it is using its rendermodel for collisions).
It should be a matter of doing a second rendermodel trace and getting its texcoords, then fetching cpu-resident texture data.
Judith

Judith

14.12.2020 17:23

reporter   ~0013129

Yeah, feels hacky to me too. The process of splitting a model for multiple materials is a known workflow in modelling, for games and beyond. There's no need to replace it, and no need to have an additional texture.

The only really useful thing that came to my mind was blending between materials for terrain models. So, if vertex painting could be interpreted as different materials, not as part of one material with different textures: https://wiki.thedarkmod.com/index.php?title=Vertex_Painting_in_Lightwave
But that ideally would be done automatically, without any additional texture.
stgatilov

stgatilov

14.12.2020 17:26

administrator   ~0013130

The current implementation by Duzenko is very raw though.

For instance, it does not pass the entity which was hit by ordinary collision into renderworld collision.
So you can hit fully-wooden object A but get effect of another fully-metal object B if they are close =)

A more serious problem is that it is integrated into game in one specific place: when projectile hits something.
So e.g. it won't work for hitting with blackjack yet.
It is a big question if this can be done properly architecturally, so that collision queries return final material directly, without caller knowing anything about it.
VanishedOne

VanishedOne

14.12.2020 17:32

reporter   ~0013131

If you have Quake 4, you can dig around in its assets for examples by searching for the materialImage keyword in the material decls. Or if you have ET:QW, I believe surfaceTypeMap is its equivalent.
Judith

Judith

14.12.2020 17:37

reporter   ~0013132

I guess it needs more input from other modelers, but IMO you may end up implementing a quirky feature no one will use.

Proper vertex painting support on the other hand, now that would be very useful, IMO.
VanishedOne

VanishedOne

14.12.2020 20:55

reporter   ~0013133

I do think there's an appreciable risk it'll go underused (it wouldn't be the first such feature, which is possibly a reflection of limited manpower in general), but it's not just a matter of utility for TDM modellers. Insofar as TDM has dedicated modellers making bespoke models for TDM, they of course can optimise their work to suit the engine's capabilities. (Not that people always have. For quite a long time there were window models where you could sink rope arrows into the 'glass' panes. There's still an open bug for 'cushioned' furniture that makes a noisy wooden sound when you drop on it. And of course people are also encouraged to keep draw calls down.) But:
* There aren't that many people with modelling skill making TDM assets: we're fairly lucky to have what we do. The other way to get assets is to scavenge CC stuff from Sketchfab, etc. How easily you can split the meshes by surface type varies, since they're often not made with that in mind.
* Some mappers continue to use brush-and-patch style building methods, either by preference or because modular building approaches the entity limit faster. The original example in this ticket is of flat beam-and-plaster surfaces, which violate the rope-arrows-stick-in-wooden-beams rule unless the mapper takes special countermeasures.
Judith

Judith

14.12.2020 21:05

reporter   ~0013134

I guess per-pixel maps would make more sense when it comes to gradual change between one material and another, but I think in most cases, especially inorganic / architectural models, such splitting is easy to do on a polygon level.
 
So that would be more of a feature for non-modellers then, in order to avoid using Blender and such. But, IIUC, such map would require unwrapping a brush or patch in 0-1 texture space anyway, right? Does DarkRadiant have such texture unwrapping tools?
VanishedOne

VanishedOne

15.12.2020 14:58

reporter   ~0013139

I took 'modeller' to mean someone with enough artistic proficiency to make decent quality assets from scratch, not just convert other people's.

The hitmap shares its texture space with the diffusemap, normal map, etc.

Yes, ground surfaces and other organic blends are the best use case. But I think the point about the beams-and-plaster example is that you're *already* using textures as your guide, to try to recreate physical information from either an unavailable highpoly (if the textures are baked) or the real world (if they're photographic). If you're lucky you have a lowpoly where there are already splits in the right places, and it'll take some time and you'll end up with more draw calls but it won't require much judgment; if you're unlucky you're redesigning a mesh that was designed to pack all that detail into the normal map, without a chance to rebake.

For example, let's look at https://opengameart.org/content/modular-castle-level-construction-kit (actually I'd probably throw out the models and apply the texture sheet to brushwork, but it'll do for a basic example). It's not *that* daunting an example, since you'll be applying the texture sheet to nice flat surfaces. Even in DR you can clip up brushes to approximate those curved door and window shapes. But if we did have this feature, then in addition to terrain and the like it could also result in a much nicer workflow even for cases like this, with faster application of materials to cleaner geometry.

Worth it? Hard to judge; I'd love to know what Raven did to get it working in Q4 if it's such a challenge in a D3 derivative. But it's not outlandish.
Judith

Judith

15.12.2020 17:09

reporter   ~0013140

Okay, so the model will need to be unwrapped in the unique 0-1 texture space, as in the example you provided: https://opengameart.org/sites/default/files/Castle_Diffuse.jpg

This is not what 'non-modeling mappers' usually do. They use multiple tiling materials on split geometry, because that's what stock TDM offers. So you need to implement some UVW unwrapping tool first, or enhance the existing texture tools, so you can easily unwrap UVs in 0-1 texture space. And then mappers would have to be able to align what they have in that texture space with a map made in Gimp, I guess? So, you need a texture exporting tool, or at least UV layout exporting tool. IMO it would really be easier if someone just started learning Blender at this point, because this way you have access to all these options.

One last thing I'm afraid of is that this would encourage increasing material complexity for bad reasons, e.g. people will create material maps for metal frameworks of stained-glass windows just to be "realistic". While in practice most people won't care or notice, since arrows would bounce off glass and metal either way, and a chance of hitting that few "metal" pixels with blackjack is like second to none. TDM stock models have way too high material count per model already, so I'd rather teach people some creative reductive techniques, instead of giving them an incentive to double down on it.
stgatilov

stgatilov

15.12.2020 17:34

administrator   ~0013141

Last edited: 15.12.2020 17:37

View 2 revisions

I don't understand the problem of texcoords.
You simply take diffuse map which already works. Now you look at it with your "mighty human eye" and see where should be glass, and where metal. So you copy this texture, draw red/green/blue on top of it, then lower its resolution with "nearest" filtering, and save as material map. Shouldn't it "just work"?

UPDATE: also, I believe that lack of material map might force someone to increase number of different materials, but not vice versa.
If artist decides to add material map, he does not increase material count, he only increases number of textures to load, plus some time when object is collided with, and engine code complexity of course.
nbohr1more

nbohr1more

15.12.2020 17:47

developer   ~0013142

I still feel that this might be a recipe for performance woes.

Using doing a UV space table lookup for every active collider on the CPU?

If everyone gets a 16-core+ Ryzen and this is well threaded..., I guess?

Hmm. On the other hand, if a pre-processor converts the image map to collision geometry maybe not so bad?

Still means more collision triangles to process though. Lots of tiny triangles everywhere...
Judith

Judith

15.12.2020 18:08

reporter   ~0013143

"I don't understand the problem of texcoords."

Most TDM materials are tiling ones, and typically they're assigned with certain dowscale. Usually mappers create several pieces of geometry using different tiling materials, they adjust the scale, and then they combine the pieces into one model. For material map to work, every surface of a model needs to be unwraped flat in unique space, so you can assign a pixel to a particular spot on the model. So every model will need to be unwrapped like that, and it will need custom _d, _n, _s map set. I think this isn't going to automagically work out of the box, it will require materials like in VanishedOne's example, the texture I linked directly.

Besides, IIUC, this increases the collision / sound detection complexity from per-polygon to per-pixel level. Isn't this going to decrease performance dramatically?
Judith

Judith

15.12.2020 18:29

reporter   ~0013144

Hm, so in fact this could be more beneficial to actual modelers, when to think of it.

Typically, I use one set of textures unwrapped in 0-1 space for a model, but to get different surfaces I need to put the same textures into e.g. two materials with different sound keywords. And then I have to split the model per surface type. So this would allow me to skip the splitting and I'd use one material, at the cost of "ID map".
VanishedOne

VanishedOne

15.12.2020 20:02

reporter   ~0013145

Right, that's the idea (I didn't say it would be of *no* use to dedicated modellers, just that if you're making assets for TDM specifically then you know what constraints the target engine imposes and can design to suit).

Performance cost is unknown, but apparently Q4 and ET:QW could have working implementations on older hardware than today's. I don't think Q4 used it very often, but in QW it's used for the megatexture terrains.

I can see that material keywords like 'scale' could break correspondence between how the diffusemap and material map tile, but wouldn't a collision check require each spot on the model to correspond to a given (not necessarily unique) pixel, rather than each pixel to correspond to a unique spot on the model?

Issue History

Date Modified Username Field Change
21.05.2012 15:57 tels New Issue
22.05.2012 02:44 nbohr1more Note Added: 0004605
23.05.2012 16:30 tels Note Added: 0004617
23.05.2012 22:51 nbohr1more Note Added: 0004618
28.03.2018 10:30 user81 Note Added: 0010233
28.03.2018 18:45 nbohr1more Note Added: 0010319
28.03.2018 18:45 nbohr1more Status new => suspended
11.04.2019 19:39 duzenko Note Added: 0011723
11.04.2019 19:40 duzenko Note Edited: 0011723 View Revisions
12.04.2019 19:31 VanishedOne Note Added: 0011726
13.04.2019 08:13 duzenko Assigned To => duzenko
13.04.2019 08:13 duzenko Status suspended => assigned
13.04.2019 08:14 duzenko File Added: ppm.txt
13.04.2019 08:15 duzenko Note Edited: 0011726 View Revisions
13.04.2019 08:15 duzenko Note Added: 0011727
13.04.2019 08:17 duzenko Note Added: 0011728
13.04.2019 08:17 duzenko Note Edited: 0011728 View Revisions
13.04.2019 15:57 VanishedOne Note Added: 0011730
13.04.2019 15:57 VanishedOne File Added: metal_pattern_overwood_hit.tga
13.04.2019 15:58 VanishedOne File Added: hitmaptest.map
13.04.2019 17:14 duzenko Note Added: 0011731
13.04.2019 17:51 VanishedOne Note Added: 0011734
13.04.2019 17:52 VanishedOne Note Edited: 0011734 View Revisions
14.04.2019 07:54 duzenko Note Added: 0011735
14.04.2019 08:12 nbohr1more Note Added: 0011736
14.04.2019 08:12 nbohr1more Note Edited: 0011736 View Revisions
14.04.2019 08:13 nbohr1more Note Edited: 0011736 View Revisions
14.04.2019 08:13 nbohr1more Note Edited: 0011736 View Revisions
14.04.2019 13:03 stgatilov Note Added: 0011737
14.04.2019 13:03 stgatilov Note Edited: 0011737 View Revisions
14.04.2019 13:07 stgatilov Note Edited: 0011737 View Revisions
14.04.2019 14:54 duzenko Note Added: 0011738
14.04.2019 15:03 VanishedOne Note Added: 0011739
22.03.2020 18:10 duzenko Target Version => TDM 2.09
16.08.2020 08:04 duzenko Assigned To duzenko =>
14.12.2020 16:50 nbohr1more Note Added: 0013123
14.12.2020 16:50 nbohr1more Target Version TDM 2.09 => TDM 2.10
14.12.2020 16:59 Judith Note Added: 0013125
14.12.2020 17:10 stgatilov Note Added: 0013127
14.12.2020 17:13 stgatilov Note Added: 0013128
14.12.2020 17:23 Judith Note Added: 0013129
14.12.2020 17:26 stgatilov Note Added: 0013130
14.12.2020 17:32 VanishedOne Note Added: 0013131
14.12.2020 17:37 Judith Note Added: 0013132
14.12.2020 20:55 VanishedOne Note Added: 0013133
14.12.2020 21:05 Judith Note Added: 0013134
15.12.2020 14:58 VanishedOne Note Added: 0013139
15.12.2020 17:09 Judith Note Added: 0013140
15.12.2020 17:34 stgatilov Note Added: 0013141
15.12.2020 17:37 stgatilov Note Edited: 0013141 View Revisions
15.12.2020 17:47 nbohr1more Note Added: 0013142
15.12.2020 18:08 Judith Note Added: 0013143
15.12.2020 18:29 Judith Note Added: 0013144
15.12.2020 20:02 VanishedOne Note Added: 0013145
16.02.2021 15:06 stgatilov Assigned To => stgatilov
16.02.2021 15:06 stgatilov Status assigned => new
16.02.2021 15:07 stgatilov Assigned To stgatilov =>
15.03.2021 15:45 stgatilov Target Version TDM 2.10 => TDM 2.11