View Issue Details

IDProjectCategoryView StatusLast Update
0004408The Dark ModCodingpublic27.12.2020 13:55
Reporterduzenko Assigned Toduzenko  
PrioritynormalSeveritynormalReproducibilityhave not tried
Status resolvedResolutionfixed 
Product VersionSVN 
Target VersionTDM 2.06Fixed in VersionTDM 2.06 
Summary0004408: Find a way to run game tics on a background thread
DescriptionSince virtually all modern pc's have more than one core it would make sense to move things out of the main thread.
DOOM3 BFG was designed to run both game tics and front renderer on a thread.
I think it should be easier in TDM to do the game tics first.
TagsNo tags attached.

Relationships

related to 0004621 resolvednbohr1more Lightgem does not initialize when installing a new mission or via vid_restart. 
related to 0005169 assignedcabalistic Clean multithreading routines 
related to 0005468 resolvedcabalistic Debug rendering like s_drawSounds does not work under com_smp 
child of 0003684 new Investigate GPL Renderer Improvements 

Activities

nbohr1more

nbohr1more

31.10.2016 19:07

developer   ~0008413

For Reference:

Fabien Sanglard - The rendering system is now broken down in a frontend/backend: It reminds me of the design of a compiler which usually has a frontend->IR->backend pipeline. What this inspired by the design of LCC which was used for Quake3 bytecode generation ? I wonder what are the advantages over a monolithic renderer like Doom, idTech1 and idTech2.

John Carmack - This was explicitly to support dual processor systems. It worked well on my dev system, but it never seemed stable enough in broad use, so we backed off from it. Interestingly, we only just found out last year why it was problematic (the same thing applied to Rage’s r_useSMP option, which we had to disable on the PC) – on windows, OpenGL can only safely draw to a window that was created by the same thread. We created the window on the launch thread, but then did all the rendering on a separate render thread. It would be nice if doing this just failed with a clear error, but instead it works on some systems and randomly fails on others for no apparent reason.

The Doom 4 codebase now jumps through hoops to create the game window from the render thread and pump messages on it, but the better solution, which I have implemented in another project under development, is to leave the rendering on the launch thread, and run the game logic in the spawned thread.

http://fabiensanglard.net/doom3/interviews.php
duzenko

duzenko

31.10.2016 19:40

developer   ~0008414

So far my attempts have been unsuccessful. Game tics have a bad habit to clear vertex ambient cache on model surfaces.
duzenko

duzenko

31.10.2016 19:42

developer   ~0008415

Last edited: 31.10.2016 19:44

@nbohr1more
Yeah, I read that a while ago.
I have no plans about touching the back renderer the way they were talking about here. Only non-opengl things, namely AI.
BTW "another project" is Doom3 BFG.

nbohr1more

nbohr1more

31.10.2016 19:46

developer   ~0008416

Last edited: 31.10.2016 19:48

Perhaps if you changed the VBO behavior per tracker:

http://bugs.thedarkmod.com/view.php?id=3234

RaynorPat has a hybrid renderer with Map Buffer Range VBO

https://github.com/raynorpat/Doom3/tree/master/neo/renderer

similar fix here:

https://github.com/revelator/Revelation-Engine/tree/master/neo/renderer

duzenko

duzenko

31.10.2016 20:03

developer   ~0008417

Last edited: 31.10.2016 20:08

I would like to comment on that but I simply don't know the renderer that good.
It looks that the vertex data in the cpu memory is cleared, especially often during the "frob check" when a model is directly in front of the player.
I mean it's definitely what I am interested in but only after I understand how the renderer works and how data is processed starting from the model animator in game tics.

nbohr1more

nbohr1more

31.10.2016 20:34

developer   ~0008418

That VBO fix makes the buffer more persistent rather than juggling two VBO streams. I imagine if you go async, you could provoke a buffer change at the wrong point in that juggling act, just hunch. I'll try implementing if I get time during this dev cycle.
duzenko

duzenko

31.10.2016 20:36

developer   ~0008419

Last edited: 31.10.2016 20:37

Honestly I can't remember anything in TDM that would look like two VBO streams - can you point to a specific place in code?

nbohr1more

nbohr1more

31.10.2016 20:51

developer   ~0008420

The patch affects:

VertexCache.cpp and VertexCache.h

but tr_render.cpp is where the counters are controlled.
duzenko

duzenko

31.10.2016 21:03

developer   ~0008421

IMHO it wouldn't be enough in my case because the data is cleared on the model level. By vertex cache I meant the model class field called ambientCache
nbohr1more

nbohr1more

31.10.2016 21:14

developer   ~0008422

Hmm, the majority of ambient cache manipulation looks to be done in tr_light.cpp...
duzenko

duzenko

31.10.2016 21:18

developer   ~0008423

Last edited: 31.10.2016 21:23

Yeah, but there is more to it.
Regret it did not crash for me today but just run TDM with async tic on and you will see warnings in console about ambientcache. If you're lucky it crashes for you and you will see in stack trace for the game tic thread where exactly it clears ambientCache.
It should help if you have dynamic models in the frobable area in front of you.
I test on the ship in Down by the riverside with notarget and just follow the captain around.
Usually renderer only complains about missing ambientCache but sometimes when it is cleared after the check in BR but before actual use then it crashes.

nbohr1more

nbohr1more

31.10.2016 21:25

developer   ~0008424

Sorry for all the grasping here but I am tracing back to where timing and events are spawned.

I looks like the next level down the rabbit hole is ModelManager.cpp.

Perhaps we need to examine what is feeding ID_TIME_T in
idRenderModelManagerLocal::ReloadModels ?

Not very confident in that though.
duzenko

duzenko

31.10.2016 21:33

developer   ~0008425

Last edited: 31.10.2016 21:41

Ok, how about this stack trace? It happens when you have e.g. a guard in a frobbing area.
> TheDarkMod.exe!R_FreeStaticTriSurfVertexCaches(srfTriangles_s * tri) Line 339 C++
     TheDarkMod.exe!idMD5Mesh::UpdateSurface(const renderEntity_s * ent, const idJointMat * entJoints, modelSurface_s * surf) Line 282 C++
> TheDarkMod.exe!idRenderModelMD5::InstantiateDynamicModel(const renderEntity_s * ent, const viewDef_s * view, idRenderModel * cachedModel) Line 715 C++
     TheDarkMod.exe!R_EntityDefDynamicModel(idRenderEntityLocal * def) Line 1107 C++
     TheDarkMod.exe!idRenderWorldLocal::ModelTrace(modelTrace_s & trace, int entityHandle, const idVec3 & start, const idVec3 & end, const float radius) Line 1095 C++
     gamex86.dll!idClip::TraceRenderModel(trace_s & trace, const idVec3 & start, const idVec3 & end, const float radius, const idMat3 & axis, idClipModel * touch) Line 949 C++
     gamex86.dll!idClip::Translation(trace_s & results, const idVec3 & start, const idVec3 & end, const idClipModel * mdl, const idMat3 & trmAxis, int contentMask, const idEntity * passEntity) Line 1129 C++
     gamex86.dll!idPlayer::PerformFrobCheck() Line 10994 C++
     gamex86.dll!idPlayer::Think() Line 7610 C++
     gamex86.dll!idGameLocal::RunFrame(const usercmd_t * clientCmds) Line 3389 C++
     TheDarkMod.exe!idSessionLocal::RunGameTic() Line 2860 C++
     TheDarkMod.exe!idSessionLocal::AsyncTick() Line 2578 C++
     TheDarkMod.exe!idCommonLocal::Async() Line 2607 C++

nbohr1more

nbohr1more

31.10.2016 21:51

developer   ~0008426

Well the VBO side ot that code is in tr_trisurf.cpp where is calls vertexCache.free in R_FreeStaticTriSurfVertexCaches but there are no timers there so it must be happening in VertexCache.cpp

Model_md5.cpp has

idRenderModelMD5::InstantiateDynamicModel

could be fighting against VertexCache actions when performing "delete cachedModel" actions.
duzenko

duzenko

31.10.2016 21:54

developer   ~0008427

That's where my expertise ends anywhere.
I can see that back renderer and game tic fight for ambientCache but I have no idea how to work around it.
nbohr1more

nbohr1more

01.11.2016 21:38

developer   ~0008434

Hmm...

Maybe make ambientCache atomic:

Model.h

struct atomic<vertCache_s> * ambientCache;
nbohr1more

nbohr1more

07.11.2016 16:13

developer   ~0008448

I spent a few moments poking at this some more.

It looks like what we want is a mutex in the srfTriangles_t struct.

Here's a discussion of an implementation:

http://stackoverflow.com/questions/29603513/assignment-operator-in-struct-after-adding-mutex-in-c

perhaps you can parse the example better than I?
duzenko

duzenko

07.11.2016 18:29

developer   ~0008449

Hmm.. it would make sense to use mutex if we had a racing condition here.
But I think it's a different kind of a problem - invalid state (caused by wrong call order).
I.e. the default order is
1. idMD5Mesh::UpdateSurface - ambient cache cleared (in this case, in player AI code - game tick)
2. ??? - ambient cache properly set, probably somewhere in front renderer
3. RB_T_FillDepthBuffer - ambient cache used in back renderer
When game tick (1) is running at the same time as back renderer (3) it gives an error because (2) was not called in between.
nbohr1more

nbohr1more

07.11.2016 19:33

developer   ~0008451

Last edited: 07.11.2016 19:45

Hmm. Step 2 sounds like

R_AddLightSurfaces

in tr_light.cpp

Loop unrolled:

http://fabiensanglard.net/doom3/doom3_unrolled.php

Hmm

R_AddAmbientDrawsurfs

// touch it so it won't get purged
vertexCache.Touch( tri->ambientCache );

duzenko

duzenko

07.11.2016 20:24

developer   ~0008453

Last edited: 07.11.2016 20:25

Yeah, R_AddAmbientDrawsurfs + R_CreateAmbientCache look very much it.
I think the best way out of this would be to somehow make idPlayer::PerformFrobCheck do its job without clearing ambient cache.
https://s15.postimg.org/o5nwikxaz/Untitled.png
I am a little confused with all the big words here like dynamic model or model trace.

duzenko

duzenko

18.11.2016 21:53

developer   ~0008509

Front renderer sends to back surfaces for dynamic models that exist at that specific time.
Game tic clears dynamic models for AI entities (probably for all others too).
Conclusion: front render must be between game tic and back render.
nbohr1more

nbohr1more

22.11.2016 18:51

developer   ~0008535

"com_asyncTic 1" crashes for me in the latest builds. As code freeze will be here shortly, can we move this to 2.06?
duzenko

duzenko

22.11.2016 18:53

developer   ~0008536

We should absolutely move it to 2.06.
Where does it crash? On md5 ambient cache?
nbohr1more

nbohr1more

22.11.2016 20:19

developer   ~0008537

I'll get more data about the crash but I do recall that it's no longer ambient cache. Appears to be font related as I recall.
duzenko

duzenko

25.11.2016 20:24

developer   ~0008556

Wrapped std::thread in an ifdef to appease the old gcc.
duzenko

duzenko

06.08.2017 12:50

developer   ~0009057

Cabalistic's patch applied
duzenko

duzenko

14.08.2017 09:41

developer   ~0009085

Added the r_smp cvar
nbohr1more

nbohr1more

17.09.2017 21:45

developer   ~0009282

Current implementation is as far as we'll get with 2.06
nbohr1more

nbohr1more

20.10.2017 05:01

developer   ~0009520

Last edited: 20.10.2017 05:01

Disabled SMP for debug render tools like r_showSurfaceInfo.

Now these no longer reset to mission start or crash when com_smp 1 is active.

Rev 7252

Dragofer

Dragofer

27.12.2020 13:55

developer   ~0013297

Last edited: 27.12.2020 13:55

This should make game tics run at a constant 60 per second (16ms per tic) regardless of visual frame rate, right? I've tried to test this with

float start_time = sys.getTime();
sys.waitFrame();
sys.println("time elapsed is " + (sys.getTime() - start_time));

and found that the time per frame does change with visual FPS, i.e. 24ms if I run at uncapped 85fps, 16ms if I run at capped 60 fps. Obsttorte has also found a dependent relationship.

Issue History

Date Modified Username Field Change
31.10.2016 18:49 duzenko New Issue
31.10.2016 18:52 duzenko Assigned To => duzenko
31.10.2016 18:52 duzenko Status new => assigned
31.10.2016 19:07 nbohr1more Note Added: 0008413
31.10.2016 19:40 duzenko Note Added: 0008414
31.10.2016 19:42 duzenko Note Added: 0008415
31.10.2016 19:42 duzenko Note Edited: 0008415
31.10.2016 19:44 duzenko Note Edited: 0008415
31.10.2016 19:46 nbohr1more Note Added: 0008416
31.10.2016 19:47 nbohr1more Note Edited: 0008416
31.10.2016 19:48 nbohr1more Note Edited: 0008416
31.10.2016 20:03 duzenko Note Added: 0008417
31.10.2016 20:07 duzenko Note Edited: 0008417
31.10.2016 20:08 duzenko Note Edited: 0008417
31.10.2016 20:34 nbohr1more Note Added: 0008418
31.10.2016 20:36 duzenko Note Added: 0008419
31.10.2016 20:37 duzenko Note Edited: 0008419
31.10.2016 20:51 nbohr1more Note Added: 0008420
31.10.2016 21:03 duzenko Note Added: 0008421
31.10.2016 21:14 nbohr1more Note Added: 0008422
31.10.2016 21:18 duzenko Note Added: 0008423
31.10.2016 21:19 duzenko Note Edited: 0008423
31.10.2016 21:20 duzenko Note Edited: 0008423
31.10.2016 21:21 duzenko Note Edited: 0008423
31.10.2016 21:21 duzenko Note Edited: 0008423
31.10.2016 21:23 duzenko Note Edited: 0008423
31.10.2016 21:25 nbohr1more Note Added: 0008424
31.10.2016 21:33 duzenko Note Added: 0008425
31.10.2016 21:35 duzenko Note Edited: 0008425
31.10.2016 21:40 duzenko Note Edited: 0008425
31.10.2016 21:40 duzenko Note Edited: 0008425
31.10.2016 21:41 duzenko Note Edited: 0008425
31.10.2016 21:51 nbohr1more Note Added: 0008426
31.10.2016 21:54 duzenko Note Added: 0008427
01.11.2016 21:38 nbohr1more Note Added: 0008434
07.11.2016 16:13 nbohr1more Note Added: 0008448
07.11.2016 18:29 duzenko Note Added: 0008449
07.11.2016 19:33 nbohr1more Note Added: 0008451
07.11.2016 19:45 nbohr1more Note Edited: 0008451
07.11.2016 20:24 duzenko Note Added: 0008453
07.11.2016 20:25 duzenko Note Edited: 0008453
18.11.2016 21:53 duzenko Note Added: 0008509
22.11.2016 17:38 nbohr1more Status assigned => resolved
22.11.2016 17:38 nbohr1more Resolution open => fixed
22.11.2016 17:38 nbohr1more Product Version => SVN
22.11.2016 17:38 nbohr1more Fixed in Version => TDM 2.05
22.11.2016 17:38 nbohr1more Target Version => TDM 2.05
22.11.2016 17:38 nbohr1more Status resolved => assigned
22.11.2016 17:38 nbohr1more Resolution fixed => open
22.11.2016 18:51 nbohr1more Note Added: 0008535
22.11.2016 18:53 duzenko Note Added: 0008536
22.11.2016 20:19 nbohr1more Note Added: 0008537
22.11.2016 20:20 nbohr1more Fixed in Version TDM 2.05 =>
22.11.2016 20:20 nbohr1more Target Version TDM 2.05 => TDM 2.06
25.11.2016 20:24 duzenko Note Added: 0008556
01.12.2016 21:11 nbohr1more Relationship added child of 0003684
06.08.2017 12:50 duzenko Note Added: 0009057
14.08.2017 09:41 duzenko Note Added: 0009085
17.09.2017 21:44 nbohr1more Status assigned => feedback
17.09.2017 21:45 nbohr1more Note Added: 0009282
18.09.2017 13:38 nbohr1more Relationship added related to 0004621
20.09.2017 04:28 nbohr1more Status feedback => resolved
20.09.2017 04:28 nbohr1more Resolution open => fixed
20.09.2017 04:28 nbohr1more Fixed in Version => TDM 2.06
20.10.2017 05:01 nbohr1more Note Added: 0009520
20.10.2017 05:01 nbohr1more Note Edited: 0009520
14.12.2020 23:08 nbohr1more Relationship added related to 0005169
27.12.2020 10:30 stgatilov Relationship added related to 0005468
27.12.2020 13:55 Dragofer Note Added: 0013297
27.12.2020 13:55 Dragofer Note Edited: 0013297