View Issue Details

IDProjectCategoryView StatusLast Update
0005479DarkRadiantDesign/Codingpublic09.01.2021 00:54
Reporterjonri Assigned Tojonri  
PrioritynormalSeverityfeatureReproducibilityalways
Status assignedResolutionopen 
Summary0005479: Auto-reload models, skins, materials, etc
DescriptionWe currently have a full series of things to manually reload when their files are changed externally:

Reload Models
Reload Skins
Reload Scripts
Reload Readable Guis
Reload Materials
Reload Defs
Reload Particles
Reload Sounds

Using wxFileSystemWatcher it should be relatively easy and straightforward to monitor the directories for each of the above for changes, and automatically call the relevant Reload function when the files change. I have done a simple proof-of-concept that will auto-reload models and it seems to work as expected.

System design considerations:

* Since this depends on wx, it belongs outside of radiantcore. It seems too small to be a plugin, so it should probably go under darkradiant. I stuck it under uimanager for now, but it might be better under eventmanager? Some guidance would be appreciated here.

* The wxWidgets documentation mentions some minor differences in the way wxFileSystemWatcher works based on platform. None of the differences will prevent the feature from working, but they need to be kept in mind during development. Testing this feature on all supported platforms is a must as well.

* I plan to only watch for changes under the current FM's root path (ModPath in DR). Is there any reason to watch the TDM directory as well?

* In the proof-of-concept, some operations (rename for example) would generate a flurry of events. To prevent excessive reloads, we can use a timer with a short interval to wait for the events to settle, and only fire off one reload command. This will also be helpful if someone is using an external tool to run a batch operation on several files at once, exporting multiple models from a source file, etc.

* Besides the main menu, there are other places in the UI where the user can call the reload functions. For example, the model browser has both Reload Models and Reload Skins buttons. We should make sure that if files are auto-reloaded while these dialogs are open, the effect is the same as if the user had pressed the button.

* There may be other dialogs that interact with the reloadable items but don't expect to have them changed out from under them. These will need to be identified and tested.

* If a file is auto-reloaded but contains an error, we should make sure this never crashes DarkRadiant. Such bugs would need to be fixed regardless, but auto-reload would make them more obvious.

* We should keep the manual reload functionality in place and also have a menu option to turn the auto-reload off. While I imagine most users will like the feature, there may be corner cases where it doesn't work. Network drives are notorious for messing with filesystem notifications for example.


TagsNo tags attached.

Activities

greebo

greebo

31.12.2020 04:44

administrator   ~0013326

As for design considerations, I'd put this feature into its own module (within the UI binary, that's fine) - you can use the AutoSaver or the ClipboardModule as blueprints. Feel free to create a separate folder in the radiant/ subtree, like "clipboard".

I'm not quite sure if all the dialogs can deal with the fact that some trees being reloaded in the background, that might be tricky. A few of them have been adjusted already, as you noticed, like the MediaBrowser: it monitors the MaterialManager for unload/load events and repopulates the tree when this happens.

Monitoring the FM root is fine, some mappers or devs might also use the TDM root - maybe make this configurable?

I'd start with some basic functionality first, and then get some mapper feedback in the forums - that way you're sure to create something people want.

(On this topic, I wasn't aware of wxFileSystemWatcher - this also gives me the idea of adding an ASE model material adjuster. Since exporting ASE models from Blender has never the correct image paths, the watcher might notice when a new model is added and adjusts the BITMAP paths to point to a correct shader name automatically. I must've done this manually like hundreds of times.)
jonri

jonri

03.01.2021 03:57

developer   ~0013350

Thanks for the tips! I'm starting off with models, skins, and materials since I believe those will be the most useful and I think all the dialogs are able to deal with them. At that point I'll request some feedback on whether I should worry about doing the others.

"Since exporting ASE models from Blender has never the correct image paths, the watcher might notice when a new model is added and adjusts the BITMAP paths to point to a correct shader name automatically. I must've done this manually like hundreds of times.)"

Can you expand more on the incorrect path issue from Blender? I've only done a couple small modular sets that way so far, but I've only once had an incorrect image path due to Blender's 64-character limit on material names. It's off-topic for this issue, but I think the other way around needs some improvements too - making some Blender plugin to import TDM materials into your Blender file with them named correctly and hooked up to their textures would be really helpful. I haven't really thought about what that would look like yet, but it would definitely help out if you were doing a large amount of modeling using existing materials (as opposed to creating and texturing a model from scratch where it's not a big deal to set up a new material for it).
greebo

greebo

03.01.2021 10:17

administrator   ~0013351

It may be related to the exporter version I used back in the day, this has been more than a several years ago. The exported ASE models used to have the image name in the BITMAP field - you could put a idTech material name in there, but then I'd have problems with rendering in Blender. It's entirely possible that I missed a step, or the exporters have improved since then.
Dragofer

Dragofer

03.01.2021 11:05

developer   ~0013352

Last edited: 03.01.2021 11:10

This sounds quite convenient, to have DR automatically take care of updating itself whenever you're working or incrementing on an asset. Hitherto it's often been simpler just to restart the application than reload models + materials + skins + entity defs when adding a custom entity.

Blender & ASE: R Soul has already made a plugin that sounds like what you want to do, but I never really managed to get it to do what I expected. For example, I imported a model with a copper material into Blender and it was assigned the preview image of a shadow texture. I believe one of the things it's supposed to do is circumvent the 64 character limit by allowing you to specify longer names in texture slots, rather than material slots.

https://forums.thedarkmod.com/index.php?/topic/19196-blender-tdm-material-manager-plugin-updated-to-version-280-blender-280-update/
Edit: seems to apply only to .lwo's

Personally I've always named Blender materials after TDM materials, never had an issue with exporting to TDM except for the notorious 64 character limit. I've never needed rendering of TDM models in Blender, just assigning the diffusemap as a preview image was sufficient, but labour-intensive.

Related Changesets

DarkRadiant: autoreload d2c36e2a

08.01.2021 21:50

jonri


Details Diff
0005479: Initial implementation of auto-reload for models, materials, and skins Affected Issues
0005479
mod - radiant/CMakeLists.txt Diff File
add - radiant/autoreload/AutoReloadManager.cpp Diff File
add - radiant/autoreload/AutoReloadManager.h Diff File

DarkRadiant: autoreload d85f8cf9

09.01.2021 00:52

jonri


Details Diff
0005479: Update MSVC build files, prevent reload while map is loading Affected Issues
0005479
mod - radiant/autoreload/AutoReloadManager.cpp Diff File
mod - tools/msvc/DarkRadiant.vcxproj Diff File
mod - tools/msvc/DarkRadiant.vcxproj.filters Diff File

Issue History

Date Modified Username Field Change
31.12.2020 02:52 jonri New Issue
31.12.2020 02:52 jonri Status new => assigned
31.12.2020 02:52 jonri Assigned To => jonri
31.12.2020 04:44 greebo Note Added: 0013326
03.01.2021 03:57 jonri Note Added: 0013350
03.01.2021 10:17 greebo Note Added: 0013351
03.01.2021 11:05 Dragofer Note Added: 0013352
03.01.2021 11:06 Dragofer Note Edited: 0013352
03.01.2021 11:09 Dragofer Note View State: 0013352: private
03.01.2021 11:10 Dragofer Note Edited: 0013352
03.01.2021 11:10 Dragofer Note View State: 0013352: public
08.01.2021 22:13 jonri Changeset attached => DarkRadiant autoreload d2c36e2a
09.01.2021 00:54 jonri Changeset attached => DarkRadiant autoreload d85f8cf9