View Issue Details

IDProjectCategoryView StatusLast Update
0005612The Dark ModTDM Installerpublic27.07.2021 14:56
Reporteres20490446e Assigned Tostgatilov  
PrioritylowSeverityfeatureReproducibilityalways
Status closedResolutionwon't fix 
Summary0005612: Downloading the game is unnecesarily slow
DescriptionThe installer always download from the main mirror, which is quite slow as all the stress is put on it.

What I'm doing on my own building scripts is ranking all the available mirrors at "http://mirrors.thedarkmod.com/tdm_mirrors.txt" by ping, and building the installer with its sources automatically modified to download from best mirror.

From that I go from 0.5MB/s to 10MB/s. Twenty times speed improvement.
TagsNo tags attached.

Activities

stgatilov

stgatilov

20.05.2021 14:59

administrator   ~0014033

Installer only downloads configuration files and manifests from the main server.
That very little traffic: something like 20 MB on fresh install of 2.09 release, and almost nothing on subsequent runs.
This is enforced for security reasons.

As for the main downloads, they are taken from randomly chosen mirrors.
The mirrors are weighted by probability, so that mirror owners could control how much traffic it takes.
Perhaps we should tweak weights so that only the fastest mirror is used unless it is unavailable.

If you put modified tdm_installer anywhere, please mark it as tweaked by you in the window title.
es20490446e

es20490446e

20.05.2021 15:09

reporter   ~0014034

The window isn't shown. I use the installer to automate asset downloading on package building, and I purposely created a tool that hides the window.

https://gitlab.com/es20490446e/darkmod-linux
https://gitlab.com/es20490446e/unwin

The only thing that is modified is the mirror. At building time the mirror with the least amount of ping is selected. The installer isn't meant to be used after the package is built, and it isn't included inside the package.

Anyway the builder is interpreted code, so anything it does is transparent. For instance it also completely deblobs all the sources and builds everything from source, for total transparency.

https://gitlab.com/es20490446e/blobs

The next improvement is compiling and building in parallel, so the package building takes likely the same time as without compiling.
es20490446e

es20490446e

20.05.2021 15:14

reporter   ~0014035

I mean downloading and compiling in parallel.
stgatilov

stgatilov

12.06.2021 12:54

administrator   ~0014096

I don't think I'll do anything about it in foreseeable future.

The mirrors weighting system is designed to distribute load across mirrors, and it'd better remain like that.
I have added a way to select one mirror in the GUI, users can use it if they want.

By the way, I don't see much difference between mirrors from my location.

Issue History

Date Modified Username Field Change
13.05.2021 08:29 es20490446e New Issue
16.05.2021 21:45 nbohr1more Category Design/Coding => TDM Updater
19.05.2021 05:34 stgatilov Category TDM Updater => TDM Installer
20.05.2021 14:59 stgatilov Note Added: 0014033
20.05.2021 15:09 es20490446e Note Added: 0014034
20.05.2021 15:14 es20490446e Note Added: 0014035
12.06.2021 12:54 stgatilov Note Added: 0014096
12.06.2021 12:54 stgatilov Priority normal => low
12.06.2021 12:54 stgatilov Severity major => feature
27.07.2021 14:56 stgatilov Assigned To => stgatilov
27.07.2021 14:56 stgatilov Status new => closed
27.07.2021 14:56 stgatilov Resolution open => won't fix