View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0005250 | The Dark Mod | Distribution | public | 11.05.2020 09:45 | 19.01.2021 15:29 |
Reporter | stgatilov | Assigned To | stgatilov | ||
Priority | high | Severity | major | Reproducibility | N/A |
Status | resolved | Resolution | fixed | ||
Product Version | TDM 2.08 | ||||
Target Version | TDM 2.09 | Fixed in Version | TDM 2.09 | ||
Summary | 0005250: New differential update and weekly builds | ||||
Description | Full description is provided here: https://forums.thedarkmod.com/index.php?/topic/19837-208-new-tdm_update/ The rough idea is to: 1) Add new type of manifests, listing all files inside zips with secure BLAKE2 hashes. 2) Differential update over HTTP, downloading chunks directly from zips (strongly ties updater to zip file format). 3) Allow using several manifest/sources to build single version of TDM. | ||||
Tags | No tags attached. | ||||
Purely out of curiosity: why BLAKE2, specifically? :) | |
It is quite recent and secure. And among such hashes, this one is very fast. Since the updater will update from arbitrary current state, it will have to compute hashes of the whole TDM installation every launch. This is the biggest downside of the new system, so the speed of hashing matters a lot. |
|
Moreover, it is 32-bit BLAKE2 hash, since our Windows updater is still 32-bit =) Although it we would stop providing 32-bit build of TDM out of the box, it would probably make sense to make updater 64-bit too. |
|
Ok, fair enough :) Btw. the BLAKE2 website states that BLAKE3 is available and even faster :D What is not clear to me, though, is if the speed comparisons against other hash algorithms take hardware acceleration into account? For example, SHA1 and SHA256 have specific SSE instructions supported by Intel and Ryzen CPUs. |
|
I think SHA instructions are not taken into account, because only the most recent CPUs have them. I wouldn't rely on them either. BLAKE2 uses something like SSE2 which is taken into account and is in fact required by most software. Besides, are there instructions to accelerate SHA2 and SHA3 computation? BLAKE3 did not even come out when I searched for a hash function. And I don't see any security analysis of it yet. |
|
SHA-256 and SHA-224 are supported as part of the SHA-2 family, but nothing above that. Also you're right; it appears that Intel has chosen to implement those instructions only on a subset of specialized CPUs, and for standard desktop CPUs the instructions are only available from Cannon Lake upwards. So actually, AMD with Ryzen now has probably more widespread support for it than Intel :D | |
I decided to settle on FLTK for GUI. Perhaps one day we can use it for tools --- although we should decide if it would be better or worse than ImGui. Added third-party libs: svn rev 8784 and 8786. First version of installer committed in svn rev 8790 and 8791. Of course, nothing works properly there =) I have just started. The zipsync subdirectory will be kept in sync with my GitHub repo: https://github.com/stgatilov/zipsync But updating that repo is purely my own responsibility, of course. |
|
After a lot more fixes, the installer has been published for preliminary testing: https://forums.thedarkmod.com/index.php?/topic/20460-new-tdm_installer-and-dev-builds/ |
|
Something like 7 updates to tdm_installer has already been published, and it is fairly usable now. I have also written a detailed wiki article about it: https://wiki.thedarkmod.com/index.php?title=Tdm_installer_and_zipsync |
|
Added two new features: 1) Better mirrors handling. First of all, mirrors can now be weighted in config file --- for better load balancing. Second, if one mirror goes down, it does not prevent installation now. Except for the core files (installer, config, target manifest), which must always come from the main TDM server. 2) Added "unattended" mode. In theory, it allows installing TDM as part of some script. |
|
The installer is being used in 2.09 beta, and the feedback has been good so far. I think I'll consider it "done" and create new issues if some problems happens. |
|
Date Modified | Username | Field | Change |
---|---|---|---|
11.05.2020 09:45 | stgatilov | New Issue | |
11.05.2020 09:45 | stgatilov | Status | new => assigned |
11.05.2020 09:45 | stgatilov | Assigned To | => stgatilov |
11.05.2020 09:45 | stgatilov | Relationship added | related to 0004833 |
11.05.2020 10:20 | cabalistic | Note Added: 0012486 | |
11.05.2020 10:28 | stgatilov | Note Added: 0012487 | |
11.05.2020 10:29 | stgatilov | Note Added: 0012488 | |
11.05.2020 10:40 | cabalistic | Note Added: 0012489 | |
11.05.2020 11:10 | stgatilov | Note Added: 0012490 | |
11.05.2020 11:17 | cabalistic | Note Added: 0012491 | |
02.07.2020 17:30 | stgatilov | Note Added: 0012644 | |
09.07.2020 16:49 | stgatilov | Note Added: 0012647 | |
19.07.2020 12:53 | stgatilov | Note Added: 0012661 | |
19.07.2020 12:53 | stgatilov | Note Edited: 0012661 | |
08.11.2020 04:01 | stgatilov | Note Added: 0012865 | |
17.12.2020 03:56 | stgatilov | Note Added: 0013154 | |
17.12.2020 03:56 | stgatilov | Status | assigned => resolved |
17.12.2020 03:56 | stgatilov | Resolution | open => fixed |
17.12.2020 03:56 | stgatilov | Fixed in Version | => TDM 2.09 |
19.01.2021 15:28 | stgatilov | Relationship added | related to 0005076 |
19.01.2021 15:29 | stgatilov | Relationship added | related to 0005495 |