View Issue Details

IDProjectCategoryView StatusLast Update
0005250The Dark ModDistributionpublic19.07.2020 12:53
Reporterstgatilov Assigned Tostgatilov  
Status assignedResolutionopen 
Product VersionTDM 2.08 
Target VersionTDM 2.09 
Summary0005250: New differential update and weekly builds
DescriptionFull description is provided here:

The rough idea is to:
1) Add new type of manifests, listing all files inside zips with secure BLAKE2 hashes.
2) Differential update over HTTP, downloading chunks directly from zips (strongly ties updater to zip file format).
3) Allow using several manifest/sources to build single version of TDM.
TagsNo tags attached.


related to 0004833 resolvedstgatilov zsync-like differential updates (unofficial prototype) 




11.05.2020 10:20

developer   ~0012486

Purely out of curiosity: why BLAKE2, specifically? :)


11.05.2020 10:28

administrator   ~0012487

It is quite recent and secure. And among such hashes, this one is very fast.

Since the updater will update from arbitrary current state, it will have to compute hashes of the whole TDM installation every launch.
This is the biggest downside of the new system, so the speed of hashing matters a lot.


11.05.2020 10:29

administrator   ~0012488

Moreover, it is 32-bit BLAKE2 hash, since our Windows updater is still 32-bit =)
Although it we would stop providing 32-bit build of TDM out of the box, it would probably make sense to make updater 64-bit too.


11.05.2020 10:40

developer   ~0012489

Ok, fair enough :) Btw. the BLAKE2 website states that BLAKE3 is available and even faster :D

What is not clear to me, though, is if the speed comparisons against other hash algorithms take hardware acceleration into account? For example, SHA1 and SHA256 have specific SSE instructions supported by Intel and Ryzen CPUs.


11.05.2020 11:10

administrator   ~0012490

I think SHA instructions are not taken into account, because only the most recent CPUs have them. I wouldn't rely on them either.
BLAKE2 uses something like SSE2 which is taken into account and is in fact required by most software.
Besides, are there instructions to accelerate SHA2 and SHA3 computation?

BLAKE3 did not even come out when I searched for a hash function. And I don't see any security analysis of it yet.


11.05.2020 11:17

developer   ~0012491

SHA-256 and SHA-224 are supported as part of the SHA-2 family, but nothing above that. Also you're right; it appears that Intel has chosen to implement those instructions only on a subset of specialized CPUs, and for standard desktop CPUs the instructions are only available from Cannon Lake upwards. So actually, AMD with Ryzen now has probably more widespread support for it than Intel :D


02.07.2020 17:30

administrator   ~0012644

I decided to settle on FLTK for GUI.
Perhaps one day we can use it for tools --- although we should decide if it would be better or worse than ImGui.

Added third-party libs: svn rev 8784 and 8786.

First version of installer committed in svn rev 8790 and 8791.
Of course, nothing works properly there =) I have just started.

The zipsync subdirectory will be kept in sync with my GitHub repo:
But updating that repo is purely my own responsibility, of course.


09.07.2020 16:49

administrator   ~0012647

After a lot more fixes, the installer has been published for preliminary testing:


19.07.2020 12:53

administrator   ~0012661

Last edited: 19.07.2020 12:53

View 2 revisions

Something like 7 updates to tdm_installer has already been published, and it is fairly usable now.

I have also written a detailed wiki article about it:

Issue History

Date Modified Username Field Change
11.05.2020 09:45 stgatilov New Issue
11.05.2020 09:45 stgatilov Status new => assigned
11.05.2020 09:45 stgatilov Assigned To => stgatilov
11.05.2020 09:45 stgatilov Relationship added related to 0004833
11.05.2020 10:20 cabalistic Note Added: 0012486
11.05.2020 10:28 stgatilov Note Added: 0012487
11.05.2020 10:29 stgatilov Note Added: 0012488
11.05.2020 10:40 cabalistic Note Added: 0012489
11.05.2020 11:10 stgatilov Note Added: 0012490
11.05.2020 11:17 cabalistic Note Added: 0012491
02.07.2020 17:30 stgatilov Note Added: 0012644
09.07.2020 16:49 stgatilov Note Added: 0012647
19.07.2020 12:53 stgatilov Note Added: 0012661
19.07.2020 12:53 stgatilov Note Edited: 0012661 View Revisions