View Issue Details

IDProjectCategoryView StatusLast Update
0005881The Dark ModCodingpublic22.04.2022 12:15
Reporterstgatilov Assigned Tostgatilov  
Status resolvedResolutionfixed 
Product VersionTDM 2.10 
Target VersionTDM 2.11Fixed in VersionTDM 2.11 
Summary0005881: Revise conan dependencies
DescriptionConan has changed a great deal since we started using it.
Packages without "@conan/stable", revisions and lockfiles, different servers, newer packages, more packages in central repo, etc.
It would make sense to recreate/reinclude all dependencies from scratch, updating them and their recipes at the same time.

I should also look at the problem of recipe updates.
Right now if someone updates recipe on the server, it can break TDM build.
We can either specify explicit revision of recipe, or rely on lockfiles somehow.
Additional InformationInternal discussion:
TagsNo tags attached.


related to 0005839 resolvedstgatilov no longer available: zlib/1.2.11 
parent of 0005921 resolvedstgatilov fails to compile: missing file: unzip.h 




01.02.2022 18:36

reporter   ~0014708

I'm currently working on fixing this. I came with an idea that I'm testing.


06.02.2022 15:55

reporter   ~0014713

Here's "ThirdParty" fixed. It's made upon trunk. I have tested it allows building stable.

It fixes many limitations the previous edition had.

- Make the components loosely coupled.
- Delegate as much as possible upstream.
- Always updated versions to reduce the livelihood of bugs.
- Make it obvious, so people would do it more often.
- Automate, so people don't need to learn how to do it.

So in this new edition:

- It always uses the latest version of dependencies as far as possible, hence the recipes upstream are more likely to not have breakages. Exception is zlib (manipulated internally), ffmpeg (different API) and any other to be deprecated (waste of time).

- Many packages that were only previously available on Bincrafters, or only in the custom dir, are now in the official Conan Center. Hence sourcing the recipes from there.

- Upgraded all the custom recipes to the latest version. Documented how to upgrade them in the future in a reliable manner.

- Provided instructions on how to control when to deprecate packages, so generated artefacts are compatible with both trunk and stable. (63,267 bytes)


07.02.2022 15:42

administrator   ~0014714

I'll look at the changes after 2.10.

I'm strongly against using "latest version" of any package.
I had a bit of experience in web frontend development, when everyone tries to use the latest version, and as the result you cannot build a revision which is a few months old.
Right now library versions are fixed, and I'll probably fix recipe versions too. Especially after they decided to throw away minizip option from the latest zlib recipe, breaking package users.

I'm not sure why we need anything like "deprecation".
Ideally, every SVN revision should be built with artefacts it provides, and ideally, these artefacts can be rebuilt from sources/recipes too.
I left DevIL package in "_unused" directory in case we need to return it back, and I'll delete it in future.
So latest stable artefacts and recipes should be in corresponding branch.
Is the problem about all branches hidden from public? I think that's stupid policy which we should drop...

Prefer conan-center to bincrafters --- yes, definitely agree here.
To honest, I started to dislike bincrafters now, since one has to put two custom commands (which are hard to find) in order to make it work.

What kind of automation did you miss?
Do you mean some excessive questions in
I think I added them because I thought it's better to use local conan cache, but I guess it's perfectly OK to use system cache.
And without CONAN_USER_HOME these questions can be removed.


07.02.2022 18:50

reporter   ~0014715

=== VERSIONS ===

In this particular scenario, observing what has happened till now, I estimate that using up to date versions will make things less likely to break because:

- My experience is having written recipes for over 50 packages. Never happened to me that using very old versions was less prone to breakages. If it is old, it rots.
- Recipes for old versions are deprecated upstream.
- Hence more likely to need a local copy, which people eventually avoid to maintain.
- Recipes for old versions are more likely to have bugs.
- In an compiled language errors are caught during compilation, not like in front-end JavaScript.
- If something doesn't build, you can easily choose any previous version.


Ideally, every SVN revision should be built with artefacts it provides, and ideally, these artefacts can be rebuilt from sources/recipes too.

But the fact is: that's not what scientifically has happened till now. Recipes broke multiple times, and fixing them in multiple places was required. Then I wasn't able to rebuild those artefact for stable in all those situations, for a long time, while in trunk it was fixed.

Some of those frequent breakages fall outside the scope of package management. They simply happen because a dependency changes behavior. For example, when GCC changed behavior and pulseaudio wasn't building anymore.

This will continue to happen. Better to arrange things in a way that, when change come again, it is manageable.


Some packages can still only be retrieved from Bincrafters.


If someone wants to see what recipes are going to be exported, they can just have a look inside the folder "custom_recipes".

Also they can still customize its conan_user_home, same as before. They just aren't remembered what it is because, well, you just set it. You already know.

In that last case maybe showing it prevented mistakes in some cases. But there's a better way for that:


I left this one out for the moment. This means turning the building instructions into code (, so there's nothing to learn, no mistakes to make, and no time to waste!


08.02.2022 02:22

administrator   ~0014716

In this particular case I see two things happened:
  1) They "updated" some recipes without changing package version, breaking their users.
  2) They dramatically changed their server infrastructure and package naming, throwing away some recipes in process.
Depending on particular version of recipe would have the first point avoided.
Copying all recipes and using local copies for everything would have the second point avoided.

Compilers do change, but slower than conan in this case.
After all, we can't guarantee TDM will build with any future compiler anyway.
In principle, you can even install old GCC if newer version does not work.


08.02.2022 12:01

reporter   ~0014717

If what you are trying to say is that what they did is bullshit, I should say it is totally true :D

There's a reason why x86 is the de facto computer architecture these days: stability. It seems that most software developers didn't get the message.

You never modify behavior of an existing interface. If you need to change something you do by extending only, and it's rare that you need more than that.

What remembers me of something, what applies to software development in general:


14.03.2022 15:48

administrator   ~0014765

I'm looking at recipe update right now, and looking at your changes.
Why did you make to work without any user input, but did not do the same to ?


15.03.2022 08:13

reporter   ~0014766

Doug McIlroy once said:

"Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input."

I didn't apply the same principle to "" simply because I haven't worked on it.


17.03.2022 17:56

administrator   ~0014769

First of all, I changed a bit how conan is used:
  r9861. Extracted list of packages and settings to packages.yaml, use conan profiles to build libs for supported archs.
Now the list of packages and settings are stored in separate YAML file.
The way platforms are specified has been changed: now they are set in conan profile files (does not affect "platform_name" approach),
Also, I added shebang and --unattended mode to both Python scripts.

Next, I fixed some immediate problems that locked recipes from working:
  r9862. ffmpeg: use "msys2/cci.latest" instead of "msys2_installer/latest@bincrafters/stable" because the latter stopped working.
  r9863. Removed no-longer-used DevIL conan package.

After that I went through all packages one-by-one, and tried to update them as far as possible, and clean as much custom mess as I can:
  r9864. vorbis package updated from 1.3.6 on bincrafters to 1.3.7 on conan-center.
  r9865. pugixml package updated from 1.9 on bincrafters to 1.12.1 on conan-center.
  r9866. doctest package updated from 2.2.3 with local modifications to 2.4.8 on conan-center.
  r9867. libjpeg package updated from 9c with local modifications to 9d from conan-center.
  r9868. BLAKE2 recipe converted to versionless format, use 20190724 tag version instead of git master.
  r9869. tinyformat package convertes to versionless format, version bumped from 2.1.0 to 2.3.0.
  r9870. tracy package converted to versionless format.
  r9871. openal package converted to versionless format.
  r9872. libpng package updated from 1.6.34 on bincrafters to 1.6.37 on conan-center.
  r9873. zlib+minizip package replaced with two separate packages.
  r9874. fltk package updated from 1.3.5 to 1.3.8.
  r9875. glfw package updated from 3.3.4 to 3.3.6 from conan-center, local modifications removed.
  r9876. libcurl+mbedtls packages updated from 7.61.1 + 2.13.0 to 7.80.0 + 2.25.0.
The greatest change was in zlib/minizip package(s): this one indirectly influences all the projects we have on all platforms.
FLTK was not fun either: the previous recipe was done from scratch by me, the new one is from conan-center, so I had to find and apply some tweaks.

One thing to note is that now recipes in what-I-call "version-less" format are popular.
That's when conan recipe does not contain particular version of the library, and often applies to many versions at once; the version-specific data is stored in separate conandata.yml file.
I converted all custom recipes to this format.

Finally, I rebuilt all artefacts:
  r9877. Rebuilt ALL third-party artefacts afresh after recent major changes.
The Linux artefacts are still built on GCC 5.4.0.
Windows artefacts are now built on MSVC2022 now (0005820).

The only package that I did not finish (did not even touch) yet is FFmpeg.
That's a huge one: I hope to update version and rewrite FFmpeg player with newer API.


26.03.2022 15:57

administrator   ~0014778

Last edited: 27.03.2022 10:08

Updated FFmpeg too:
  r9880. ffmpeg package updated from 4.0.2 to 4.4.1.
  r9882. Added forgotten VC props update for svn rev 9880 (FFmpeg lib names).
  r9881. Rebuilt ffmpeg artefacts on all supported systems.

Still has to reimplement the code in TDM which uses the library to avoid deprecated APIs.
Right now it gives warning spam on GCC.


27.03.2022 10:10

administrator   ~0014779

Refactored cinematics code, fixed all deprecation warnings:
  r9884. Simple fixes of deprecated FFmpeg functions.
  r9885. Replaced deprecated AVPacketList with our own equivalent.
  r9886. Avoid accessing deprecated AVStream::codec.
  r9887. Use avcodec_send_packet + avcodec_receive_frame instead of deprecated avcodec_decode_video2/avcodec_decode_audio4.

Also did a few cinematics fixes unrelated to libs update:
  r9883 Fixed "testVideo withAudio" command.
  r9888 Don't pass URL parameter to avformat_open_input.


27.03.2022 10:14

administrator   ~0014780

I did everything I wanted, except for locking package revisions.

It seems that conan 2 is being released, and it will change a lot regarding how revisions work.
For instance, now you have to enable revisions manually, plus you can only have one revision of package in local cache.
If you change only the revision of the package (without changing version), then conan will use old package, despite the fact that it has wrong revision.
Also, conan 2 will most likely break a lot of packages, and I'll have to review all packages from scratch again.

So... better leave it as is until conan 2 becomes widespread.


27.03.2022 11:38

reporter   ~0014781

Conan is barbarian.


27.03.2022 12:09

administrator   ~0014783

Conan is frogarian.


28.03.2022 12:12

reporter   ~0014789

Frogarians are badass.


22.04.2022 04:09

administrator   ~0014809

It turned out that libjpeg is broken on Linux, because of the stupid "boolean" type breaking binary compatibility.

Here is the fix:
  r9913. [libjpeg] Forked stock recipe and added tdm-boolean-always-unsigned-char.patch to fix sizeof(boolean).
  r9914. Rebuilt libjpeg artefacts on all platforms.

Explained in detail here:


22.04.2022 12:15

reporter   ~0014810

Unable to view that link:

Issue History

Date Modified Username Field Change
14.01.2022 01:18 stgatilov New Issue
14.01.2022 01:18 stgatilov Status new => assigned
14.01.2022 01:18 stgatilov Assigned To => stgatilov
14.01.2022 01:18 stgatilov Relationship added related to 0005839
01.02.2022 18:36 es20490446e Note Added: 0014708
06.02.2022 15:55 es20490446e Note Added: 0014713
06.02.2022 15:55 es20490446e File Added:
07.02.2022 15:42 stgatilov Note Added: 0014714
07.02.2022 18:50 es20490446e Note Added: 0014715
08.02.2022 02:22 stgatilov Note Added: 0014716
08.02.2022 12:01 es20490446e Note Added: 0014717
14.03.2022 15:48 stgatilov Note Added: 0014765
15.03.2022 08:13 es20490446e Note Added: 0014766
17.03.2022 17:56 stgatilov Note Added: 0014769
17.03.2022 22:18 nbohr1more Relationship added parent of 0005921
26.03.2022 15:57 stgatilov Note Added: 0014778
27.03.2022 10:08 stgatilov Note Edited: 0014778
27.03.2022 10:10 stgatilov Note Added: 0014779
27.03.2022 10:14 stgatilov Status assigned => resolved
27.03.2022 10:14 stgatilov Resolution open => fixed
27.03.2022 10:14 stgatilov Fixed in Version => TDM 2.11
27.03.2022 10:14 stgatilov Note Added: 0014780
27.03.2022 11:38 es20490446e Note Added: 0014781
27.03.2022 12:09 stgatilov Note Added: 0014783
28.03.2022 03:07 stgatilov Additional Information Updated
28.03.2022 12:12 es20490446e Note Added: 0014789
22.04.2022 04:09 stgatilov Note Added: 0014809
22.04.2022 12:15 es20490446e Note Added: 0014810