 |
I'm currently working on fixing this. I came with an idea that I'm testing. |
|
|
Here's "ThirdParty" fixed. It's made upon trunk. I have tested it allows building stable. It fixes many limitations the previous edition had. Principles: - Make the components loosely coupled. - Delegate as much as possible upstream. - Always updated versions to reduce the livelihood of bugs. - Make it obvious, so people would do it more often. - Automate, so people don't need to learn how to do it. So in this new edition: - It always uses the latest version of dependencies as far as possible, hence the recipes upstream are more likely to not have breakages. Exception is zlib (manipulated internally), ffmpeg (different API) and any other to be deprecated (waste of time). - Many packages that were only previously available on Bincrafters, or only in the custom dir, are now in the official Conan Center. Hence sourcing the recipes from there. - Upgraded all the custom recipes to the latest version. Documented how to upgrade them in the future in a reliable manner. - Provided instructions on how to control when to deprecate packages, so generated artefacts are compatible with both trunk and stable. |
 |
Thanks! I'll look at the changes after 2.10. I'm strongly against using "latest version" of any package. I had a bit of experience in web frontend development, when everyone tries to use the latest version, and as the result you cannot build a revision which is a few months old. Right now library versions are fixed, and I'll probably fix recipe versions too. Especially after they decided to throw away minizip option from the latest zlib recipe, breaking package users. I'm not sure why we need anything like "deprecation". Ideally, every SVN revision should be built with artefacts it provides, and ideally, these artefacts can be rebuilt from sources/recipes too. I left DevIL package in "_unused" directory in case we need to return it back, and I'll delete it in future. So latest stable artefacts and recipes should be in corresponding branch. Is the problem about all branches hidden from public? I think that's stupid policy which we should drop... Prefer conan-center to bincrafters --- yes, definitely agree here. To honest, I started to dislike bincrafters now, since one has to put two custom commands (which are hard to find) in order to make it work. What kind of automation did you miss? Do you mean some excessive questions in 1_export_custom.py? I think I added them because I thought it's better to use local conan cache, but I guess it's perfectly OK to use system cache. And without CONAN_USER_HOME these questions can be removed. |
|
|
=== VERSIONS === In this particular scenario, observing what has happened till now, I estimate that using up to date versions will make things less likely to break because: - My experience is having written recipes for over 50 packages. Never happened to me that using very old versions was less prone to breakages. If it is old, it rots. - Recipes for old versions are deprecated upstream. - Hence more likely to need a local copy, which people eventually avoid to maintain. - Recipes for old versions are more likely to have bugs. - In an compiled language errors are caught during compilation, not like in front-end JavaScript. - If something doesn't build, you can easily choose any previous version. === DEPRECATION === Ideally, every SVN revision should be built with artefacts it provides, and ideally, these artefacts can be rebuilt from sources/recipes too. But the fact is: that's not what scientifically has happened till now. Recipes broke multiple times, and fixing them in multiple places was required. Then I wasn't able to rebuild those artefact for stable in all those situations, for a long time, while in trunk it was fixed. Some of those frequent breakages fall outside the scope of package management. They simply happen because a dependency changes behavior. For example, when GCC changed behavior and pulseaudio wasn't building anymore. This will continue to happen. Better to arrange things in a way that, when change come again, it is manageable. === BINCRAFTERS === Some packages can still only be retrieved from Bincrafters. === EXPORT CUSTOM === If someone wants to see what recipes are going to be exported, they can just have a look inside the folder "custom_recipes". Also they can still customize its conan_user_home, same as before. They just aren't remembered what it is because, well, you just set it. You already know. In that last case maybe showing it prevented mistakes in some cases. But there's a better way for that: === AUTOMATION === I left this one out for the moment. This means turning the building instructions into code (build.py), so there's nothing to learn, no mistakes to make, and no time to waste! |
|
|
In this particular case I see two things happened: 1) They "updated" some recipes without changing package version, breaking their users. 2) They dramatically changed their server infrastructure and package naming, throwing away some recipes in process. Depending on particular version of recipe would have the first point avoided. Copying all recipes and using local copies for everything would have the second point avoided. Compilers do change, but slower than conan in this case. After all, we can't guarantee TDM will build with any future compiler anyway. In principle, you can even install old GCC if newer version does not work. |
|
|
If what you are trying to say is that what they did is bullshit, I should say it is totally true :D There's a reason why x86 is the de facto computer architecture these days: stability. It seems that most software developers didn't get the message. You never modify behavior of an existing interface. If you need to change something you do by extending only, and it's rare that you need more than that. What remembers me of something, what applies to software development in general: https://www.youtube.com/watch?v=Pzl1B7nB9Kc |
|
|
I'm looking at recipe update right now, and looking at your changes. Why did you make 1_export_custom.py to work without any user input, but did not do the same to 2_build_all.py ? |
|
|
Doug McIlroy once said: "Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input." I didn't apply the same principle to "2_build_all.py" simply because I haven't worked on it. |
|
|
First of all, I changed a bit how conan is used: r9861. Extracted list of packages and settings to packages.yaml, use conan profiles to build libs for supported archs. Now the list of packages and settings are stored in separate YAML file. The way platforms are specified has been changed: now they are set in conan profile files (does not affect "platform_name" approach), Also, I added shebang and --unattended mode to both Python scripts. Next, I fixed some immediate problems that locked recipes from working: r9862. ffmpeg: use "msys2/cci.latest" instead of "msys2_installer/latest@bincrafters/stable" because the latter stopped working. r9863. Removed no-longer-used DevIL conan package. After that I went through all packages one-by-one, and tried to update them as far as possible, and clean as much custom mess as I can: r9864. vorbis package updated from 1.3.6 on bincrafters to 1.3.7 on conan-center. r9865. pugixml package updated from 1.9 on bincrafters to 1.12.1 on conan-center. r9866. doctest package updated from 2.2.3 with local modifications to 2.4.8 on conan-center. r9867. libjpeg package updated from 9c with local modifications to 9d from conan-center. r9868. BLAKE2 recipe converted to versionless format, use 20190724 tag version instead of git master. r9869. tinyformat package convertes to versionless format, version bumped from 2.1.0 to 2.3.0. r9870. tracy package converted to versionless format. r9871. openal package converted to versionless format. r9872. libpng package updated from 1.6.34 on bincrafters to 1.6.37 on conan-center. r9873. zlib+minizip package replaced with two separate packages. r9874. fltk package updated from 1.3.5 to 1.3.8. r9875. glfw package updated from 3.3.4 to 3.3.6 from conan-center, local modifications removed. r9876. libcurl+mbedtls packages updated from 7.61.1 + 2.13.0 to 7.80.0 + 2.25.0. The greatest change was in zlib/minizip package(s): this one indirectly influences all the projects we have on all platforms. FLTK was not fun either: the previous recipe was done from scratch by me, the new one is from conan-center, so I had to find and apply some tweaks. One thing to note is that now recipes in what-I-call "version-less" format are popular. That's when conan recipe does not contain particular version of the library, and often applies to many versions at once; the version-specific data is stored in separate conandata.yml file. I converted all custom recipes to this format. Finally, I rebuilt all artefacts: r9877. Rebuilt ALL third-party artefacts afresh after recent major changes. The Linux artefacts are still built on GCC 5.4.0. Windows artefacts are now built on MSVC2022 now (0005820). The only package that I did not finish (did not even touch) yet is FFmpeg. That's a huge one: I hope to update version and rewrite FFmpeg player with newer API. |
|
|
Updated FFmpeg too: r9880. ffmpeg package updated from 4.0.2 to 4.4.1. r9882. Added forgotten VC props update for svn rev 9880 (FFmpeg lib names). r9881. Rebuilt ffmpeg artefacts on all supported systems. Still has to reimplement the code in TDM which uses the library to avoid deprecated APIs. Right now it gives warning spam on GCC. |
|
|
Refactored cinematics code, fixed all deprecation warnings: r9884. Simple fixes of deprecated FFmpeg functions. r9885. Replaced deprecated AVPacketList with our own equivalent. r9886. Avoid accessing deprecated AVStream::codec. r9887. Use avcodec_send_packet + avcodec_receive_frame instead of deprecated avcodec_decode_video2/avcodec_decode_audio4. Also did a few cinematics fixes unrelated to libs update: r9883 Fixed "testVideo withAudio" command. r9888 Don't pass URL parameter to avformat_open_input. |
|
|
I did everything I wanted, except for locking package revisions. It seems that conan 2 is being released, and it will change a lot regarding how revisions work. For instance, now you have to enable revisions manually, plus you can only have one revision of package in local cache. If you change only the revision of the package (without changing version), then conan will use old package, despite the fact that it has wrong revision. Also, conan 2 will most likely break a lot of packages, and I'll have to review all packages from scratch again. So... better leave it as is until conan 2 becomes widespread. |
|
|
Conan is barbarian. |
|
|
Conan is frogarian. |
|
|
Frogarians are badass. |
|
|
It turned out that libjpeg is broken on Linux, because of the stupid "boolean" type breaking binary compatibility. Here is the fix: r9913. [libjpeg] Forked stock recipe and added tdm-boolean-always-unsigned-char.patch to fix sizeof(boolean). r9914. Rebuilt libjpeg artefacts on all platforms. Explained in detail here: https://forums.thedarkmod.com/index.php?/topic/21334-conan-recipes-update-5881/&do=findComment&comment=473094 |
|
|
Unable to view that link: https://i.imgur.com/CBl2rp3.png |
- Anonymous
