Proposal: extension name include package version

[jazzbiker]

Hi Core!

Some extensions already use this convention. If all extensions including the system libraries will follow it then in future TinyCore will withstand the coming dependency hell easier. Packages and their dependency trees are growing, thriving and evolving.

This proposal has the subproposal part concerning the *-dev.tcz directories naming. I have a dream, that dev files will be placed in the manner, which will not overlap with the next version of the same package dev files.

As an example let's take readline-dev.tcz. Now I'm in TC10 x86. readline's version is 7.0.
readline-dev.tcz.list:

Code: [Select]

/usr/local/include/readline/keymaps.h
/usr/local/include/readline/readline.h
/usr/local/include/readline/rlconf.h
/usr/local/include/readline/rlstdc.h
/usr/local/include/readline/rltypedefs.h
/usr/local/include/readline/tilde.h

So in TC13 for example the dev files will conflict with the TC13's 8 version of readline-dev.tcz.

The library extension may be renamed and used in newer versions. But not the dev files because they will overlap with the native version dev files. It means that in order to use some package linked against one of the previous versions I must build it inside the previous TC version, which may lack some other packages present in the newer TC version.

Returning to the readline-dev.tcz example, if readline-dev.tcz will be named as readline7-dev.tcz and its files will be located in a slightly different way:

Code: [Select]

/usr/local/include/readline7/keymaps.h
/usr/local/include/readline7/readline.h
/usr/local/include/readline7/rlconf.h
/usr/local/include/readline7/rlstdc.h
/usr/local/include/readline7/rltypedefs.h
/usr/local/include/readline7/tilde.h
then it gives much more freedom for building dependent extensions.

This approach is implemented in the certain extensions (lua-dev, guile-dev as an examples) and I think it is not a kind of revolution if the system libs will follow it too. It will simply make TinyCore more convenient and developer friendly.

Have a nice Core!

[GNUser]

Returning to the readline-dev.tcz example, if readline-dev.tcz will be named as readline7-dev.tcz and its files will be located in a slightly different way:

Code: [Select]

/usr/local/include/readline7/keymaps.h
/usr/local/include/readline7/readline.h
/usr/local/include/readline7/rlconf.h
/usr/local/include/readline7/rlstdc.h
/usr/local/include/readline7/rltypedefs.h
/usr/local/include/readline7/tilde.h
then it gives much more freedom for building dependent extensions.

Hi jazzbiker. The nix package manager does this to an extreme--every library and application is installed into its own directory with a unique name. It allows you to install multiple versions of any application and any library, if you wish. You may find the project interesting.

The package manager is distro-agnostic and can be used alongside the native package manager on any distro. I don't use it on TCL because nix assumes a traditional installation with persistent filesystem, but I used it with good results alongside apt back in my Devuan days.

[jazzbiker]

Hi GNUser. Yes, I've heard of Nix and Guix. There are several threads on the forum touching the topic, but none of them seems to include reports about successful integration none of those package managers with TC. I never used none of these package managers and can say nothing why it is not possible to use them with TinyCore, though I suspect that if these package managers not yet are built for TinyCore there exist some significant complications to do it.

[CNK]

I don't like version numbers in extensions at all. If I write a script that loads/downloads/updates readline.tcz, I don't want to have to change the script from loading readline7.tcz to readline8.tcz to readline9.tcz to readline9.7.tcz to readline10.3.20280734.tcz, as TC versions and readline versions advance over the years. I also don't want to have to pick from ten different versioned readline packages in the extension manager.

I definately don't want to check all the time whether updates for programs are now applied to a new extension with the next version's name. Right now I've got ffmpeg4.tcz installed and have no reason not to upgrade to the latest version with an extension. Is there a ffmpeg5.tcz extension which I should switch to now? I don't know, but I've had to check before, and that's a waste of time for me. It also means that the automatically-generated "ondemand" script doesn't work because the ffmpeg executable isn't named ffmpeg4 (and renaming that executable would be an even worse idea!).

Some extensions are likely version-named for good reasons, where a compatibility issue means that both might be desired on the current version of TC. Renaming all of the extensions just for people cross-compiling for older TC versions is silly. If you want to do that, figure out your own solution. I suggest keeping TC10 extensions in a separate directory and manually loading them or symlinking them into tce/optional. Maybe keep extensions all in separate "TC(version)" directories and symlink all desired extensions to tce/optional as required. Or write your own version of tce-load. There are lots of ways to fix this problem at your end (more with Tiny Core than with most distros) without making other people's Tiny Core experience worse.

Yes, I've heard of Nix and Guix. There are several threads on the forum touching the topic, but none of them seems to include reports about successful integration none of those package managers with TC. I never used none of these package managers and can say nothing why it is not possible to use them with TinyCore, though I suspect that if these package managers not yet are built for TinyCore there exist some significant complications to do it.

Nix hasn't interested me particularly, but last night I tried installing Pkgsrc on TC14 (x86_64). With /usr/pkg and /usr/pkgsrc mounted to persistant storage using bind. It didn't work, with the bootstrap script getting stuck looking for /usr/pkg/bin/nawk (which it seems to have been supposed to create itself, but didn't, and symlinking to /usr/local/bin/gawk didn't work either). I tried various options to the bootstrap script, but always got the same result.

This morning though, I remembered a pre-built bootstrap tar.gz that's available for use in RHEL 7, so I'll try that when I get the chance.

[jazzbiker]

Hi CNK!

Thanks for Your answer.

You wrote:
> I don't like version numbers in extensions at all.
I damn agree with You! Especially the versions of packages. I don't understand why they can not write the software and don't touch it again and again pushing twisted updates? Why not to think well before starting to write the software? Eight versions of the readline. Are You kidding me? What for the software is written with errors? I am serious by the way. But this is not the way of thinking of those who make this not-yet-finished-cool-buggy package1.x.y as the dependence of another cool-but-buggy-and-not-yet-ready package2.u.w. Riding the bike with the square wheels may not be named comfortable. Better to say it may be named comfortable, but it is not definitely comfortable as is.

You wrote:
> I definately don't want to check all the time whether updates for programs are now applied to a new extension with the next version's name.
I damn agree with You twice!

You wrote:
> Some extensions are likely version-named for good reasons, where a compatibility issue means that both might be desired on the current version of TC.
Oh no ... Why versions? What are these versions? I don't want to know anything about it.

I was a little bit ironic, this was intentional to show that I totally agree with You when You are describing  what You want, like, prefer etc. And TinyCore developers as I can understand try to make the best to simplify the usage of TC. But even they are forced to keep the different versions of the same package in every release directory.

You wrote:
> Or write your own version of tce-load. There are lots of ways to fix this problem at your end (more with Tiny Core than with most distros) without making other people's Tiny Core experience worse.

Ok, got it. This is exactly the way I do. Sorry, if hypothetical appliance of the proposal in the topic name will  really destroy the people's experience please ignore this thread. Unfortunately their experience is already partially destroyed by several versioned extensions.

Have a nice Core!

[jazzbiker]

Hi Rich!

Can You please remove the topic? I am already not able to do it :-(

[GNUser]

Are You kidding me? What for the software is written with errors? I am serious by the way. But this is not the way of thinking of those who make this not-yet-finished-cool-buggy package1.x.y as the dependence of another cool-but-buggy-and-not-yet-ready package2.u.w.

Hi jazzbiker. I feel your pain. In my opinion, it is a waste of user's time to chase "upgrades" when user's needs have not changed, underlying protocol has not changed, and current version works well enough.

Sorry if my previous recommendation (nix) was too abstract and ill-suited for TCL. I'll try to redeem myself by making a practical recommendation that promotes sanity and integrates seamlessly with TCL: AppImage.

I've been using an AppImage of Thunderbird 45.4.0 since 2016 with success, for example. I also use AppImages for gimp, libreoffice, and imagemagick, among others. Basically, if an application is complex (i.e., has GUI and/or vast dependency tree) and deals with very mature protocols or filetypes, then I'll use an AppImage for it.

An AppImage is a squashfs of an application plus all its dependencies, coupled with a little C utility that mounts the squashfs (using fuse) and launches the application. The main downside of AppImages is their large size (e.g., 161 MB for gimp, 268 MB for libreoffice), but that is the price to pay for bundling the entire dependency tree.

Call me boring and middle-aged, but I have zero interest in upgrading large applications unless I absolutely have to (no pros coupled with high probability of new bugs or dependency hell). AppImages allow me to use a frozen-in-time version of an application for as long as I want to, on any distro including TCL.

Have a nice Core (sprinkled with AppImages if it helps maintain sanity)

[GNUser]

I don't understand why they can not write the software and don't touch it again and again pushing twisted updates?

TL;DR version of my previous post: I use AppImages to protect myself from twisted updates

[mocore]

Hi GNUser. Yes, I've heard of Nix and Guix. There are several threads on the forum touching the topic, but none of them seems to include reports about successful integration none of those package managers with TC. I never used none of these package managers and can say nothing why it is not possible to use them with TinyCore, though I suspect that if these package managers not yet are built for TinyCore there exist some significant complications to do it.

i have some success , useing nix (not so recently ) and guix a bit more recently  from pre-buit binary's

so it is possible , but ... afair  the way they are designed/intended to work under their own isolated paths ect
makes packaging these package managers as squashed tcz less desirable (however! they can/could be made to produce tcz of the package's they manage!)
they are more suited to be installed under /opt
and linked bind mounted on to respective  / root paths   

this method worked well for me ,
and i had made some modifications (bind mounting and other tweaks) to the install scripts
to facilitate (hybrid)reinstall under various *live* environment's inc tiny core

[I suspect that if these package managers not yet are built for TinyCore there exist some significant complications to do it.]

they are designed to work on top of existing linux install , but as mentioned above
 need a little convincing to operate on top of fixed ro root fs
basically that just consists of re-doing install initiation on boot (add users ect) and binding the paths from /opt/*whatever* to /*whatever*

i also looked at pkgsrc after playing with bsd but as i never managed to push my guix install modification's as patch didnt want to get to much more distracted 

[Rich]

Hi jazzbiker

Hi Rich!

Can You please remove the topic? I am already not able to do it :-(

Software versioning issues and how to deal with them sounds like
a valid topic to me. I see no reason to remove this thread.

[andyj]

As a maintainer of PHP and PostgreSQL, I use version numbers for the packages because the major new versions introduce breaking changes. But I don't number down to the patch levels because those are meant to be safe from such things. It makes sense that libraries have major numbers when they have a new ABI or significant changes, and the downstream packages look for particular versions in their config files. Yes, it's a pain, but again yes, it matters.

[CNK]

I've been using an AppImage of Thunderbird 45.4.0 since 2016 with success, for example. I also use AppImages for gimp, libreoffice, and imagemagick, among others. Basically, if an application is complex (i.e., has GUI and/or vast dependency tree) and deals with very mature protocols or filetypes, then I'll use an AppImage for it.

An AppImage is a squashfs of an application plus all its dependencies, coupled with a little C utility that mounts the squashfs (using fuse) and launches the application. The main downside of AppImages is their large size (e.g., 161 MB for gimp, 268 MB for libreoffice), but that is the price to pay for bundling the entire dependency tree.

If an AppImage exists that you like, then that's great. So far that's yet to happen for me.

A while ago I wanted to try out a program that required Qt 6, which no distro I'm running has available. I installed a recent Ubuntu on a VPS (so I didn't have to download many things over my limited home internet). First I tried to build a static binary, but it turns out you need a commercial version of Qt for static builds to work (yet another reason why I dislike Qt). So I tried to build an AppImage. Very long (228 lines of notes) story short, it took a hours of manual fiddling with the parameters to linuxdeploy (I also tried appimagetool, but didn't work for me at all) to get something that half worked. I still couldn't get the program to use the AppImage-included OpenSSL 3, so any functions requiring encryption failed because TC14 only has OpenSSL 1.1.1. Which is exactly one of those awkward version problems (OpenSSL seems to be very good at creating them lately) which AppImages are meant to solve!

Many AppImage developer's guides suggest building AppImages on older distros, which wouldn't use new versions of libraries like OpenSSL 3 and therefore only people on even older distros would encounter such problems. But old distros don't have Qt 6 packaged, so it's a catch-22.

Lesson learnt: If nobody's made an AppImage already, it's probably because it's just too damn hard to do so.

Static binaries are still better than AppImages in my optinion. I use them for pandoc and fsarchiver. But they're not always possible to build for complex software.

Pkgsrc isn't going much better. I wanted to use the binary packages available for either RHEL 7 or Slackware on x86_64, but binaries from either of those sources won't run on TC 14 Pure64. The dynamic linker (/lib/ld-linux-x86-64.so.2) doesn't seem to like them. With the RHEL 7 ones it says it can't find a shared object file, but the name of the shared object is blank and I don't see anything missing with "ldd". With the Slackware ones, both /lib/ld-linux-x86-64.so.2 and ldd return "Bus error".

So the only option is compiling from source using pkgsrc's automated system, and so far that's proving slow and error-prone. Probably no easier than compiling manually and building TC extensions. Certainly not a quick way to try out newly discovered software and see if it's useful, which is what I was looking for.

So my only answer is static binaries, and when they're not an option it seems it's just tough luck!

[aus9]

CNK writes

I use them for pandoc and fsarchiver

I used to use Knoppix and partimage. Now for my Debian based distro I use fsarchiver with SystemRescue
formerly called SystemRescuecd

Saved my bacon more than once. So am free to make more mistakes 

[GNUser]

Hi CNK. You made many good points. I'd like to share a few comments:

If an AppImage exists that you like, then that's great. So far that's yet to happen for me.

I agree that a lot of the time there is no AppImage available for the application I need. Sometimes when this happens I can build one myself. Depending on the application, building an AppImage can range from trivial to impossible.

Many AppImage developer's guides suggest building AppImages on older distros

Yes, because of the awful fact that glibc is not forwards compatible (i.e., AppImage built on a system with old glibc will work on systems with newer glibc, but AppImage built on system with new gllibc will not work on systems with older glibc).

So my only answer is static binaries, and when they're not an option it seems it's just tough luck!

At a high conceptual level, static binaries and AppImages are similar in that both consist of an application and the libraries it needs. Main difference is that in one case the libraries are statically linked, in the other they are bundled and use LD_LIBRARY_PATH hack. Both will save the user from being forced to upgrade an application because of dependency hell (i.e., a dependency no longer exists in the repository or has been upgraded to a new major version that breaks ABI compatibility with the application).

ABI-incompatible library upgrades create a situation with no ideal solution. Traditional (including TCL) solutions are a) the new library version replaces the old version in the repository and applications have to be rebuilt and b) old version and new version have to coexist in the repository under different names. Non-traditional solutions include nix-style packaging, AppImages, and static binaries. All the solutions come at a price--added complexity, added size, missed security updates, or all the above

[CNK]

@aus9: Yes I was using SystemRescueCD for fsarchiver too until they dropped x86 support. I still keep the last x86 release handy.

@GNUuser: Well said, and all agreed. Good to hear that AppImages sometimes can turn out to be trivial to create even when nobody's published one already.

Musl seems to be used sometimes instead of glibc for making better-compatible static binaries.

On AppImages vs static binaries. One difference is that a static binary only includes the parts from libraries that are actually used by the application. AppImages include the whole libraries, so you end up keeping a lot of code that will never be run. So I prefer static binaries on that principle. But both are indeed a dodgy work-around for the fundamental underlying problem of changing ABIs.