Tiny Core Extensions > TCE Talk
New package manager
CNK:
--- Quote ---
--- Quote ---Yes, but I'm not sure why you said Debian's package system is more "server-side" than on TC. Debian also downloads a full package list to search locally, and it downloads the description and dependency info equivalent to the info and dep files for all packages too. Debian do have a web interface that allows searching in a web browser, but apt-get and other package managers I've used there all locally search the downloaded package lists.
--- End quote ---
Hmmm can you point to where this list is? I checked /var/lib/apt and the files in there only contain hashes to (past?) files, or brief info for installed packages. I couldn't find anything where the entire repo's package listing is stored locally.
--- End quote ---
Checking on a Debian system, they are in /var/lib/apt/lists. Clearly all the packages are in the lists, including those not installed. 30MB total for Debian Bullseye.
--- Code: ---$ head /var/lib/apt/lists/deb.debian.org_debian_dists_bullseye-updates_main_binary-arm64_Packages
Package: ca-certificates-java
Version: 20190909+deb11u1
Installed-Size: 43
Maintainer: Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>
Architecture: all
Depends: ca-certificates (>= 20121114), default-jre-headless | java8-runtime-headless, libnss3 (>= 3.12.10-2~)
Description: Common CA certificates (JKS keystore)
Multi-Arch: foreign
Description-md5: 304cd3554728e5d076f8ecbb3b5057d8
Tag: protocol::ssl, role::app-data, security::authentication,
$ aptitude show ca-certificates-java | grep ^State:
State: not installed
--- End code ---
The lists are downloaded from URLs like http://ftp.debian.org/debian/dists/bullseye/main/binary-arm64/Packages.gz
--- Quote ---An online search would be much more efficient in terms of bandwidth and storage, right?
--- End quote ---
Yes but Linux mirror sites only host static content, so instead of having the load spread over various mirrors it would all be on the site hosting the search, and that would be a single point of failure (although I guess a package manager could fall back to downloading the lists the old way from a static mirror if it went down).
Also, keep the sizes in perspective. That tags.db.gz file tce-ab downloads in order to search extension names and tags/keywords is about 30KB. One thread page on these forums is bigger than that. The provides.db.gz file is bigger at 2MB, but it gets cached. The extra server CPU load from getting hit by lots of search requests (these days probably including lots of bots just trying to hack or crash it, like I've experienced on my own website) would probably be more of an issue than transferring that little extra data.
wysiwyg:
Hey CNK, thanks for the reply!
--- Quote from: CNK on January 29, 2026, 04:26:41 PM ---
--- Quote ---
--- Quote ---Yes, but I'm not sure why you said Debian's package system is more "server-side" than on TC. Debian also downloads a full package list to search locally, and it downloads the description and dependency info equivalent to the info and dep files for all packages too. Debian do have a web interface that allows searching in a web browser, but apt-get and other package managers I've used there all locally search the downloaded package lists.
--- End quote ---
Hmmm can you point to where this list is? I checked /var/lib/apt and the files in there only contain hashes to (past?) files, or brief info for installed packages. I couldn't find anything where the entire repo's package listing is stored locally.
--- End quote ---
Checking on a Debian system, they are in /var/lib/apt/lists. Clearly all the packages are in the lists, including those not installed. 30MB total for Debian Bullseye.
--- Code: ---$ head /var/lib/apt/lists/deb.debian.org_debian_dists_bullseye-updates_main_binary-arm64_Packages
Package: ca-certificates-java
Version: 20190909+deb11u1
Installed-Size: 43
Maintainer: Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>
Architecture: all
Depends: ca-certificates (>= 20121114), default-jre-headless | java8-runtime-headless, libnss3 (>= 3.12.10-2~)
Description: Common CA certificates (JKS keystore)
Multi-Arch: foreign
Description-md5: 304cd3554728e5d076f8ecbb3b5057d8
Tag: protocol::ssl, role::app-data, security::authentication,
$ aptitude show ca-certificates-java | grep ^State:
State: not installed
--- End code ---
The lists are downloaded from URLs like http://ftp.debian.org/debian/dists/bullseye/main/binary-arm64/Packages.gz
--- End quote ---
Well, I stand corrected! Oddly enough, I renamed the /var/lib/apt directory to have a .org extension, then ran apt-cache search and it still populated with results. I disconnected from the Internet, and it still gave results! I'm not sure how that was possible if the local database was unreachable, as well as the Internet...
--- Quote from: CNK on January 29, 2026, 04:26:41 PM ---
--- Quote ---An online search would be much more efficient in terms of bandwidth and storage, right?
--- End quote ---
Yes but Linux mirror sites only host static content, so instead of having the load spread over various mirrors it would all be on the site hosting the search, and that would be a single point of failure (although I guess a package manager could fall back to downloading the lists the old way from a static mirror if it went down).
Also, keep the sizes in perspective. That tags.db.gz file tce-ab downloads in order to search extension names and tags/keywords is about 30KB. One thread page on these forums is bigger than that. The provides.db.gz file is bigger at 2MB, but it gets cached. The extra server CPU load from getting hit by lots of search requests (these days probably including lots of bots just trying to hack or crash it, like I've experienced on my own website) would probably be more of an issue than transferring that little extra data.
--- End quote ---
If they only host static content, then yes you are setting up a single point of failure. However, depending on the hosting company, they will have numerous mirrors themselves to prevent just such an issue. Now that does not preclude an issue with the server itself having a problem...
I agree that tags.db.gz file is probably not too big of an issue. But multiply the provides.db.gz by just 100 users and that's already at 200mb. That's a lot of searching that can be done to meet or exceed that using an online script. I'm quite frankly surprised that Debian does this. There's no telling what their bandwidth bill is at the end of every month lol
I guess I'll throw in the ability to process searches this way too. It seems like a giant waste of resources (more so for someone like Debian), but I'm not footing the bill so...
nick65go:
I followed this discussion with interest, tce* scripts are good, no perfect! For me, all is just an intellectual challenge, but not a BIG win in any direction-solution developers will chose to pursuit.
For "modern" hardware (multi-core CPU, SDD or Nvram storage, fast 100MB..1GB network links, etc) is .. not worthy. So the "new" pax package could have a meaningful gain only for outdated surviving hardware like 486 CPU, with small HDD storage, small 128MB RAM, 10MB NIC card etc.
Because, as Rich said 0.2 .. 2 seconds is enough to load all "non-software" repo metadata. Looking into https://mirrors.dotsrc.org/tinycorelinux/17.x/x86_64/tcz/ I see provides.db.gz =3.1 MB, but provides.db.zsync = 175KB!!
1. Using TC on USB, users will not premature destroy their pen/stick with too many updates.
2. Using TC on HDD, will have permanent /tce/optional folder (aka static on HDD, sometime even cache in RAM for a session), and seldom using zsync for 175KB extra.
From my past experience, many TCZ extension are not often updated, because new versions have bigger size. But tiny is about just core (not very useful by itself without apps), not the full system as used by most users. It seams that TC developers think that users initially search for apps, then users settle with some apps and updates seldom.
In summary the idea to use a new package management revolves in replacing wget (from busybox) with SQLite, or maybe CURL+dependencies, by increasing the TINY base/core.
Leee:
--- Quote from: nick65go on February 01, 2026, 05:14:19 PM ---It seams that TC developers think that users initially search for apps, then users settle with some apps and updates seldom.
--- End quote ---
That's pretty much what I do. I have recently started doing occasional extension updates but I haven't exactly made a habit of it.
wysiwyg:
Hey Nick! Thanks for adding to the discussion!
--- Quote from: nick65go on February 01, 2026, 05:14:19 PM ---I followed this discussion with interest, tce* scripts are good, no perfect! For me, all is just an intellectual challenge, but not a BIG win in any direction-solution developers will chose to pursuit.
For "modern" hardware (multi-core CPU, SDD or Nvram storage, fast 100MB..1GB network links, etc) is .. not worthy. So the "new" pax package could have a meaningful gain only for outdated surviving hardware like 486 CPU, with small HDD storage, small 128MB RAM, 10MB NIC card etc.
--- End quote ---
I agree that the TC developers are most likely going to leave the tce-* scripts as is, which is fine. They are a barebones way to accomplish their task. But to say that pax is only meaningful for older hardware, would be incorrect. pax has a much larger feature set than the default tce-* scripts, including one that has been requested for many years - the ability to unload software. I would invite you to read the .info file associated with the project to see the abilities it has over the default manager(s).
http://tinycorelinux.net/16.x/x86/tcz/pax.tcz.info
Which is actually the original version. I sent the latest version at the end of last week that hasn't been processed yet. There isn't much change by way of code, but it does provide an in-depth man page to further explain some of the features. Of course, feedback is always welcome to make it better if something is unclear.
--- Quote from: nick65go on February 01, 2026, 05:14:19 PM ---Because, as Rich said 0.2 .. 2 seconds is enough to load all "non-software" repo metadata. Looking into https://mirrors.dotsrc.org/tinycorelinux/17.x/x86_64/tcz/ I see provides.db.gz =3.1 MB, but provides.db.zsync = 175KB!!
1. Using TC on USB, users will not premature destroy their pen/stick with too many updates.
2. Using TC on HDD, will have permanent /tce/optional folder (aka static on HDD, sometime even cache in RAM for a session), and seldom using zsync for 175KB extra.
From my past experience, many TCZ extension are not often updated, because new versions have bigger size. But tiny is about just core (not very useful by itself without apps), not the full system as used by most users. It seams that TC developers think that users initially search for apps, then users settle with some apps and updates seldom.
--- End quote ---
My original gripe was more so about being efficient with things. I did also think that the way TC was handling it was an exception, not the rule. And yes, 3.1MB is not very much in size. But when 10 people use it, that becomes 31MB, when 100 people use it is becomes 310MB, when 1000 people use it is becomes 3.1GB, etc. And remember that this is donated bandwidth. I'll still add in the ability to search using an online URL, but I'll also have to expand that to include entire repo db's to work across various distros.
--- Quote from: nick65go on February 01, 2026, 05:14:19 PM ---In summary the idea to use a new package management revolves in replacing wget (from busybox) with SQLite, or maybe CURL+dependencies, by increasing the TINY base/core.
--- End quote ---
I'm not sure what advantage that would give you. You would still have to download the entire database to your local device, defeating the efficiency I was aiming for. And the reason wget was chosen is because with basic package management, you only need to download packages, which wget works very well at - plus its part of busybox so no additional dependencies. cURL adds another dependency to the mix, adding extra size to the base - of which TC tries to keep as small as possible. I will attempt to use wget to handle the searching side, but I may have to revert to cURL. But my design goals are different than TC. I do want small and efficient, but not at the expense of traditional features.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version