Is TCL's philosophy of minimalism going the wrong way!?

[CNK]

Thanks for that Rich, based on that size I should be able to set up a mirror for old versions myself on free Oracle Cloud storage, and maybe someone else can upload the old files that have gone offline to it. I'm already in the middle of something at the moment, but I'll see if I can get something set up this week (probably optimistic).

[GNUser]

Hi Rich.

In case someone is interested in the storage required for this:

Code: [Select]

tc@E310:/mnt/sdb1$ for C in `seq 1 8`; do du -hs "$C".x/; done
2.3G    1.x/
8.4G    2.x/
14G     3.x/
26G     4.x/
9.1G    5.x/
13G     6.x/
15G     7.x/
16G     8.x/
tc@E310:/mnt/sdb1$ calc 2.3+8.4+14+26+9.1+13+15+16
103.8
tc@E310:/mnt/sdb1$

Why is TCL4 such an outlier? It would be interesting to know the history behind that.

[gadget42]

Hi Rich.
...
Why is TCL4 such an outlier? It would be interesting to know the history behind that.

yes, i was wondering the same thing but hadn't posed the question yet. Thanks @GNUser!

[Rich]

Hi GNUser
I would say there was a lot more participation when it
came to extension submissions.

Extension counts:
TC4=4268    TC8=2431

TC4 top 25 submitters:

Code: [Select]

   1089 Arslan S.
    404 bmarkus
    334 juanito
    305 Jason W
    293 Curaga
    144 robc
    125 gutmensch
     78 aus9
     77 AbNoRMiS
     67 vinnie
     64 TaoTePuh
     60 solorin
     59 jls_legalize <jlslegalize@gmail.com>
     53 jlslegalize@gmail.com
     47 Daniel Barnes
     44 AmatCoder
     41 andriscom
     37 juanito, Arslan S.
     37 jls_legalize (jlslegalize at gmail dot com)
     36 jls_legalize (unsenepopiu at tin dot it)
     35 AlabamaPaul
     25 jls_legalize <unsenepopiu at tin dot it>
     25 dentonlt
     23 SvOlli (using tcbuild
     21 Kingdomcome
TC8 top 25 submitters:

Code: [Select]

    618 bmarkus
    517 juanito
    202 Curaga
    134 Arslan S.
    106 Jason W
     88 andyj
     44 gutmensch
     34 madRat
     31 justincb
     29 aus9
     28 jlslegalize@gmail.com
     25 robc
     24 polikuo
     21 gordon64
     18 jls_legalize <jlslegalize@gmail.com>
     17 gordons64
     17 coreplayer2
     17 Onyarian
     16 roberts
     16 rhermsen
     14 dentonlt
     13 oeai
     13 malbo
     12 Misalf
     12 Daniel Barnes
Individual extensions appeared to be larger.

TC top 25 extensions by size:

Code: [Select]

-rw-r--r-- 1 tc staff  441M Oct 22  2011 /flightgear-data.tcz
-rw-r--r-- 1 tc staff  159M Mar 20  2011 /libreoffice.tcz
-rw-r--r-- 1 tc staff  152M Sep 22  2010 /openoffice3.tcz
-rw-r--r-- 1 tc staff  152M Jun  9  2012 /gcompris.tcz
-rw-r--r-- 1 tc staff  131M Jan 13  2013 /fluidsynth-soundfont.tcz
-rw-r--r-- 1 tc staff  128M Apr 29  2013 /bullet-docs.tcz
-rw-r--r-- 1 tc staff  122M Oct  3  2011 /hedgewars.tcz
-rw-r--r-- 1 tc staff  122M Apr 29  2010 /oo2.tcz
-rw-r--r-- 1 tc staff  110M Dec 10  2011 /kdeartwork.tcz
-rw-r--r-- 1 tc staff  103M Apr 13  2013 /lazarus.tcz
-rw-r--r-- 1 tc staff   97M Jun  3  2010 /wormux.tcz
-rw-r--r-- 1 tc staff   81M May  9  2010 /openclipart-svg.tcz
-rw-r--r-- 1 tc staff   74M Apr 29  2010 /openlierox.tcz
-rw-r--r-- 1 tc staff   65M May  7  2011 /blobwars.tcz
-rw-r--r-- 1 tc staff   65M Jun 13  2011 /sun-jdk.tcz
-rw-r--r-- 1 tc staff   65M Aug 26  2012 /qt-4.x-htmldoc.tcz
-rw-r--r-- 1 tc staff   63M Mar  5  2011 /kdeedu.tcz
-rw-r--r-- 1 tc staff   60M Dec 10  2011 /kdegames.tcz
-rw-r--r-- 1 tc staff   57M Oct  8  2011 /plt.tcz
-rw-r--r-- 1 tc staff   55M Aug 26  2012 /qt-4.x-doc.tcz
-rw-r--r-- 1 tc staff   51M Nov 25  2010 /erlang.tcz
-rw-r--r-- 1 tc staff   51M Jul  3  2012 /ati-fglrx.tcz
-rw-r--r-- 1 tc staff   49M Jun 13  2011 /sun-jre.tcz
-rw-r--r-- 1 tc staff   48M Dec 10  2011 /kde-wallpapers.tcz
-rw-r--r-- 1 tc staff   47M Apr 13  2013 /fpc.tcz
TC8 top 25 extensions by size:

Code: [Select]

-rw-r--r-- 1 tc staff  137M Jun 10  2017 /libreoffice.tcz
-rw-r--r-- 1 tc staff  123M Mar  7  2017 /lazarus.tcz
-rw-r--r-- 1 tc staff   81M Mar  7  2017 /mono.tcz
-rw-r--r-- 1 tc staff   77M Mar  7  2017 /qt4-htmldoc.tcz
-rw-r--r-- 1 tc staff   65M Jun  5  2017 /firefox-ESR.tcz
-rw-r--r-- 1 tc staff   65M Sep 16  2017 /clang.tcz
-rw-r--r-- 1 tc staff   60M Mar  7  2017 /openjdk-8-jdk.tcz
-rw-r--r-- 1 tc staff   57M Mar  7  2017 /qt4-doc.tcz
-rw-r--r-- 1 tc staff   54M Mar  7  2017 /seamonkey.tcz
-rw-r--r-- 1 tc staff   50M Mar  7  2017 /fpc-src.tcz
-rw-r--r-- 1 tc staff   49M Nov  8  2016 /qemu-all.tcz
-rw-r--r-- 1 tc staff   47M Jun  5  2017 /wine-gecko.tcz
-rw-r--r-- 1 tc staff   45M Mar  7  2017 /sun-jre7.tcz
-rw-r--r-- 1 tc staff   44M Jun  5  2017 /wine-mono.tcz
-rw-r--r-- 1 tc staff   44M Mar  7  2017 /chromium-browser.tcz
-rw-r--r-- 1 tc staff   43M Dec  1  2017 /nvidia-384.90-4.8.17-tinycore.tcz
-rw-r--r-- 1 tc staff   43M Mar  7  2017 /openjdk-8-jre.tcz
-rw-r--r-- 1 tc staff   38M Mar  7  2017 /openjdk-7-jre.tcz
-rw-r--r-- 1 tc staff   37M Dec  1  2017 /nvidia-384.90-cuda-4.8.17-tinycore.tcz
-rw-r--r-- 1 tc staff   37M Mar  7  2017 /fpc.tcz
-rw-r--r-- 1 tc staff   37M Mar  7  2017 /db-doc.tcz
-rw-r--r-- 1 tc staff   37M Mar  7  2017 /go.tcz
-rw-r--r-- 1 tc staff   35M Jun 10  2017 /libreoffice_lang.tcz
-rw-r--r-- 1 tc staff   35M Mar  7  2017 /samba3.tcz
-rw-r--r-- 1 tc staff   34M Apr 19  2017 /wine.tcz

[CentralWare]

Final Diagnosis:

@CNK suggests uploading 2.x through x.x to archive.org.  This is a painfully slow process (though if arrangements were made, a physical drive could probably be mailed to them and possibly someday that content would become available through their system) but thus far testing their history with repo.tinycore and distro.ibiblio.org/tinycore none of the actual binary content (.tcz files, for example) looks to have ever been crawled and saved.  This IS, however, the closest to a drop-in replacement where /opt/tcemirror could possibly be updated to utilize if necessary.  Additionally, there's really no telling whether or not archive.org is going to stand the test of time.

GitHub - this direction is much, much faster than Archive.org, SEEMS so far to be unlimited in space and speed, but isn't overly drop-in friendly "as is"

Google Drive(s) - again, much faster than Archive, but not drop-in friendly at all.

Free web hosting services - so far I have not found a service with the amount of space and speed needed to make it worthwhile

PLAN "F" - THE OTHER: Don't hold your breath yet as I am not in a position yet to make promises, BUT I have one effort in mind that might make a lot of this (including NEW releases) so much more lean and efficient while utilizing third party systems such as GitHub by merely making a few CORE logic changes with the tce* files.  This will take a bit to experiment with.

[CentralWare]

PLAN F : INQUIRY
@Rich @Curaga and crew, is there any reason you guys can think of as to why Ibiblio doesn't mirror the "entire" release/src content?

Example: http://repo.tinycorelinux.net/14.x/aarch64/releases/RPi/src/busybox/
Versus: https://distro.ibiblio.org/tinycorelinux/14.x/aarch64/releases/RPi/src/busybox/ (empty)
../toolchain is wonky, too

Now that I'm aware of this I can adjust our mirror accordingly (ours is empty, too - domino effect) but I figured I'd ask

[GNUser]

Hi GNUser
I would say there was a lot more participation when it
came to extension submissions.

Extension counts:
TC4=4268    TC8=2431

Hi Rich. Thank your for piecing this together. Yes, more numerous contributors sending in more and bigger extensions would certainly explain why TC4 was such an outlier.

TCL is still here and doing fine, so it obviously still has a critical mass of developer/contributor manpower.

Even though TC4 may represent a peak in extension submissions, my prediction is the future will bring similar or taller peaks. As other distros become increasingly complex, TCL will become increasingly appealing to users who desire to understand and control their OS.

[CNK]

@CNK suggests uploading 2.x through x.x to archive.org.  This is a painfully slow process

I'm not sure what you mean by this. Slow servers? Or I've seen people reference the clunky uploader interface but others seem to get around that by uploading ZIPs which get expanded into directories (example - bigger ZIP links that I know are still broken).

but thus far testing their history with repo.tinycore and distro.ibiblio.org/tinycore none of the actual binary content (.tcz files, for example) looks to have ever been crawled and saved.

That's their Wayback Machine project, which often doesn't save everything. Submitted items are different, eg. people submitting 66GB archives of MAME ROMs.

This IS, however, the closest to a drop-in replacement where /opt/tcemirror could possibly be updated to utilize if necessary.

That's the whole point of it to me. With GitHub, Google Drive, etc. no good for using as repo settings in TC itself, I don't see how uploading them there achieves much.

Free web hosting services - so far I have not found a service with the amount of space and speed needed to make it worthwhile

Oracle Cloud can store 200GB for free. I've already started setting up a VPS there but I'm picky about avoiding Systemd and am battling to get Devuan installed there at the moment - three 'bricked' virtual servers and counting... Real hardware is so much easier.

[curaga]

PLAN F : INQUIRY
@Rich @Curaga and crew, is there any reason you guys can think of as to why Ibiblio doesn't mirror the "entire" release/src content?

Example: http://repo.tinycorelinux.net/14.x/aarch64/releases/RPi/src/busybox/
Versus: https://distro.ibiblio.org/tinycorelinux/14.x/aarch64/releases/RPi/src/busybox/ (empty)
../toolchain is wonky, too

Now that I'm aware of this I can adjust our mirror accordingly (ours is empty, too - domino effect) but I figured I'd ask

This dir appears to contain absolute symlinks, when they should be relative. Paul?

[gadget42]

does anyone have or have_access_to ibiblio stats regarding repository activity?

for example, traffic/downloads/etc at this directory level(and perhaps below)?
https://distro.ibiblio.org/tinycorelinux/4.x/x86/archive/

if this data is available it would give a better understanding of the necessity or lack_thereof of more specific offerings.

also, how are/were releases 1.x/2.x/3.x removed from ibiblio? does this happen automatically when they are removed from http://repo.tinycorelinux.net/downloads.html ?

[Rich]

Hi gadget42

... also, how are/were releases 1.x/2.x/3.x removed from ibiblio? does this happen automatically when they are removed from http://repo.tinycorelinux.net/downloads.html ?

Maybe ibiblio was mirroring our repo?

If mirroring is a literal term, then what happens on their site would
be a reflection of what happened on our site:
We add a file, they add that file.
We update a file, they update that file.
We remove a directory, they remove that directory.

I'm not involved in this aspect, so this is just guesswork on my part.

[Paul_123]

This dir appears to contain absolute symlinks, when they should be relative. Paul?

Not intentional

[CentralWare]

@Rich: The Ibiblio project was likely signed up for quite some time ago (very likely they're found in mirrors.tcz) and it's one we found to be fast and frequently updated with TCL and a number of other useful repositories so we tend to lean on them instead of repo.tinycore and other master resources, but yes, when we add/remove something on our repo.tinycorelinux.net server, they'll reflect those changes usually within 24 hours or so, many times much less than that it seemed.

@Curaga @Paul_123: We just found something odd and passed it along; I have not tested 15x as 14x was our focus group for a server test, being seen a little more as "tried and tested."  If memory serves, 13.x only has KERNEL source; busybox and toolchain don't exist on repo - how far back I don't know; that was Béla's wheel house back then if I'm not mistaken and may just not be posted or may be elsewhere.

@CNK:

I'm not sure what you mean by this. Slow servers?

Yes, very slow both for uploading and downloading.  Fear not, though...  a functional method will present itself in the end!
Oracle Cloud's 200GB of "Block Volume Storage" does not sound, from what I just read, as though it would be useful for drop-in functionality (ie: could be made to work with apps or tce-load) BUT you also note VPS and SystemD so maybe there's a web based functionality I didn't see in their Always Free tier.

[CNK]

@CNK:

I'm not sure what you mean by this. Slow servers?

Yes, very slow both for uploading and downloading.  Fear not, though...  a functional method will present itself in the end!
Oracle Cloud's 200GB of "Block Volume Storage" does not sound, from what I just read, as though it would be useful for drop-in functionality (ie: could be made to work with apps or tce-load) BUT you also note VPS and SystemD so maybe there's a web based functionality I didn't see in their Always Free tier.

Ah right, well I have downloaded a lot from the Internet Archive but my internet's so slow usually, especially from overseas servers, that it doesn't make much difference. Someone might have uploaded the TC3 files to it at Tiny Core Linux ver. 3.x Full Archive, but since that's offline I'm just going from the description that was in a web search result: "Its contains all Tiny Core v 3.x data files - ISO, repository, source files, etc."

As for Oracle Cloud, the block storage allows you to create a drive that is used by a VPS, but there are quite a few hoops to jump through. Anyway I gave up on avoiding Systemd (I think the cloud-init system it uses requires Systemd), it's now stock Debian and downloading TC 4 - 10 extension repos via Rsync. When it's done I'll set up a way for someone here to (hopefully) upload the repos that are offline.

[patrikg]

Do you gain some size of compressing the files (like iso) before uploading them?
Archiving files can be stored compressed.