@mocore: I'm not certain the purpose you're after, but we already have a number of text based files found in the repo itself which may suit your purpose.
Bare in mind they're a part of the repository as opposed to the forum.
tcz/info.lst is basically a file listing of TCZs
tcz/sizelist is the above plus file lengths
tcz/tags.db is a g-zipped version of info plus tag words
tcz/provides.db is similar to an INI file with file listings for each extension
tcz/md5.db is, as you guessed it, the signatures for each file
* most of the items above are available in g-zip form, some are gz only.
to match all current md5 files against the ones in the repository, which generates a LOT of small web server requests to the repository and can take a long time, depending on how many extensions you have,
Title: tdb.tcz
Description: trivial database
Version: 1.2.12
Size: 48KB
Extension_by: juanito
----------
Change-log: first version
Current: 2013/11/21 Anybody can share their experience of tcz alternative-management?My tcz alternative for large applications that I only use rarely (e.g., libreoffice, gimp, qbittorrent, electrum) is AppImage. AppImages work perfectly, load quickly, and add nothing to boot time.
@GNUser: Thanks, good tip. But now you must trust TC and AppImage.Hi nick65go. A few thoughts about this.
But.. you do not use TC applications. I was asking for something like scripts, to search TCZ, for example...
for i in $your_list; do unshashfs -F $i /tmp/x; done. So all tcz into same /tmp/x folder. Your list is seen from cpanel /stat, to see all loop AFTER all mandatory Xorg loops already loaded. Do not forget to populate /usr/local/tclocal with fake null dep and have a start script named like your extension (ex: gnumeric needs this script for gkt-icons/menu/schema init).Info and tree combined files would have less of use, IMHO.Well, in the absence of a good html page (as was with tc9), the contactenation of *.tcz,info into sothing like tcz.info.db.gz will allow to search for few criteria/fields, like Version:, Current: Extension_by: Comments:;
... I think the size will be small, but could you give a size for a tc13_x86 sum(*.tcz.info.) ...The sum of the apparent sizes is 3801251 bytes.
tree combined files would have less of use,
Hi nick65goCorrection:... I think the size will be small, but could you give a size for a tc13_x86 sum(*.tcz.info.) ...The sum of the apparent sizes is 3801251 bytes.
The sum of the apparent sizes for x86 is 1698939 bytes.
The sum of the apparent sizes for x86_64 is 2102312 bytes.
Total 3801251 bytes.we could see how tc evolves (from tc-5 to tc-10 to tc-14 etc).Hi nick65go. I only sync my local mirror to the TCL version I'm currently using (TCL14 x86_64 right now). So unfortunately you will not be able to track evolution of TCL contributors over time with my unofficial solution.
Hi Greg Erskine
I'm not sure I follow what you are looking for. Repositories should always be
mirrors, unless maybe you are running private repository.
Another caveat is that my wife says I have a computer problem and made me promise to skip TCL releases. So I'm stuck on TCL14 until TCL16 comes out. It's either that or the doghouse ;DFortunately we do not have a doghouse, but my better half still has few methods to convince (blackmail) me that computer "work" is not a priority. Some wise person said: "happy wife, happy life" :)
Sorry, you don't have permission to visit this site.
Website blocked as per Company policy.
Not allowed to access this file type
You tried to visit:http://forum.tinycorelinux.net/index.php?action=dlattach;topic=25982.0;attach=6373. Hi nick65go. The script is attached to Reply #22.
Edit:
Here is a direct link to the script for folks who cannot login to download it from Reply #22. It's better to use the link below anyway, as it's easier for me to make any necessary changes to it:
https://gnuser.ddns.net/public/contributor.sh (https://gnuser.ddns.net/public/contributor.sh)
Sorry, you don't have permission to visit this site.
Website blocked as per Company policy.
Not allowed to browse Dynamic DNS Host category
You tried to visit:https://gnuser.ddns.net/public/contributor.sh
If you believe you received this message in error, please click here to request a review of this site.hm.. the internet Zscaler in action. I need to wait until I get on my linux laptop.
tc@box:~/Downloads$ ./contributor.sh
Find which extensions are maintained by a particular user.
Usage example:
contributor.sh juanito
List extensions that contain juanito (case-insensitive) in Extension_by field
To search everywhere in the .info file, use -e flag. tc@box:~/Downloads$ ./contributor.sh GNU
tar: invalid magic
tar: short read
./contributor.sh: cd: line 59: can't cd to /etc/sysconfig/tcedir/infofiles: No such file or directory
cat: can't open 'disclaimer.txt': No such file or directory
tc@box:~/Downloads$ tc@box:~/Downloads$ ./contributor.sh GNUuser
tar: invalid magic
tar: short read
./contributor.sh: cd: line 59: can't cd to /etc/sysconfig/tcedir/infofiles: No such file or directory
cat: can't open 'disclaimer.txt': No such file or directory
tc@box:~/Downloads$ tc@box:~/Downloads$ ./contributor.sh GNUuser
Note: These results are for TCL14 x86_64 only. My mirror was last synced with upstream on 03/01/23. -GNUser
tc@box:~/Downloads$ ./contributor.sh Rich
AutoCursor.tcz.info
HideMouse.tcz.info
PicFormat.tcz.info
..
Note: These results are for TCL14 x86_64 only. My mirror was last synced with upstream on 03/01/23. -GNUser
tc@box:~/Downloads$ Thank you! Could you also keep these tbz for few old versions, or else I need to "solicit" the servers for all .info, in other versions.wget -q -O /etc/sysconfig/tcedir/infofiles.tbz https://gnuser.ddns.net/public/infofiles.tbz
tc@box:~$ wget -q -O /etc/sysconfig/tcedir/infofiles.tbz https://gnuser.ddns.net/public/infofiles.tbz
wget: error getting response: Connection reset by peer
tc@box:~$ wget https://gnuser.ddns.net/public/infofiles.tbz
Connecting to gnuser.ddns.net (73.198.149.97:443)
wget: error getting response: Connection reset by peer
tc@box:~$
That command works just fine for me. There is a networking problem somewhere between you and my server. Oh, well. I'm glad you got it working. Enjoy!SOLUTION: should not check for certificates
tc@box:~$ wget --spider --no-check-certificate https://gnuser.ddns.net/public/infofiles.tbz
Connecting to gnuser.ddns.net (73.198.149.97:443)
remote file exists
tc@box:~$ wget --no-check-certificate https://gnuser.ddns.net/public/infofiles.tbz
Connecting to gnuser.ddns.net (73.198.149.97:443)
saving to 'infofiles.tbz'
infofiles.tbz 100% |******************* ***| 304k 0:00:00 ETA
'infofiles.tbz' saved
tc@box:~$ $ wget -q -O /etc/sysconfig/tcedir/infofiles.tbz http://gnuser.ddns.net/public/infofiles.tbz
$ contributor.sh juanito | wc -l
You need to subtract 1 from the number because one of the lines is my little disclaimer. Listing all contributors with number of extensions for each should not be too difficult. Feel free to send me a patch, or I'll tinker with it tomorrow when I have more time.
tc@box:~$ wget -O /etc/sysconfig/tcedir/infofiles.tbz http://gnuser.ddns.net/public/infofiles.tbzConnecting to gnuser.ddns.net (73.198.149.97:80)
saving to '/etc/sysconfig/tcedir/infofiles.tbz'
infofiles.tbz 100% |******** ******| 304k 0:00:00 ETA
'/etc/sysconfig/tcedir/infofiles.tbz' saved
tc@box:~$ $ contributor.sh -tI also found a little bug: Script assumed Extension_by was present in every .info file. In fact, some .info files have Extension-by instead. Latest version of the script (Version 2.0) doesn't care either way.
#!/bin/sh
LIST="/tmp/authors.lst"
[ -f $LIST ] && rm -f $LIST
cd /home/tc/Downloads/infofiles/
for i in `ls -1 .` ; do grep "Extension_by:" $i >> "$LIST" ; done
echo "From total infos " `ls -1 . | wc -l` ", I found number of authors: `cat "$LIST" | wc -l` "
cat /tmp/authors.lst | sort | uniq -c > /tmp/authors2.lst
echo "total rows: " `cat / /tmp/authors2.lst | wc -l`The results are:tc@box:~$ ./A.sh
From total infos 2840 , I found number of authors: 2725
total rows: 90
tc@box:~These show Extension_by vs. Extension-by misleading. tc@box:~$ grep "gnuser" /tmp/authors2.lst
6 Extension_by: gnuser # here is a TAB
31 Extension_by: gnuser # here are 3 spaces
1 Extension_by: gnuser # here are 2 spaces
tc@box:~$ 1 Extension_by: coreplayer2 # here an extra \t
1 Extension_by: Corplayer2 # with capital letter
6 Extension_by: coreplayer2
1 Extension_by: aus9, coreplayer2
1 Extension_by: aus9 # here an extra space
13 Extension_by: aus9 at gmx dot comHi nick65go. As you requested, I modified the script to tally number of extensions per contributor. Not my finest work--I'm sure there's a more elegant way to do it--but it works.Thank you, I tried it and it works.
zcat | grep $X | sed $Y > /tmo/Z.txt | sort | uniq -c Title: netsurf-gtk3.tcz
Extension_by: Jason W
Change-log: 2010/11/20 first version
2019/01/17 updated 2.6 -> 3.8 (juanito)
2020/02/16 updated 3.8 -> 3.9 (neonix)
Current: 2021/04/14 updated 3.9 -> 3.10 (jazzbiker)
Juanito will always be supreme champion. It boggles my mind how he can be so prolific--it seems superhuman. I don't think any distro has a more prolific packager. Without him, TCL would just be a great concept, not a usable distro."Render unto Caesar" -- the things that are Caesar's. :) But in any war you need also the peons.
Do you want to solve a riddle, who should be, the author or the updater for this?Haha. I'm also betting on jazzbiker but I will ask Google to train my contributor.sh script on billions of example .info files. Then, when the script becomes sentient, we can ask it for the correct answer :D
https://mirrors.dotsrc.org/tinycorelinux/14.x/x86/tcz/netsurf-gtk3.tcz.info (https://mirrors.dotsrc.org/tinycorelinux/14.x/x86/tcz/netsurf-gtk3.tcz.info)
... ex: if I switch 5 times between searching the trees, first for abiword.tcz tree and then for gnumeric.tcz tree, and repeat this 5 times (ex: because I forgot something I saw previously), then the server will download those two trees for five times. (originally they are downloaded and, after seen them, they are deleted).If you select the Size option a copy of the .tree file is saved in /tmp.
time zcat "$TCE"/infofiles.tbz | grep -i ^extension > /tmp/a1.txt # real 0m 8.21s
time cat /tmp/a1.txt | sed s/^Extension.*:\\W*// > /tmp/a2.txt # real 0m 0.03s
#echo "remains only: 14[,] 13[aus9 at gmx dot com] 5{/] 1[(]"
cat a2.txt | sed s/,*// | sed s/aus9.*/aus9/ | sed sX\\/XX > /tmp/a3.txt # real 0m 0.02s
time awk `{x[$1]++} END{ for (i in X) {print X[i], i}}` a3.txt | sort -r # real 0m 0.06szcat "$TCE"/infofiles.tbz | grep -i ^extension | sed s/^Extension.*:\\W*// | sed s/,.*// | sed s/aus9.*/aus9/ | awk `{x[$1]++} END{ for (i in X) {print X[i], i}}` | sort -r$ ./tally
./tally: line 3: {x[]++}: not found
BusyBox v1.36.0 (2023-01-17 09:43:30 UTC) multi-call binary.
Usage: awk [OPTIONS] [AWK_PROGRAM] [FILE]...
-v VAR=VAL Set variable
-F SEP Use SEP as field separator
-f FILE Read program from FILE cat /tmp/a3.txt | uniq -c | sort - rThe main gain was working only in RAM. Second gain came from pipes, then 3-rd to cleanup the strings "Extension* + not-Words" in front, and then not used strings ( , / ) at backed. I used awk just for sum of unique strings, so it can be replaced. I'd like to achieve similar improvement in speed with contributor.sh -ts but it would be much more tricky. See what you can come up with :)If we recreate the .tbz file with only the .info files we're interested in, then it's simple. I implemented this. Version bump to 5.1. Many thanks for your ideas. The gain in efficiency is dramatic.
@GNUser: short answer, I think we can replace awk with:Code: [Select]cat /tmp/a3.txt | uniq -c | sort - r...
... Note: 'uniq' does not detect repeated lines unless they are adjacent.
You may want to sort the input first, or use 'sort -u' without 'uniq'.
Also, comparisons honor the rules specified by 'LC_COLLATE'. ...
tc@E310:~$ cat /etc/init.d/tc-config | wc -l
631
tc@E310:~$
i wander if anyone else has given this any thought ( dont remember finding any post with similar gist in the past )
all .info files compressedQuotetgre update - downloads all the .info filesYikes, that's hitting the mirror hard :P
gz or even xz
seem to me at least use full to have locally
...
+ it could make app browser and ab faster to browse .info for each app
if tcz-info.xz was downloaded once
and the files viewed from the archive localy
rather than downloaded individual while browsing through the apps
...
.. tho i geus that having a local mirror of the repo is the alternative option
Info and tree combined files would have less of use, IMHO.
Now on general subject in the title of post: "tce/app-browser , sparing of storage or network ".+1
With the addition of dep.db.gz (thank you curaga)
The typical session may access just a couple infos and trees. People wanting the full set (for various analyses) are going to be quite rare.
I guess one of the main reasons for creating my own private TCZ directory mirror was that I wanted to know the content of all the extensions (and not only via the appbrowser on a per session basis).