General TC > General TC Talk

Trying to provide better to provides.db

(1/5) > >>

CentralWare:
Geed evening everyone!

I have a set of scripts which rebuild our local repository and I'm looking to see if anyone can help us tweak the one which creates provides.db

1. We get a list of files (filename.tcz) and the filename is sent to provides.db
2. After the filename, comes the content of filename.tcz.list (listing of all files found inside filename.tcz)
3. We add an empty line into provides.db and rinse, lather and repeat for the remaining files in the repo.

Our command which does all of this is as follows:

--- Code: ---find *.tcz -type f -print -exec cat "${SRC}/{}.list" >> "$OUT/provides.db" \; -exec echo >> "$OUT/provides.db" \;
--- End code ---
SRC is our path to /mnt/array/tinycorelinux/(VERSION).x/(PLATFORM)/tcz
OUT is where we're going to save provides.db for processing before it's moved to SRC

Creating a FOR loop seems to take just as long as -exec above.
Any thoughts on how to make it better?

Thanks!

Rich:
Hi CentralWare
I couldn't get your version to run, so I didn't have a benchmark
to compare to. I attached a copy of what I came up with.
I ran it against the TC14 x86 repo 3 times with a cold cache
for each run. These were the times:

--- Code: ---rich@tcbox:~$ ./ClearCache.sh
rich@tcbox:~$ time ./BuildProvides.sh

real    0m4.728s
user    0m0.088s
sys     0m0.592s
rich@tcbox:~$ ./ClearCache.sh
rich@tcbox:~$ time ./BuildProvides.sh

real    0m4.754s
user    0m0.068s
sys     0m0.596s
rich@tcbox:~$ ./ClearCache.sh
rich@tcbox:~$ time ./BuildProvides.sh

real    0m4.715s
user    0m0.096s
sys     0m0.620s
rich@tcbox:~$
--- End code ---

CentralWare:
Benchmarks are about the same

--- Code: ---# time ./BuildProvides.sh

real    0m13.649s
user    0m4.320s
sys     0m8.220s


# time ./build_prov
Creating Provides Database...
cat: can't open '/mnt/array1/tinycorelinux/14.x/x86/tcz/Xorg-7.7-lib.tcz.list': No such file or directory
cat: can't open '/mnt/array1/tinycorelinux/14.x/x86/tcz/alsa-plugins-dev.tcz.list': No such file or directory
cat: can't open '/mnt/array1/tinycorelinux/14.x/x86/tcz/geany-plugins-dev.tcz.list': No such file or directory
cat: can't open '/mnt/array1/tinycorelinux/14.x/x86/tcz/gnome-keyring-dev.tcz.list': No such file or directory
cat: can't open '/mnt/array1/tinycorelinux/14.x/x86/tcz/hplip-dev.tcz.list': No such file or directory
cat: can't open '/mnt/array1/tinycorelinux/14.x/x86/tcz/original-modules-6.1.2-tinycore.tcz.list': No such file or directory
cat: can't open '/mnt/array1/tinycorelinux/14.x/x86/tcz/original-modules-6.1.2-tinycore64.tcz.list': No such file or directory
cat: can't open '/mnt/array1/tinycorelinux/14.x/x86/tcz/strongswan-dev.tcz.list': No such file or directory

real    0m13.952s
user    0m4.970s
sys     0m8.440s

--- End code ---
Mine throws MIA errors because of meta extensions and actual "missing" files.

Example: 14.x x86 strongswan-dev.tcz
This one's broken (missing .list file where there should be one)
NOTE: This could be just a missing file on THIS END -- which is why this system is being updated in the first place; to ensure things are properly (and efficiently) synchronized! :)
Nope, it's missing on that end as well: https://distro.ibiblio.org/tinycorelinux/14.x/x86/tcz/strongswan-dev.tcz.list

...whereas original-modules-6.1.2-tinycore64.tcz.list "should" be missing, but has a file size and md5 >0bytes so there's no "visible" marker to work with yet without extracting the tcz and seeing whether files exist inside it.

Thank you for your efforts!

Rich:
Hi CentralWare

--- Quote from: CentralWare on January 26, 2024, 11:28:16 PM --- ... so there's no "visible" marker to work with yet without extracting the tcz and seeing whether files exist inside it. ...
--- End quote ---
No need to extract, get a listing:

--- Code: ---rich@tcbox:~$ unsquashfs -ll tinycorelinux/15.x/x86_64/tcz/original-modules-6.6.8-tinycore64.tcz
Parallel unsquashfs: Using 6 processors
0 inodes (0 blocks) to write

drwx------ root/root                26 2023-12-23 17:54 squashfs-root
drwxr-xr-x root/root                 3 2023-12-23 17:54 squashfs-root/lib
rich@tcbox:~$
--- End code ---

Rich:
Hi CentralWare
Test if extension contains files.

This contains files:

--- Code: ---rich@tcbox:~$ unsquashfs -ll tinycorelinux/14.x/x86_64/tcz/alsamixergui.tcz | grep -q '^-'
rich@tcbox:~$ echo $?
0
rich@tcbox:~$
--- End code ---

This contains only empty directories:

--- Code: ---rich@tcbox:~$ unsquashfs -ll tinycorelinux/14.x/x86_64/tcz/alsa-plugins-dev.tcz | grep -q '^-'
rich@tcbox:~$ echo $?
1
rich@tcbox:~$
--- End code ---

Navigation

[0] Message Index

[#] Next page