... when i use the "provides" function and search for "ps" it reports many results. how would the uninitiated decide which to use? ...Good question.
tc@E310:~$ provides.sh ps | wc -l
580
tc@E310:~$tc@E310:~$ provides.sh cal | wc -l
2661
tc@E310:~$That's because it's also matching on every usr/local/ string in the provides.dbtc@E310:~$ provides.sh "bin\/cal"
ax25-apps.tcz
fox-apps.tcz
fox.tcz
util-linux.tcz
valgrind.tcz
xastir.tcz
tc@E310:~$Notice we need to escape the path separator "/" with a backslash "\".tc@E310:~$ provides.sh "bin\/ps"
aspell-dev.tcz
ghostscript.tcz
gnutls3.6.tcz
gnutls.tcz
lcms2.tcz
libcap-ng.tcz
libpsl.tcz
pax-utils.tcz
postgresql-10-client.tcz
postgresql-10.tcz
postgresql-11-client.tcz
postgresql-11.tcz
postgresql-12-client.tcz
postgresql-12.tcz
postgresql-9.5-client.tcz
postgresql-9.5.tcz
postgresql-9.6-client.tcz
postgresql-9.6.tcz
procps-ng.tcz
procps.tcz
pstree.tcz
putty.tcz
sc.tcz
tc@E310:~$23 results.tc@E310:~$ provides.sh "local\/bin\/ps"
aspell-dev.tcz
ghostscript.tcz
gnutls3.6.tcz
gnutls.tcz
lcms2.tcz
libcap-ng.tcz
libpsl.tcz
pax-utils.tcz
procps-ng.tcz
procps.tcz
pstree.tcz
putty.tcz
sc.tcz
tc@E310:~$That's a little better./usr/local/pgsql10/bin/psql/usr/local/pgsql10/bin/psqlcd /./then do a directory listing and you'll see you're in the root directory.sudo ln -rs /usr/bin/provides.sh /usr/bin/Provides.sh
tc@E310:~/Scripting/Provides$ ls -l /usr/bin/Provides.sh
lrwxrwxrwx 1 root root 11 Nov 14 00:02 /usr/bin/Provides.sh -> provides.shtc@E310:~/Scripting/Provides$ time provides.sh "\/local\/bin\/ps"
aspell-dev.tcz
ghostscript.tcz
gnutls3.6.tcz
gnutls.tcz
lcms2.tcz
libcap-ng.tcz
libpsl.tcz
pax-utils.tcz
procps-ng.tcz
procps.tcz
pstree.tcz
putty.tcz
sc.tcz
real 0m 1.81s
user 0m 0.90s
sys 0m 0.18stc@E310:~/Scripting/Provides$ time Provides.sh "\/local\/bin\/ps"
procps-ng.tcz
procps.tcz
real 0m 1.92s
user 0m 1.00s
sys 0m 0.20stc@E310:~/Scripting/Provides$ time provides.sh "\/local\/bin\/cal"
ax25-apps.tcz
fox-apps.tcz
fox.tcz
util-linux.tcz
valgrind.tcz
xastir.tcz
real 0m 1.99s
user 0m 0.86s
sys 0m 0.24stc@E310:~/Scripting/Provides$ time Provides.sh "\/local\/bin\/cal"
util-linux.tcz
real 0m 1.96s
user 0m 0.98s
sys 0m 0.23s#!/bin/busybox ash
. /etc/init.d/tc-functions
useBusybox
TARGET="$1"
[ -z "$TARGET" ] && exit 1
TCEDIR="/etc/sysconfig/tcedir"
DB="provides.db"
getMirror
cd "$TCEDIR"
if zsync -i "$TCEDIR"/"$DB" -q "$MIRROR"/"$DB".zsync
then
rm -f "$DB".zs-old
else
if [ ! -f "$TCEDIR"/"$DB" ]
then
wget -O "$TCEDIR"/"$DB".gz "$MIRROR"/"$DB".gz
gunzip "$TCEDIR"/"$DB".gz
fi
fi
cd - > /dev/null
if [ "${0##*/}" == "provides.sh" ]
then
awk 'BEGIN {FS="\n";RS=""} /'${TARGET}'/{print $1}' "$TCEDIR"/"$DB"
else
awk -v Find="$TARGET" -v FindEnd="$TARGET$" 'BEGIN {FS="\n";RS=""} {if ( $0 ~ Find ) { for (i=2; i <= NF; i++) { if ( $i ~ FindEnd ) {print $1; break} } } }' "$TCEDIR"/"$DB"
fi
chmod g+rw "$TCEDIR"/"$DB"#!/bin/sh
[ -z "$1" ] && exit 1
TCEDIR="/etc/sysconfig/tcedir"
DB="provides.db"
awk 'BEGIN {FS="\n";RS=""} /'"$1"'/{print $1}' "$TCEDIR"/"$DB"
tc@pi4:~$ prov 'bin\/ps\n'
procps-ng.tcz
What I don't understand is the purpose of the chmod command
at the end that executes after the script has finished running.
... While searching, you can add "\n" to the end of the query. ...I was so focused figuring out how to include $ as an anchor that I
... I believe it was meant to make sure the file is modifiable by any user to sync with the repo.Makes sense. It just seemed like an odd place to put that line.
squashfs-tools.tcz
/usr/local/bin/mksquashfs
/usr/local/bin/unsquashfstc@E310:~/Scripting/Provides$ provides.sh "bin\/mksquashfs\n"
squashfs-tools.tcz
tc@E310:~/Scripting/Provides$tc@E310:~/Scripting/Provides$ provides.sh "bin\/unsquashfs\n"
tc@E310:~/Scripting/Provides$
It seems using \n does not work if the item
you are searching for is in the last field of the record.
tc@pi4:/etc/sysconfig/tcedir$ grep -A3 squashfs provides.db
squashfs-tools.tcz
usr/local/bin/mksquashfs
usr/local/bin/unsquashfs
usr/local/bin/sqfscat
usr/local/bin/sqfstar
tc@pi4:/etc/sysconfig/tcedir$ grep -A3 'tar.tcz' provides.db
tar.tcz
usr/local/lib/tar/rmt
usr/local/bin/tar
tc@pi4:/etc/sysconfig/tcedir$ prov 'bin\/tar\n'
tc@pi4:/etc/sysconfig/tcedir$ prov 'bin\/tar$'
tar.tcz
tc@pi4:/etc/sysconfig/tcedir$ prov 'bin\/tar\n/||/bin\/tar$'
tar.tcz
tc@pi4:/etc/sysconfig/tcedir$ provides.sh 'bin\/tar\n'
tc@pi4:/etc/sysconfig/tcedir$ provides.sh 'bin\/tar$'
tar.tcz
tc@pi4:/etc/sysconfig/tcedir$ provides.sh 'bin\/tar\n/||/bin\/tar$'
tar.tcz
tc@pi4:~$ p(){ awk 'BEGIN {FS="\n";RS=""} /'"$1"'\/'"$2"'\n/||/'"$1"'\/'"$2"'$/{print $1}' /etc/sysconfig/tcedir/provides.db; }
tc@pi4:~$ p bin tar
tar.tcz
tc@pi4:~$ p(){ awk -v FS="\n" -v RS="" "/$1\/$2\n/||/$1\/$2$/{print \$1}" /etc/sysconfig/tcedir/provides.db; }
tc@pi4:~$ p bin tar
tar.tcz
tc@pi4:/etc/sysconfig/tcedir$ grep -A3 submitqc.tcz provides.db
submitqc.tcz
/usr/local/bin/submitqc
/usr/local/share/doc/submitqc/COPYING
tc@pi4:/etc/sysconfig/tcedir$ p bin submitqc
submitqc.tcz
tc@pi4:/etc/sysconfig/tcedir$ p submitqc COPYING
submitqc.tcz
tc@pi4:/etc/sysconfig/tcedir$ grep -A3 wget.tcz provides.db
wget.tcz
/usr/local/etc/wgetrc
/usr/local/bin/wget
tc@pi4:/etc/sysconfig/tcedir$ p bin wget
wget.tcz
tc@pi4:/etc/sysconfig/tcedir$ p etc wgetrc
wget.tcz
p "local\/bin" pstc@E310:~$ Provides.sh local\/bin\/ps
procps-ng.tcz
procps.tcz
tc@E310:~$
Hi polikuoI think any search tool should default to an exact search, possibly with an option for a fuzzy search. Less ideal (IMO) would be defaulting to a fuzzy search with an option for an exact search. Having only a fuzzy search is just frustrating.
That looks like it works, but I don't think a user should be
forced to use an exact match. A user may not be sure what
they are searching for. For example, they might want to
find all extensions with files that have crypt in their name.
I've done those types of searches myself on occasion.
... I tried coding the exact search, using other tools, to search for lines ending in /${TARGET} and it worked perfectly... but took four minutes to run. ...I once tried that too, but nothing came close to touching the speed of awk.
Field1
field2
field3
Field1 is the .tcz name. Field2 and Field3 are filenames. The blank line atField1\nField2\nField3The record is treated as one long string with embedded \n recordtc@E310:~/Scripting/Provides$ ./prov.sh "bin\/egrep\n"
grep.tcz
/usr/local/bin/egrep
/usr/local/bin/fgrep
/usr/local/bin/greptc@E310:~/Scripting/Provides$You can see it print Field1\nField2\nField3\n but Field4 is immediatelymake.sh
cat << 'EOF' > make.sh
#!/bin/bash
sqlite3 files.db "CREATE TABLE tcz_files (name TEXT NOT NULL,file TEXT NOT NULL);"
for filename in $(ls *.tcz)
do
tczfile=$filename;for file in $(unsquashfs -lc $tczfile); do sqlite3 files.db "INSERT INTO tcz_files ( NAME , FILE ) VALUES ( '$tczfile' , '$(basename $file)' );" ; done
done
EOF
chmod u+x make.sh
search.sh
cat << 'EOF' > search.sh
sqlite3 -box files.db "SELECT * FROM tcz_files WHERE file LIKE '%$1%';"
EOF
chmod u+x search.sh
ls
files.db gimp2.tcz make.sh search.sh sqlite3-bin.tcz sqlite3.tcz
./search.sh pdf
┌───────────┬───────────────┐
│ name │ file │
├───────────┼───────────────┤
│ gimp2.tcz │ file-pdf-load │
│ gimp2.tcz │ file-pdf-save │
└───────────┴───────────────┘
./search.sh sql
┌─────────────────┬─────────────────────┐
│ name │ file │
├─────────────────┼─────────────────────┤
│ sqlite3-bin.tcz │ sqlite3 │
│ sqlite3.tcz │ libsqlite3.so │
│ sqlite3.tcz │ libsqlite3.so.0 │
│ sqlite3.tcz │ libsqlite3.so.0.8.6 │
└─────────────────┴─────────────────────┘
sqlite3 -box files.db "SELECT COUNT(*) AS 'Number of filenames' FROM tcz_files;"
┌─────────────────────┐
│ Number of filenames │
├─────────────────────┤
│ 4191 │
└─────────────────────┘
ls -alh files.db
-rw-r--r-- 1 patrik users 164K 14 nov 22.34 files.db
% as
as found in: binutils
% time as
as found in: binutils
0.00s user 0.01s system 97% cpu 0.007 total
% time cpp
cpp found in: gcc
0.00s user 0.00s system 97% cpu 0.008 total
% time users
users found in: coreutils
0.01s user 0.00s system 95% cpu 0.008 total
... OR, if the speed is not the main goal (but nice for multi-CPU core) then maybe the user confort for seach parameters (GUI options will be nice also). But of course, the pro[fessionals] already know the use of "" and back-quotes, etc.I think backwards compatibility and user friendly search parameters
tc@E310:~/Scripting/Provides$ ./provides.sh grep | wc -l
51
tc@E310:~/Scripting/Provides$tc@E310:~/Scripting/Provides$ ./provides.sh /grep
boost-1.65-dev.tcz
boost-dev.tcz
emacs.cedet.tcz
emacs.doc.tcz
emacs.minimal.tcz
emacs.prog.tcz
fpc-src.tcz
fpc.tcz
grep.tcz
kompozer.tcz
libreoffice.tcz
mariadb-test.tcz
python3.6.tcz
tc@E310:~/Scripting/Provides$tc@E310:~/Scripting/Provides$ ./provides.sh bin/grep
emacs.minimal.tcz
fpc.tcz
grep.tcz
tc@E310:~/Scripting/Provides$tc@E310:~/Scripting/Provides$ ./provides.sh bin/grep$
grep.tcz
tc@E310:~/Scripting/Provides$tc@E310:~/Scripting/Provides$ ./provides.sh --help
Finds extension(s) that provide a filename.
Filenames in list being searched include full paths, for example:
usr/local/bin/grep
Usage:
provides.sh FileName
Examples:
This also finds all extensions that include local in their pathes.
provides.sh cal
This finds *bin/cal*. Including sbin/cal, bin/calibrate, etc.
provides.sh bin/cal
This finds *bin/cal. Including sbin/cal, usr/bin/cal, etc.
provides.sh bin/cal$
The more specific the search, the fewer the results returned.
tc@E310:~/Scripting/Provides$#!/bin/busybox ash
. /etc/init.d/tc-functions
useBusybox
# --------------------------------------------------------------------------- #
Usage()
{
echo "
Finds extension(s) that provide a filename.
Filenames in list being searched include full paths, for example:
usr/local/bin/grep
Usage:
${0##*/} FileName
Examples:
This also finds all extensions that include local in their pathes.
${0##*/} cal
This finds *bin/cal*. Including sbin/cal, bin/calibrate, etc.
${0##*/} bin/cal
This finds *bin/cal. Including sbin/cal, usr/bin/cal, etc.
${0##*/} bin/cal$
The more specific the search, the fewer the results returned.
"
exit
}
# --------------------------------------------------------------------------- #
# See if user needs help.
case "$1" in
-h) Usage;;
-help) Usage;;
--help) Usage;;
"") exit 1;;
esac
# Flag to indicate whether we want an exact match. 0=No 1=Yes.
Exact=0
TARGET="$1"
TCEDIR="/etc/sysconfig/tcedir"
DB="provides.db"
getMirror
cd "$TCEDIR"
if zsync -i "$TCEDIR"/"$DB" -q "$MIRROR"/"$DB".zsync
then
rm -f "$DB".zs-old
else
if [ ! -f "$TCEDIR"/"$DB" ]
then
wget -O "$TCEDIR"/"$DB".gz "$MIRROR"/"$DB".gz
gunzip "$TCEDIR"/"$DB".gz
fi
fi
chmod g+rw "$TCEDIR"/"$DB"
cd - > /dev/null
# Save string length of TARGET.
Length=${#TARGET}
# Remove trailing $ (exact match request) if present.
TARGET="${TARGET%$}"
# If TARGET is now shorter, exact match was requested.
[ ${#TARGET} -lt $Length ] && Exact=1
# Replace all instances of / with \/.
TARGET="${TARGET//\//\\\/}"
if [ $Exact -eq 0 ]
then
awk 'BEGIN {FS="\n";RS=""} /'${TARGET}'/{print $1}' "$TCEDIR"/"$DB"
else
awk 'BEGIN {FS="\n";RS=""} /'${TARGET}'\n/||/'${TARGET}'$/{print $1}' "$TCEDIR"/"$DB"
fi
# Maximum age of provides.db in seconds before it is considered to be stale and
# needs to be redownloaded.
MaxAge=3600
PROVIDES="provides.db"
RefreshProvidesDB()
{
# This downloads a fresh copy of provides.db if any of the following are true:
# 1. The file is not in the current directory.
# 2. The file is older than 1 hour (3600 seconds).
cd /etc/sysconfig/tcedir
# Make sure the proper provides.db exists.
if [ -f "$PROVIDES" ]
then
# Compute number of seconds since provides.db modified (downloaded).
Age=$(( $(date +%s) - $(date -r "$PROVIDES" +%s) ))
if [ $Age -lt $MaxAge ]
then
# File is recent enough to use.
return
fi
# File is too old, delete it.
rm "$PROVIDES"
fi
# Fetch a fresh copy of the file.
wget -q -O "$PROVIDES" "$URL$PROVIDES"
if [ $? -ne 0 ]
then
echo "Download failed for: $URL$PROVIDES"
exit 1
fi
# Make sure it has a current timestamp.
touch "$PROVIDES"
cd - > /dev/null
}
... Perhaps you could wrap up the syncing code into a shell function ?Done. :)
#!/bin/busybox ash
. /etc/init.d/tc-functions
useBusybox
# --------------------------------------------------------------------------- #
UpdateProvidesDB()
{
getMirror
cd "$TCEDIR"
if zsync -i "$TCEDIR"/"$DB" -q "$MIRROR"/"$DB".zsync
then
rm -f "$DB".zs-old
else
if [ ! -f "$TCEDIR"/"$DB" ]
then
wget -O "$TCEDIR"/"$DB".gz "$MIRROR"/"$DB".gz
gunzip "$TCEDIR"/"$DB".gz
fi
fi
chmod g+rw "$DB"
cd - > /dev/null
}
# --------------------------------------------------------------------------- #
# --------------------------------------------------------------------------- #
Usage()
{
echo "
Finds extension(s) that provide a filename.
Filenames in list being searched include full paths, for example:
usr/local/bin/grep
Usage:
${0##*/} FileName
Examples:
This also finds all extensions that include local in their pathes.
${0##*/} cal
This finds *bin/cal*. Including sbin/cal, bin/calibrate, etc.
${0##*/} bin/cal
This finds *bin/cal. Including sbin/cal, usr/bin/cal, etc.
${0##*/} bin/cal$
The more specific the search, the fewer the results returned.
"
exit
}
# --------------------------------------------------------------------------- #
# See if user needs help.
case "$1" in
-h) Usage;;
-help) Usage;;
--help) Usage;;
"") exit 1;;
esac
# Flag to indicate whether we want an exact match. 0=No 1=Yes.
Exact=0
TARGET="$1"
TCEDIR="/etc/sysconfig/tcedir"
DB="provides.db"
# Run zsync on the provides.db file.
UpdateProvidesDB
# Save string length of TARGET.
Length=${#TARGET}
# Remove trailing $ (exact match request) if present.
TARGET="${TARGET%$}"
# If TARGET is now shorter, exact match was requested.
[ ${#TARGET} -lt $Length ] && Exact=1
# Replace all instances of / with \/.
TARGET="${TARGET//\//\\\/}"
if [ $Exact -eq 0 ]
then
awk 'BEGIN {FS="\n";RS=""} /'${TARGET}'/{print $1}' "$TCEDIR"/"$DB"
else
awk 'BEGIN {FS="\n";RS=""} /'${TARGET}'\n/||/'${TARGET}'$/{print $1}' "$TCEDIR"/"$DB"
fi
provides.sh "\/usr_1\/"provides.sh /usr_1/
tc@E310:~/Scripting/Provides$ time ./provides.sh bin/grep$
grep.tcz
real 0m 1.36s
user 0m 0.49s
sys 0m 0.23s
tc@E310:~/Scripting/Provides$tc@E310:~/Scripting/Provides$ time ./provides.sh -nz bin/grep$
grep.tcz
real 0m 0.22s
user 0m 0.19s
sys 0m 0.03s
tc@E310:~/Scripting/Provides$-nz switch can be placed before or after the search term.tc@E310:~/Scripting/Provides$ time ./provides.sh Black Gnome -nz
volumeicon.tcz
real 0m 0.13s
user 0m 0.11s
sys 0m 0.02s
tc@E310:~/Scripting/Provides$
tc@E310:~/Scripting/Provides$ time ./provides.sh Make Cocoa Headers.txt -nz
fpc-src.tcz
real 0m 0.13s
user 0m 0.10s
sys 0m 0.02s
tc@E310:~/Scripting/Provides$No quotes required.tc@E310:~/Scripting/Provides$ ./provides.sh --help
Find extension(s) that provide a filename.
Filenames in list being searched include full paths, for example:
usr/local/bin/grep
Usage:
provides.sh [ -nz ] FileName
-nz Skip updating (zsync) the provides.db file. This speeds up
the search, but might miss items if provides.db is outdated.
Examples:
provides.sh cal Finds cal anywhere in FileName
provides.sh bin/cal Finds bin/cal anywhere in FileName
provides.sh bin/cal$ Finds FileName that ends in bin/cal
provides.sh Black Gnome Finds FileName with embedded spaces
The more specific the search, the fewer the results returned.
tc@E310:~/Scripting/Provides$sudo cp provides.sh /usr/binThe new provides.sh will now be functional for both command line and Apps GUI.--- provides.sh
+++ /usr/local/bin/provides.sh
@@ -32,6 +32,7 @@
fi
chmod g+rw "$DB"
cd - > /dev/null
+touch $LIST
}
# --------------------------------------------------------------- #
@@ -85,7 +86,7 @@
[ -z "$TARGET" ] && exit 1
# Run zsync on the provides.db file if NZ equals 1.
-[ $NZ -eq 1 ] && UpdateProvidesDB
+[ $NZ -eq 1 ] || [ -n "$(find $LIST -mmin +15)" ] && UpdateProvidesDB
# Save string length of TARGET.
Length=${#TARGET}
tc@pi4:/tmp$ uname -a
Linux pi4 6.6.47-piCore-v8 #21 SMP PREEMPT Sat Aug 31 15:08:05 EDT 2024 aarch64 GNU/Linux
tc@pi4:/tmp$ busybox find --help
BusyBox v1.36.1 (2024-04-07 15:54:04 UTC) multi-call binary.
...
...
-mtime DAYS mtime is greater than (+N), less than (-N),
or exactly N days in the past
-mmin MINS mtime is greater than (+N), less than (-N),
or exactly N minutes in the past
... Using just awk and shell, the search time (for you) is less than 2 seconds! ...Most of that time is consumed by zsync checking if the servers version of provides.db
... I wonder what (old?) computer specification you used (CPU, HDD, RAM), if you want to share, for your test.I retired my old 800Mhz i686 (coppermine), 512 Meg RAM, PATA drives system
tc@E310:~$ sudo inxi -CfmDGnSM
System:
Host: E310 Kernel: 4.19.10-tinycore i686 bits: 32 Desktop: FLWM
Distro: TinyCore 10.1
Machine:
Type: Desktop System: Dell product: Dell DV051 v: N/A serial: 9Q5X2B1
Mobo: Dell model: 0JC474 serial: ..CN481115B2077P. BIOS: Dell v: A04
date: 04/04/2006
Memory:
RAM: total: 2.96 GiB used: 1.30 GiB (44.1%)
Array-1: capacity: 4 GiB note: est. slots: 2 EC: Single-bit ECC
Device-1: DIMM_1 size: 2 GiB speed: 533 MHz
Device-2: DIMM_2 size: 2 GiB speed: 533 MHz
CPU:
Topology: Single Core model: Intel Pentium 4 bits: 64 type: MT
L2 cache: 1024 KiB
Speed: 2793 MHz min/max: N/A Core speeds (MHz): 1: 2793 2: 2793
Flags: acpi apic bts cid clflush cmov constant_tsc cpuid cx16 cx8 de
ds_cpl dtes64 dts fpu fxsr ht lahf_lm lm mca mce mmx monitor msr mtrr pae
pat pbe pebs pge pni pse pse36 sep ss sse sse2 tm tsc vme xtpr
Graphics:
Device-1: Intel 82915G/GV/910GL Integrated Graphics driver: i915 v: kernel
Display: server: X.Org 1.20.4 driver: intel resolution: 1920x1080~60Hz
OpenGL: renderer: Mesa DRI Intel 915G x86/MMX/SSE2 v: 1.4 Mesa 19.2.3
Network:
Device-1: Intel 82562ET/EZ/GT/GZ - PRO/100 VE Ethernet driver: e100
IF: eth0 state: up speed: 100 Mbps duplex: full mac: 00:13:20:c4:4a:20
IF-ID-1: dummy0 state: down mac: 6e:93:1c:04:f2:dc
IF-ID-2: ip_vti0 state: down mac: 00:00:00:00
IF-ID-3: tunl0 state: down mac: 00:00:00:00
Drives:
Local Storage: total: 931.52 GiB used: 144.75 GiB (15.5%)
ID-1: /dev/sda vendor: HP model: MB0500EBNCR size: 465.76 GiB
ID-2: /dev/sdb vendor: HP model: MB0500EBNCR size: 465.76 GiB
tc@E310:~$
... Cause I really like to skip the zsync routine. ...I get the feeling you don't like waiting for results from your computer. ::)
tc@E310:~/Scripting/Provides$ time ./provides.sh -nz glxinfo$
mesa-demos.tcz
real 0m 0.45s
user 0m 0.41s
sys 0m 0.04s
tc@E310:~/Scripting/Provides$ time ./provides.sh -nz glxinfo$
mesa-demos.tcz
real 0m 0.47s
user 0m 0.42s
sys 0m 0.04s
tc@E310:~/Scripting/Provides$tc@E310:~/Scripting/Provides$ time ./provides.sh -nz glxinfo$
mesa-demos.tcz
real 0m 0.21s
user 0m 0.16s
sys 0m 0.04s
tc@E310:~/Scripting/Provides$ time ./provides.sh -nz glxinfo$
mesa-demos.tcz
real 0m 0.22s
user 0m 0.18s
sys 0m 0.03s
tc@E310:~/Scripting/Provides$I override the busybox default and use GNU awk if I detect it's present.... The newer version of busybox supports -mmin option. ...The current version of busybox on TC10 x86 does too.
That's pretty clever, but was the intent to override the -nz commandCode: [Select]+[ $NZ -eq 1 ] || [ -n "$(find $LIST -mmin +15)" ] && UpdateProvidesDB
Maybe it makes more sense to remove the -nz command line switch
and just let it run a check when the file is at least 15 minutes old.
I would like to hear some opinions on this please.
... I'd also suggest showing the help info if provides.sh is run with no arguments.I initially did that, but then reverted back to exit 1 because I didn't want to
Hi, Rich.QuoteThat's pretty clever, but was the intent to override the -nz commandCode: [Select]+[ $NZ -eq 1 ] || [ -n "$(find $LIST -mmin +15)" ] && UpdateProvidesDB
line switch every 15 minutes?
Maybe it makes more sense to remove the -nz command line switch
and just let it run a check when the file is at least 15 minutes old.
I would like to hear some opinions on this please.
--- provides.sh
+++ /usr/local/bin/provides.sh
@@ -18,6 +18,7 @@
# --------------------------------------------------------------- #
UpdateProvidesDB()
{
+[ -n "$(find $LIST -mmin +15)" ] || return 0
getMirror
cd "$TCEDIR"
if zsync -i "$LIST" -q "$MIRROR"/"$DB".zsync
@@ -32,6 +33,7 @@
fi
chmod g+rw "$DB"
cd - > /dev/null
+touch $LIST
}
# --------------------------------------------------------------- #
time provides.sh stupid-filename-nz
real 0m 0.13s
user 0m 0.11s
sys 0m 0.02s
... For instance, this would trigger the -nz switch: ...Actually, the way it's coded the -nz switch will not trigger:
tc@E310:~/Scripting/Provides$ time ./provides.sh stupid-filename-nz
NZ=1 TARGET=stupid-filename
real 0m 1.74s
user 0m 0.48s
sys 0m 0.22s
tc@E310:~/Scripting/Provides$As you can see, NZ is still set to 1. However, because of the way -nz istc@E310:~/Scripting/Provides$ time ./provides.sh -nzstupid-filename-nz
NZ=1 TARGET=-nzstupid-filename-nz
real 0m 1.19s
user 0m 0.41s
sys 0m 0.23s
tc@E310:~/Scripting/Provides$Both a leading and trailing -nz in a filename are safe. And it can still... As for the -nz switch, I incline to remove it cause extra code results in extra chance for unintentional behavior. ...I'm leaning in that direction too. Adding the file age test to UpdateProvidesDB() is
... Are you planning to upload the newer version into the Github Core-scripts repo ?I'd like it to be added to Core-scripts. We'll see what curaga has to say about it.
--- provides.sh
+++ /usr/local/bin/provides.sh
@@ -18,6 +18,7 @@
# --------------------------------------------------------------- #
UpdateProvidesDB()
{
+[ -f $LIST ] && [ -z "$(find $LIST -mmin +15)" ] && return 0
getMirror
cd "$TCEDIR"
if zsync -i "$LIST" -q "$MIRROR"/"$DB".zsync
@@ -32,6 +33,7 @@
fi
chmod g+rw "$DB"
cd - > /dev/null
+touch $LIST
}
# --------------------------------------------------------------- #
I'd like to see if we get any more comments on this.Hi Rich. I love your improvements to provides.sh. Thank you. I always thought that script could be easier to use.
Here's a loophole that I missed ...Looks like you plugged it, well done.
Hi Rich. I love your improvements to provides.sh. Thank you. I always thought that script could be easier to use. ...That was the plan:
... I'd say if provides.db is older than 24 hours and there is a network connection, to go ahead and automatically update provides.db. ...I guess we could add something like this to the beginning of UpdateProvidesDB():
ping -W 1 -c 1 8.8.8.8 2>&1 > /dev/null || return 1It doesn't check DNS, but it's fast and will stop you from trying to access... Maybe provide a switch if user wants to update provides.db regardless of its age? ...Instead of a command line switch, an environmental variable to override the
PROVIDESUPDATE=-0PROVIDESUPDATE=+0PROVIDESUPDATE=+1440... It would be nice for no arguments to cause help to be printed, but I understand your reason for not doing that. The -h and --help are working as expected.I also include -help as an option. :P
I guess we could add something like this to the beginning of UpdateProvidesDB():Why not go with Robert's approach (which shows up in tc-config and settime.sh ):Code: [Select]ping -W 1 -c 1 8.8.8.8 2>&1 > /dev/null || return 1
ifconfig | grep -q Bcast || return 1This way there are no hardcoded ip addresses, no calls to Google, no danger of breakage if firewall blocks icmp traffic. Plus, if Busybox's ifconfig someday changes Bcast to some other string, tc-config, settime.sh, and provides.sh would all need an identical repair.... Why not go with Robert's approach (which shows up in tc-config and settime.sh ):Funny you should mention it. That will detect the NIC was brought up.Code: [Select]ifconfig | grep -q Bcast || return 1...
tc@box:~$ ifconfig eth0 | grep Bcast
inet addr:192.168.1.45 Bcast:192.168.1.255 Mask:255.255.255.0
tc@box:~$ sudo ifconfig eth0 down
tc@box:~$ ifconfig eth0 | grep Bcast
inet addr:192.168.1.45 Bcast:192.168.1.255 Mask:255.255.255.0
tc@box:~$ifconfig eth0 | grep RUNNINGThat will detect taking down the NIC and unplugging the network cable.getMirror
DOMAIN="$(echo "$MIRROR" | busybox awk -F[/:] '{print $4}')"
nslookup "$DOMAIN" >/dev/null 2>&1 || return 1ping -W 1 -c 1 8.8.8.8 2>&1 > /dev/nullIt returns instantly with connectivity.ping -W 1 -c 1 google.com 2>&1 > /dev/nullnslookup "repo.tinycorelinux.net" 2>&1 > /dev/null
Hi GNUserHi Rich. Thanks for the tests. Sounds like this one is both fast and reliable, so it gets my vote despite the two cosmetic issues (hardwired address, big company) I mentioned previously.
This timed out in 1 second with no connectivity:Code: [Select]ping -W 1 -c 1 8.8.8.8 2>&1 > /dev/nullIt returns instantly with connectivity.
Hi CNK... I'd also suggest showing the help info if provides.sh is run with no arguments.I initially did that, but then reverted back to exit 1 because I didn't want to
risk changing a behavior that a program or script might be relying on.
... The 1s ping might be so short it gives false negatives sometimes, ... ... But I guess I can always make my own version too.This is not cast in stone. That's why I requested comments.
ping -A -W 1 -c 2 8.8.8.8 2>&1 > /dev/nullIt timed out in 2 seconds with no connectivity and still-A Ping as soon as reply is recevied... Maybe only output the help info when called with no arguments if "[ -t 1 ]" is true, ...Done.
... No autoupdate would also be a gotcha for less experienced users. ...There will be an auto update (zsync) by default for that very reason. I think
... P.S. Folks that want maximum speed or more granular control can use the environmental variable you proposed to stop db updates (and/or you could add a "no db update" flag).That's already in place. Including -nz (no zsync) in the command tells
... The 1s ping might be so short it gives false negatives sometimes ...
If I change it to this:Code: [Select]ping -A -W 1 -c 2 8.8.8.8 2>&1 > /dev/null
It timed out in 2 seconds with no connectivity and still
returned instantly with connectivity.
The -A flag made the instant response possible:Code: [Select]-A Ping as soon as reply is recevied
ping: invalid option -- 'A'
... Latest version attached. Give it a shake and let me know if any bugs fall out. ;D ...I did some more testing and found it ignored + signs and choked on the [ character.
tc@E310:~/Scripting/Provides$ ./provides.sh be@latin.strings
abiword.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh corrupt_t1#P#p1.MYI
mariadb-10.3-test.tcz
mariadb-10.4-test.tcz
mariadb-test.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh 100%_Opaque.myb
mypaint-brushes.tcz
mypaint.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh 9vx-iso
9vx.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh [
coreutils.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh {972ce4c6-7e08-4474-a285-3208198ce6fd}
firefox-ESR.tcz
icecat.tcz
kompozer.tcz
minefield21.tcz
palemoon.tcz
seamonkey-noSSE2.tcz
seamonkey.tcz
thunderbird.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh NetSNMP::ASN.3
net-snmp-dev.tcz
net-snmp-doc.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh ISO_8859-1,GL.gz
glibc_i18n_locale.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh avahi-ui-gtk3~1.desktop
avahi-ui-gtk3.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh c++filt
binutils.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh AAC_\(Nero_FAAC\).txt
deadbeef.tcz
tc@E310:~/Scripting/Provides$As you can see, it handles @ # % _ - [ { } : , ~ and +.... I'll start looking into adding an environmental variable for advanced users.Accommodations for a PROVIDESUPDATE environmental variable have been added.
tc@pi4:/mnt/mmcblk0p2/notes/Shell-Tricks$ cat hex.awk
#!/usr/bin/awk -f
function hex(h,_,r,cmd) {
cmd = "printf \x27" h "\x27 | hexdump -e \x272/1 \x22%X\x22\x27"
cmd | getline r
close(cmd)
return r
}
function qry2ptn(str,_,i,c,s) {
for (i=1;i<=length(str);i++){
c = substr(str,i,1)
if(c ~ /[[:alnum:]]/) s = s c
else {
if(c ~ /\x27/) s = s "\x5Cx27" # single quote
else if(c ~ /\x25/) s = s "\x5Cx25" # %
else s = s "\x5Cx" hex(c)
}
}
return s
}
{
print qry2ptn($0)
}
$ printf '\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2A\x2B\x2C\x2D\x2E\x2F\x3A\x3B\x3C\x3D\x3E\x3F\x40\x5B\x5C\x5D\x5E\x5F\x60\n' > ptn
$ cat ptn
!"#$%&'()*+,-./:;<=>?@[\]^_`
$ ./hex.awk ptn
\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2A\x2B\x2C\x2D\x2E\x2F\x3A\x3B\x3C\x3D\x3E\x3F\x40\x5B\x5C\x5D\x5E\x5F\x60
That was the plan:
Use a / to mark the beginning of the search term if desired.
Type in your search term.
Use a $ sign to mark the end of the search term if desired.
Hit enter.
! # % + , - . / : = ? @ [ ] ^ _ { } ~ and blank spacesMost appear in various provides.db files.< > & * ; ' " `The shell will try to act on them and may do bad things.( )tc@E310:~/Scripting/Provides$ grep '(' /etc/sysconfig/tcedir/provides.db
usr/local/share/gtk-doc/html/cairo/cairo-Quartz-(CGFont)-Fonts.html
/usr/local/lib/deadbeef/convpresets/AAC_(Nero_FAAC).txt
/usr/local/lib/deadbeef/convpresets/FLAC_(compression_level_5).txt
/usr/local/lib/deadbeef/convpresets/MP3_CBR_320_Kbps_(Lame).txt
/usr/local/lib/deadbeef/convpresets/MP3_VBR_192Kbps_(Lame).txt
/usr/local/lib/deadbeef/convpresets/Ogg_Vorbis_(-q_5).txt
/usr/local/share/vim/vim82/lang/menu_chinese(gb)_gb.936.vim
/usr/local/share/vim/vim82/lang/menu_chinese(taiwan)_taiwan.950.vim
/usr/local/share/mypaint-data/1.0/brushes/kaerhon_v1/smudge_ink(0.7)_sm.myb
/usr/local/share/mypaint-data/1.0/brushes/kaerhon_v1/smudge_ink(0.7)_sm_prev.png
/usr/local/lib/python3.6/site-packages/setuptools/script (dev).tmpl
/usr/local/lib/python2.7/site-packages/setuptools-39.0.1-py2.7.egg/setuptools/script (dev).tmpl
usr/local/share/vim/vim72/lang/menu_chinese(gb)_gb.936.vim
usr/local/share/vim/vim72/lang/menu_chinese(taiwan)_taiwan.950.vim
usr/local/share/gtk-doc/html/libxfce4panel-1.0/libxfce4panel-Panel-Plugin-Register-Macros-(4.6-Style).html
tc@E310:~/Scripting/Provides$... The only thing that needs adjusting is the leading "^" ...There is no leading ^ in this type of search. All filenames include a path, so
tc@E310:~/Scripting/Provides$ ./provides.sh apps | wc -l
95That's way too many. To narrow the search, you add an anchor:tc@E310:~/Scripting/Provides$ ./provides.sh /apps | wc -l
78That narrowed it some, but you know it's a program, so you extend the anchor:tc@E310:~/Scripting/Provides$ ./provides.sh bin/apps
Xprogs.tcz
tc@E310:~/Scripting/Provides$ ./provides.sh
Version 0.3 Nov 25, 2024
Find extension(s) that provide a filename.
Filenames in list being searched include full paths, for example:
usr/local/bin/grep
Usage:
provides.sh [ -nz ] FileName
-nz Skip updating (zsync) the provides.db file. This speeds up
the search, but might miss items if provides.db is outdated.
Examples:
provides.sh cal Finds cal anywhere in FileName
provides.sh bin/cal Finds bin/cal anywhere in FileName
provides.sh bin/cal$ Finds FileName that ends in bin/cal
provides.sh Black Gnome Finds FileName with embedded spaces
Searches are case sensitive.
The $ sign can only be used at the end of your search term.
tc@E310:~/Scripting/Provides$Sorry for not getting the performance working with sqlite3 :( ---
Tried to get it working so the command line should report the performance.
I can't getting it working with the ".timer on" with the command line version of sqlite3.
The ".timer on" doesn't report the speed sadly when executing from the command line.
But if you get into the interactive mode of sqlite3 it just works so you get a report how many seconds the question took.
It says in "man" you could supply the -cmd ".timer on" but it doesn't work for me, but some another commands like -cmd ".help" works out of the box.
And if "you" choose sqlite3 you could also store some more metadata like version, date and so on.
Maybe also store ldd output of the executable, so you can easily see the dependency's.
echo "SELECT * FROM tcz_files WHERE file LIKE '%$1%';" | sqlite3 -cmd ".timer on" -box files.dbcat << 'EOF' > make.sh
#!/bin/bash
sqlite3 files.db "CREATE TABLE tcz_files (name TEXT NOT NULL,file TEXT NOT NULL);"
for filename in $(ls *.tcz)
do
tczfile=$filename;for file in $(unsquashfs -lc $tczfile); do sqlite3 files.db "INSERT INTO tcz_files ( NAME , FILE ) VALUES ( '$tczfile' , '$(basename $file)' );" ; done
done
EOF
chmod u+x make.shcat << 'EOF' > search.sh
#!/bin/bash
echo "SELECT * FROM tcz_files WHERE file LIKE '%$1%';" | sqlite3 -cmd ".timer on" -box files.db
EOF
chmod u+x search.shTitle: provides.tcz
Description: Enhanced version of Tinycore provides.sh.
Version: 0.3
Author: Robert Shingledecker
Original-site: https://github.com/tinycorelinux/Core-scripts/blob/master/usr/bin/provides.sh
Copying-policy: GPLv2
Size: 4.0K
Extension_by: Rich
Tags: find filename extension
Comments: Run provides.sh for this help screen:
Version 0.3 Nov 25, 2024
Find extension(s) that provide a filename.
Filenames in list being searched include full paths, for example:
usr/local/bin/grep
Usage:
provides.sh [ -nz ] FileName
-nz Skip updating (zsync) the provides.db file. This speeds up
the search, but might miss items if provides.db is outdated.
Examples:
provides.sh cal Finds cal anywhere in FileName
provides.sh bin/cal Finds bin/cal anywhere in FileName
provides.sh bin/cal$ Finds FileName that ends in bin/cal
provides.sh Black Gnome Finds FileName with embedded spaces
Searches are case sensitive.
The $ sign can only be used at the end of your search term.
-------------------------------------------------------------
Note: The default age of provides.db for checking if an update
is available is 15 minutes.
Environmental variable PROVIDESUPDATE can override this default.
Add export PROVIDESUPDATE=+N to .profile file.
log out, log in.
N must be an integer.
Set to -0 to block updates.
Set to +0 for frequent updates.
Set to +N where N is the update interval in minutes.
Ex. PROVIDESUPDATE=+1440 for 24 hours.
Change-log: 2024/11/25 Original release v0.30
Current: 2024/11/25 Original release v0.30I hope that's the plan for TCL16.Hi, GNUser.
... May I submit a pull request on your behalf? ...Yes, by all means, please.
... it seems final decision is Juanito's and he hasn't weighed in one way or the other ...I thought he did:
I’d prefer to continue using /usr for everything in the base and /usr/local for extensions.
Depending on how you read that, Juanito's preference would be best honored by implementing /usr merge.Quote... it seems final decision is Juanito's and he hasn't weighed in one way or the other ...I thought he did:I’d prefer to continue using /usr for everything in the base and /usr/local for extensions.
Pull request has been made: https://github.com/tinycorelinux/Core-scripts/pull/70 ...Thank you.
... Current situation: Base uses not only /usr/bin, /usr/lib, and /usr/sbin but also /bin, /lib, and /sbin. /usr/local used for extensions.That reminded me of this:
/usr merge situation: Base uses /usr for everything. /bin is just a link to /usr/bin, /lib is just a link to /usr/lib, /sbin is just a link to /usr/sbin. /usr/local for extensions. ...
sbin still has a purpose, root-requiring stuff, so not sure on that one.
Yes, at least sbin has a purpose. Debian/Devuan did not do sbin merge (i.e., kept /usr/sbin as a real directory).