Tiny Core Linux
General TC => Programming & Scripting - Unofficial => Topic started by: richardh on July 27, 2010, 05:06:20 AM
-
Is there anything in AppBrowser/AppAudit which will tell me the size of an extention and all of its depedencies? If not does anyone have a script which will do this? I have searched the forums but I couldn't find anything relevant.
Many thanks,
Richard.
-
Here is a script that will calculate the total size including dependencies.
#!/bin/sh
# depinfo.sh - Copyright 2010 Brian Smith
# Some edits by Curaga
# Licensed under GPLv2 License
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
alias grep="busybox grep"
alias awk="busybox awk"
alias wget="busybox wget"
alias dc="busybox dc"
alias nc="busybox nc"
get_dependencies()
{
app=${1//KERNEL/$kernel}
case "$tempfile" in
*":${app}:"*)
;;
*)
tempfile="${tempfile}${app}:"
size=`echo -e "HEAD /$server_path/$app HTTP/1.0\n" | nc $server 80 | grep "^Content-Length" | awk '{print $2'}`
case $size in
[0-9]*)
sizemb=`dc $size 1024 / 1024 / p`
if [ -f "$localtce"/optional/"$app" ]; then
totalsize_installed=$(($totalsize_installed + $size))
echo -n "${GREEN}"
printf "%-40s" $app
echo -n "${NORMAL}"
printf " size(bytes): %10d, %6.2f MB\n" $size $sizemb
else
totalsize_needed=$(($totalsize_needed + $size))
printf "%-40s size(bytes): %10d, %6.2f MB\n" $app $size $sizemb
fi
deplist=`wget -q -O - "$MIRROR"/"$app.dep" 2>/dev/null`
for depapp in $deplist; do
get_dependencies $depapp
done
;;
*)
printf "%-40s Error, not found \n" $app
;;
esac
;;
esac
}
app=$1
if [ -n "$app" ]; then
app=${app%.tcz}
app="$app.tcz"
localtce=`cat /opt/.tce_dir`
. /etc/init.d/tc-functions
getMirror
server=${MIRROR#http://}
server_path=${server#*/}
server=${server%%/*}
kernel=`uname -r`
totalsize_needed=0
totalsize_installed=0
tempfile=":"
get_dependencies $app
totalsize=$(($totalsize_needed + $totalsize_installed))
totalsizemb=`dc $totalsize 1024 / 1024 / p`
totalsize_neededmb=`dc $totalsize_needed 1024 / 1024 / p`
totalsize_installedmb=`dc $totalsize_installed 1024 / 1024 / p`
printf "\n Total size (bytes) %10d, %6.2f MB\n" $totalsize $totalsizemb
printf " (Not currently installed) %10d, %6.2f MB\n" $totalsize_needed $totalsize_neededmb
printf " (Already installed) %10d, %6.2f MB\n\n" $totalsize_installed $totalsize_installedmb
else
echo "Specify extension in command line:"
echo "Example: $0 firefox.tcz"
fi
When you run it, it produces the following output:
$ ./depinfo.sh firefox.tcz
firefox.tcz size(bytes): 13664256, 13.03 MB
libasound.tcz size(bytes): 372736, 0.36 MB
curl.tcz size(bytes): 380928, 0.36 MB
openssl-0.9.8.tcz size(bytes): 1118208, 1.07 MB
libnotify.tcz size(bytes): 94208, 0.09 MB
dbus-glib.tcz size(bytes): 167936, 0.16 MB
dbus.tcz size(bytes): 520192, 0.50 MB
expat2.tcz size(bytes): 77824, 0.07 MB
glib2.tcz size(bytes): 1040384, 0.99 MB
gtk2.tcz size(bytes): 3182592, 3.04 MB
atk.tcz size(bytes): 53248, 0.05 MB
cairo.tcz size(bytes): 274432, 0.26 MB
pixman.tcz size(bytes): 200704, 0.19 MB
fontconfig.tcz size(bytes): 135168, 0.13 MB
graphics-libs-1.tcz size(bytes): 790528, 0.75 MB
pango.tcz size(bytes): 405504, 0.39 MB
libxml2.tcz size(bytes): 696320, 0.66 MB
Xorg-7.5-lib.tcz size(bytes): 204800, 0.20 MB
Total size (bytes): 23379968, 22.30 MB
-
tc@box:~$ depsize.sh firefox.tcz
/home/tc/.local/bin/depsize.sh: line 38: getMirror: not found
expr: syntax error
firefox.tcz size(bytes): sh: : invalid number
0
Total size: bytes
:'(
Using 2.10, in case that would matter
-
Updated script, it will now display an indented tree listing of the dependencies
As is, the script will only work on Tiny Core 3.x. If you want to run it on 2.x, try replacing the 2 lines:
. /etc/init.d/tc-functions
getMirror
With this single line:
MIRROR="http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/2.x/tcz"
-
Updated script, it will now display an indented tree listing of the dependencies
Umm, and where is the update? ???
As is, the script will only work on Tiny Core 3.x. If you want to run it on 2.x, try replacing the 2 lines:
. /etc/init.d/tc-functions
getMirror
With this single line:
MIRROR="http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/2.x/tcz"
Heh, i was halfway there having tried just "http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux" instead :P
Seems to work fine now with this modification. :)
Thank you so much for this invaluable tool, this is brilliant! ;D
I think this will help a lot in taking decisions of installing apps and also might even be of use for discussions in the forum, when now instead of estimating/speculating one could just quickly check total size including all deps. I take it all deps are proceeded recursively.
-
Umm, and where is the update? ???
See my original post in this thread, I just updated it with the updated script.
-
Ohhh... thank you, gotcha now! ::)
However, i made following observation:
An extension may be listed multiply in proceeding of recursive deps.
A rather random example: When doing "./depsize.sh epiphany.tcz", e.g. gtk2.tcz (and in consequence all its deps) is listed in multiple instances. Haven't made any calculations, but suspecting all instances will be added to total.
-
Ohhh... thank you, gotcha now! ::)
However, i made following observation:
An extension may be listed multiply in proceeding of recursive deps.
A rather random example: When doing "./depsize.sh epiphany.tcz", e.g. gtk2.tcz (and in consequence all its deps) is listed in multiple instances. Haven't made any calculations, but suspecting all instances will be added to total.
Good catch. I posted a new version that fixes this (and I also dropped the dependency tabbed tree display)
-
thanks great tool, i have a bug to report :)
your script does not handle KERNEL entries in dep files, you should replace KERNEL with `uname -r`
tc@box:~$ ./depinfo lm_sensors.tcz
lm_sensors.tcz size(bytes): 131072
expr: syntax error
hwmon-KERNEL.tcz size(bytes): sh: : invalid number
0
Total size: bytes
-
Thanks for that - this will be very useful in comparing packages.
-
thanks great tool, i have a bug to report :)
your script does not handle KERNEL entries in dep files, you should replace KERNEL with `uname -r`
tc@box:~$ ./depinfo lm_sensors.tcz
lm_sensors.tcz size(bytes): 131072
expr: syntax error
hwmon-KERNEL.tcz size(bytes): sh: : invalid number
0
Total size: bytes
Works fine here in 2.10, but i note the difference in syntax of deps between extensions of 2.x and 3.x
tc@box:~$ depinfo.sh lmsensors.tcz
lmsensors.tcz size(bytes): 135168
perl5.tcz size(bytes): 12529664
hwmon-2.6.29.1-tinycore.tcz size(bytes): 507904
Total size: 13172736 bytes
-
Would be more readable to display size in kbyte or Mbyte and using thousands separator.
-
I just use
dc $bytes 1024 / p
(where $bytes is number printed on last line of output) to get size in kilobytes.
Quite sure someone with better knowledge than me could manage to grep that number and pipe it to dc.
-
Thanks for the suggestions.
tinypoodle/bmarkus: It now shows the size in bytes and megabytes. Thanks tinypoodle for the "dc" command idea, I used this in the script.
Arslan S.: I have fixed the issue with Kernel dependencies.
I also added in some additional error checking so it will display an error if an extension is not found.
I modified my original posting in this thread with the updated code.
-
You should be able to work with just the base, use nc (netcat) in place of curl:
echo -e "HEAD $FILE HTTP/1.0\n" | nc $MIRROR 80
MIRROR being the host name (distro.ibiblio.org), and FILE the full path (/pub/linux/distributions/tinycorelinux/2.x/tcz/firefox.tcz)
-
You should be able to work with just the base, use nc (netcat) in place of curl:
echo -e "HEAD $FILE HTTP/1.0\n" | nc $MIRROR 80
MIRROR being the host name (distro.ibiblio.org), and FILE the full path (/pub/linux/distributions/tinycorelinux/2.x/tcz/firefox.tcz)
curaga,
Very nice! I have updated the script with your suggestion and it should now work with just the base Tiny Core.
-
I hacked on the script some more:
Removed the temp file (var instead), used more shell builtins: ./depinfo.sh firefox.tcz is now 28% faster on a slow comp ;)
edit:
I had turned spaces to tabs to save ~200 bytes and look better, but it looks worse pasted here..
#!/bin/sh
# depinfo.sh - Copyright 2010 Brian Smith
# Some edits by Curaga
# Licensed under GPLv2 License
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
alias grep="busybox grep"
alias awk="busybox awk"
alias wget="busybox wget"
alias expr="busybox expr"
alias sed="busybox sed"
alias dc="busybox dc"
alias nc="busybox nc"
get_dependencies()
{
app=${1//KERNEL/$kernel}
case "$tempfile" in
*"$app"*)
;;
*)
tempfile="$tempfile $app"
size=`echo -e "HEAD /$server_path/$app HTTP/1.0\n" | nc $server 80 | grep "^Content-Length" | awk '{print $2'}`
case $size in
[0-9]*)
totalsize=$(($totalsize + $size))
sizemb=`dc $size 1024 / 1024 / p`
printf "%-40s size(bytes): %10d, %6.2f MB\n" $app $size $sizemb
deplist=`wget -q -O - "$MIRROR"/"$app.dep" 2>/dev/null`
for depapp in $deplist; do
get_dependencies $depapp
done
;;
*)
printf "%-40s Error, not found \n" $app
;;
esac
;;
esac
}
app=$1
if [ -n "$app" ]; then
. /etc/init.d/tc-functions
getMirror
server=`echo $MIRROR | sed -e "s/http:\/\///g`
server_path=${server#*/}
server=${server%%/*}
kernel=`uname -r`
totalsize=0
get_dependencies $app
totalsizemb=`dc $totalsize 1024 / 1024 / p`
printf "\nTotal size (bytes): %10d, %6.2f MB\n\n" $totalsize $totalsizemb
else
echo "Specify extension in command line:"
echo "Example: $0 firefox.tcz"
fi
-
I hacked on the script some more:
Removed the temp file (var instead), used more shell builtins: ./depinfo.sh firefox.tcz is now 28% faster on a slow comp ;)
edit:
I had turned spaces to tabs to save ~200 bytes and look better, but it looks worse pasted here..
#!/bin/sh
# depinfo.sh - Copyright 2010 Brian Smith
# Some edits by Curaga
# Licensed under GPLv2 License
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
alias grep="busybox grep"
alias awk="busybox awk"
alias wget="busybox wget"
alias expr="busybox expr"
alias sed="busybox sed"
alias dc="busybox dc"
alias nc="busybox nc"
get_dependencies()
{
app=${1//KERNEL/$kernel}
case "$tempfile" in
*"$app"*)
;;
*)
tempfile="$tempfile $app"
size=`echo -e "HEAD /$server_path/$app HTTP/1.0\n" | nc $server 80 | grep "^Content-Length" | awk '{print $2'}`
case $size in
[0-9]*)
totalsize=$(($totalsize + $size))
sizemb=`dc $size 1024 / 1024 / p`
printf "%-40s size(bytes): %10d, %6.2f MB\n" $app $size $sizemb
deplist=`wget -q -O - "$MIRROR"/"$app.dep" 2>/dev/null`
for depapp in $deplist; do
get_dependencies $depapp
done
;;
*)
printf "%-40s Error, not found \n" $app
;;
esac
;;
esac
}
app=$1
if [ -n "$app" ]; then
. /etc/init.d/tc-functions
getMirror
server=`echo $MIRROR | sed -e "s/http:\/\///g`
server_path=${server#*/}
server=${server%%/*}
kernel=`uname -r`
totalsize=0
get_dependencies $app
totalsizemb=`dc $totalsize 1024 / 1024 / p`
printf "\nTotal size (bytes): %10d, %6.2f MB\n\n" $totalsize $totalsizemb
else
echo "Specify extension in command line:"
echo "Example: $0 firefox.tcz"
fi
Curaga,
Looks great! Thanks for the improvements, I like your scripting style.
Thanks again,
Brian
-
I have updated my original posting with an updated script. It has Curaga's improvements and I also added in a couple of new features:
- Extensions that are already installed on your system will be green when listed.
- At the bottom, it will display the total size, how much of that is already installed, and how much would need to be downloaded if you installed the extension (example below)
Total size (bytes) 35753984, 34.10 MB
(Not currently installed) 27631616, 26.35 MB
(Already installed) 8122368, 7.75 MB
Brian
-
Could you please attach the script file as well to the 1st post? Trouble is I am mostly at a windows terminal and copy-paste does not work, all the line feeds are a problem in Win.
Thanks!
-
- Extensions that are already installed on your system will be green when listed.
No green neither any other way to distinguish here... :-\ (Total at bottom working as expected :))
-
- Extensions that are already installed on your system will be green when listed.
No green neither any other way to distinguish here... :-\ (Total at bottom working as expected :))
Did you take out the " . /etc/init.d/tc-functions" line of the script when you defined the MIRROR variable for Tiny Core 2.X? Try putting back in the " . /etc/init.d/tc-functions" and see if the green color works then.
Brian
-
FIXED, Thank you ;D
getting close to apt-get in functionality, minus the bloat :)
Suggestion: As the suffix ".tcz" is common to every package, would it be possible to eliminate the need to specify it on command line each time?
-
Let's get rid of the last sed as well ;)
server=`echo $MIRROR | sed -e "s/http:\/\///g`
to
server=${MIRROR#http://}
getting close to apt-get in functionality, minus the bloat
Apt-get didn't have color IIRC, so we already surpass it. :P
-
FIXED, Thank you ;D
getting close to apt-get in functionality, minus the bloat :)
Suggestion: As the suffix ".tcz" is common to every package, would it be possible to eliminate the need to specify it on command line each time?
That's a good idea. I will modify the script so you can specify the extension with the .tcz extension or without it and get a new version posted tonight.
I'll also get rid of that last sed line like Curaga suggested.
Thanks,
Brian
-
Suggestion: As the suffix ".tcz" is common to every package, would it be possible to eliminate the need to specify it on command line each time?
This has been added in, you can now either specify the extension with .tcz or without. Thanks for the suggestion. I also replaced the last "sed" reference as Curaga suggested (thanks Curaga).
I modified the original posting with the updated script.
Brian
-
The script seems to ignore the gtk2.tcz dep of MPlayer-svn-gtk2.tcz (in 2.x) while deptree.sh does pick it up.
-
The script seems to ignore the gtk2.tcz dep of MPlayer-svn-gtk2.tcz (in 2.x) while deptree.sh does pick it up.
Good catch. There is a problem with the way it is determining if the extension is a duplicate. I have a couple ideas on how to correct this and should be able to post a fix this afternoon when I have a few minutes to try it out.
Thanks,
Brian
-
The script seems to ignore the gtk2.tcz dep of MPlayer-svn-gtk2.tcz (in 2.x) while deptree.sh does pick it up.
I updated my original post with a new version that fixes this issue. Thanks for finding this bug. The problem was with how it detected if the extension was a duplicate.
Thanks again,
Brian
-
When connection to server is less than ideal, it fails upon some single items in deps, e.g.
tc@box:~$ depinfo7.sh VBox-OSE
VBox-OSE.tcz size(bytes): 11202560, 10.68 MB
hal_support.tcz size(bytes): 40960, 0.04 MB
nc: bad address 'distro.ibiblio.org'
libcap.tcz Error, not found
libxslt.tcz size(bytes): 303104, 0.29 MB
libxml2.tcz size(bytes): 696320, 0.66 MB
nc: bad address 'distro.ibiblio.org'
libIDL.tcz Error, not found
SDL.tcz size(bytes): 380928, 0.36 MB
-----------------------snip------------------------------------------
Would there be any way to add something like more "grace time"?
-
When connection to server is less than ideal, it fails upon some single items in deps, e.g.
tc@box:~$ depinfo7.sh VBox-OSE
VBox-OSE.tcz size(bytes): 11202560, 10.68 MB
hal_support.tcz size(bytes): 40960, 0.04 MB
nc: bad address 'distro.ibiblio.org'
libcap.tcz Error, not found
libxslt.tcz size(bytes): 303104, 0.29 MB
libxml2.tcz size(bytes): 696320, 0.66 MB
nc: bad address 'distro.ibiblio.org'
libIDL.tcz Error, not found
SDL.tcz size(bytes): 380928, 0.36 MB
-----------------------snip------------------------------------------
Would there be any way to add something like more "grace time"?
It looks like the busybox "nc" command has a "-w SEC" parameter where you can specify the timeout for connects. You could try updating the nc line to something like this (I haven't tried this out myself), and I'm not sure if that parameter would help if you are having a timeout during DNS name resolution (which the "bad adress" error message might indicate).
size=`echo -e "HEAD /$server_path/$app HTTP/1.0\n" | nc -w 60 $server 80 | grep "^Content-Length" | awk '{print $2'}`
Are you using Tiny Core 2.x? If you are on 3.1rc, Robert updated the script so that it fetches a file from the TCZ repository (sizelist.gz) that has the file sizes for every extension, and it fetches the extension.tree file (i.e. firefox.tce.tree) which lists all of the dependencies. So the script is much faster because it only has to download 2 files.
Brian
-
Thank you very much for all the insight, Brian
Indeed, running TC 2.10
The "-w SEC" parameter of 'nc' doesn't seem to make a difference.
However, passing the resolved IP of the mirror to MIRROR= completely fixes the issue ;D
-
With latest events on ibiblio repo this script ceased to work here - as opposed to deptree.sh which continues to work as per usual.
Workaround:
Commenting out
# getMirrorand adding
MIRROR="http://ftp.nluug.nl/os/Linux/distr/tinycorelinux/3.x/tcz"after that
seems to restore expected functionality ...for the moment.
-
In response to the original question, if you click on the size tab on Appbrowser it will
list the dependencies, the total size, and the total size of the files you actually need to
download. Files listed without a + sign infront of them are already on your system.
-
With latest events on ibiblio repo this script ceased to work here - as opposed to deptree.sh which continues to work as per usual.
Workaround:
Commenting out
# getMirrorand adding
MIRROR="http://ftp.nluug.nl/os/Linux/distr/tinycorelinux/3.x/tcz"after that
seems to restore expected functionality ...for the moment.
Yeah well... that moment is gone now :P