WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Recursive Dependency size.  (Read 13344 times)

Offline ixbrian

  • Retired Admins
  • Sr. Member
  • *****
  • Posts: 436
Re: Recursive Dependency size.
« Reply #15 on: July 28, 2010, 09:55:23 PM »
You should be able to work with just the base, use nc (netcat) in place of curl:

Quote
echo -e "HEAD $FILE HTTP/1.0\n" | nc $MIRROR 80

MIRROR being the host name (distro.ibiblio.org), and FILE the full path (/pub/linux/distributions/tinycorelinux/2.x/tcz/firefox.tcz)

curaga,
Very nice!  I have updated the script with your suggestion and it should now work with just the base Tiny Core. 

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11044
Re: Recursive Dependency size.
« Reply #16 on: July 29, 2010, 05:23:28 AM »
I hacked on the script some more:

Removed the temp file (var instead), used more shell builtins: ./depinfo.sh firefox.tcz is now 28% faster on a slow comp ;)

edit:
I had turned spaces to tabs to save ~200 bytes and look better, but it looks worse pasted here..

Quote
#!/bin/sh
# depinfo.sh - Copyright 2010 Brian Smith
# Some edits by Curaga
# Licensed under GPLv2 License
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.

alias grep="busybox grep"
alias awk="busybox awk"
alias wget="busybox wget"
alias expr="busybox expr"
alias sed="busybox sed"
alias dc="busybox dc"
alias nc="busybox nc"

get_dependencies()
{
   app=${1//KERNEL/$kernel}

   case "$tempfile" in
   *"$app"*)
   ;;
   *)
      tempfile="$tempfile $app"
      size=`echo -e "HEAD /$server_path/$app HTTP/1.0\n" | nc $server 80 | grep "^Content-Length" | awk '{print $2'}`

      case $size in
      [0-9]*)
         totalsize=$(($totalsize + $size))
         sizemb=`dc $size 1024 / 1024 / p`
         printf "%-40s size(bytes): %10d, %6.2f MB\n" $app $size $sizemb

         deplist=`wget -q -O - "$MIRROR"/"$app.dep" 2>/dev/null`
         for depapp in $deplist; do
            get_dependencies $depapp
         done
      ;;
      *)
         printf "%-40s Error, not found \n" $app
      ;;
      esac
   ;;
   esac
}

app=$1

if [ -n "$app" ]; then
   . /etc/init.d/tc-functions
   getMirror
   server=`echo $MIRROR | sed -e "s/http:\/\///g`
   server_path=${server#*/}
   server=${server%%/*}

   kernel=`uname -r`
   totalsize=0
   get_dependencies $app

   totalsizemb=`dc $totalsize 1024 / 1024 / p`
   printf "\nTotal size (bytes): %10d, %6.2f MB\n\n" $totalsize $totalsizemb
else
   echo "Specify extension in command line:"
   echo "Example:   $0 firefox.tcz"
fi
« Last Edit: July 29, 2010, 05:29:01 AM by curaga »
The only barriers that can stop you are the ones you create yourself.

Offline ixbrian

  • Retired Admins
  • Sr. Member
  • *****
  • Posts: 436
Re: Recursive Dependency size.
« Reply #17 on: July 29, 2010, 08:35:09 AM »
I hacked on the script some more:

Removed the temp file (var instead), used more shell builtins: ./depinfo.sh firefox.tcz is now 28% faster on a slow comp ;)

edit:
I had turned spaces to tabs to save ~200 bytes and look better, but it looks worse pasted here..

Quote
#!/bin/sh
# depinfo.sh - Copyright 2010 Brian Smith
# Some edits by Curaga
# Licensed under GPLv2 License
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.

alias grep="busybox grep"
alias awk="busybox awk"
alias wget="busybox wget"
alias expr="busybox expr"
alias sed="busybox sed"
alias dc="busybox dc"
alias nc="busybox nc"

get_dependencies()
{
   app=${1//KERNEL/$kernel}

   case "$tempfile" in
   *"$app"*)
   ;;
   *)
      tempfile="$tempfile $app"
      size=`echo -e "HEAD /$server_path/$app HTTP/1.0\n" | nc $server 80 | grep "^Content-Length" | awk '{print $2'}`

      case $size in
      [0-9]*)
         totalsize=$(($totalsize + $size))
         sizemb=`dc $size 1024 / 1024 / p`
         printf "%-40s size(bytes): %10d, %6.2f MB\n" $app $size $sizemb

         deplist=`wget -q -O - "$MIRROR"/"$app.dep" 2>/dev/null`
         for depapp in $deplist; do
            get_dependencies $depapp
         done
      ;;
      *)
         printf "%-40s Error, not found \n" $app
      ;;
      esac
   ;;
   esac
}

app=$1

if [ -n "$app" ]; then
   . /etc/init.d/tc-functions
   getMirror
   server=`echo $MIRROR | sed -e "s/http:\/\///g`
   server_path=${server#*/}
   server=${server%%/*}

   kernel=`uname -r`
   totalsize=0
   get_dependencies $app

   totalsizemb=`dc $totalsize 1024 / 1024 / p`
   printf "\nTotal size (bytes): %10d, %6.2f MB\n\n" $totalsize $totalsizemb
else
   echo "Specify extension in command line:"
   echo "Example:   $0 firefox.tcz"
fi

Curaga,
Looks great!   Thanks for the improvements, I like your scripting style. 

Thanks again,
Brian

Offline ixbrian

  • Retired Admins
  • Sr. Member
  • *****
  • Posts: 436
Re: Recursive Dependency size.
« Reply #18 on: July 29, 2010, 05:22:15 PM »
I have updated my original posting with an updated script.   It has Curaga's improvements and I also added in a couple of new features:

- Extensions that are already installed on your system will be green when listed.
- At the bottom, it will display the total size, how much of that is already installed, and how much would need to be downloaded if you installed the extension (example below)

Code: [Select]
           Total size (bytes)       35753984,  34.10 MB
    (Not currently installed)       27631616,  26.35 MB
          (Already installed)        8122368,   7.75 MB

Brian

Offline jur

  • Hero Member
  • *****
  • Posts: 863
    • cycling photo essays
Re: Recursive Dependency size.
« Reply #19 on: July 29, 2010, 06:18:35 PM »
Could you please attach the script file as well to the 1st post? Trouble is I am mostly at a windows terminal and copy-paste does not work, all the line feeds are a problem in Win.

Thanks!

Offline tinypoodle

  • Hero Member
  • *****
  • Posts: 3857
Re: Recursive Dependency size.
« Reply #20 on: July 29, 2010, 07:02:21 PM »
- Extensions that are already installed on your system will be green when listed.
No green neither any other way to distinguish here...  :-\ (Total at bottom working as expected  :))
"Software gets slower faster than hardware gets faster." Niklaus Wirth - A Plea for Lean Software (1995)

Offline ixbrian

  • Retired Admins
  • Sr. Member
  • *****
  • Posts: 436
Re: Recursive Dependency size.
« Reply #21 on: July 29, 2010, 07:21:31 PM »
- Extensions that are already installed on your system will be green when listed.
No green neither any other way to distinguish here...  :-\ (Total at bottom working as expected  :))

Did you take out the "   . /etc/init.d/tc-functions" line of the script when you defined the MIRROR variable for Tiny Core 2.X?  Try putting back in the " . /etc/init.d/tc-functions" and see if the green color works then.

Brian

Offline tinypoodle

  • Hero Member
  • *****
  • Posts: 3857
Re: Recursive Dependency size.
« Reply #22 on: July 29, 2010, 11:25:54 PM »
FIXED, Thank you  ;D

getting close to apt-get in functionality, minus the bloat  :)

Suggestion: As the suffix ".tcz" is common to every package, would it be possible to eliminate the need to specify it on command line each time?
"Software gets slower faster than hardware gets faster." Niklaus Wirth - A Plea for Lean Software (1995)

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11044
Re: Recursive Dependency size.
« Reply #23 on: July 30, 2010, 04:31:13 AM »
Let's get rid of the last sed as well ;)

Quote
server=`echo $MIRROR | sed -e "s/http:\/\///g`
to
Quote
server=${MIRROR#http://}


Quote
getting close to apt-get in functionality, minus the bloat

Apt-get didn't have color IIRC, so we already surpass it. :P
The only barriers that can stop you are the ones you create yourself.

Offline ixbrian

  • Retired Admins
  • Sr. Member
  • *****
  • Posts: 436
Re: Recursive Dependency size.
« Reply #24 on: July 30, 2010, 08:09:37 AM »
FIXED, Thank you  ;D

getting close to apt-get in functionality, minus the bloat  :)

Suggestion: As the suffix ".tcz" is common to every package, would it be possible to eliminate the need to specify it on command line each time?

That's a good idea.   I will modify the script so you can specify the extension with the .tcz extension or without it and get a new version posted tonight. 

I'll also get rid of that last sed line like Curaga suggested. 

Thanks,
Brian

Offline ixbrian

  • Retired Admins
  • Sr. Member
  • *****
  • Posts: 436
Re: Recursive Dependency size.
« Reply #25 on: July 30, 2010, 09:21:47 PM »
Suggestion: As the suffix ".tcz" is common to every package, would it be possible to eliminate the need to specify it on command line each time?

This has been added in, you can now either specify the extension with .tcz or without.   Thanks for the suggestion.   I also replaced the last "sed" reference as Curaga suggested (thanks Curaga). 

I modified the original posting with the updated script. 

Brian

Offline tinypoodle

  • Hero Member
  • *****
  • Posts: 3857
Re: Recursive Dependency size.
« Reply #26 on: August 08, 2010, 06:55:45 AM »
The script seems to ignore the gtk2.tcz dep of MPlayer-svn-gtk2.tcz (in 2.x) while deptree.sh does pick it up.
"Software gets slower faster than hardware gets faster." Niklaus Wirth - A Plea for Lean Software (1995)

Offline ixbrian

  • Retired Admins
  • Sr. Member
  • *****
  • Posts: 436
Re: Recursive Dependency size.
« Reply #27 on: August 08, 2010, 10:31:37 AM »
The script seems to ignore the gtk2.tcz dep of MPlayer-svn-gtk2.tcz (in 2.x) while deptree.sh does pick it up.

Good catch.   There is a problem with the way it is determining if the extension is a duplicate.  I have a couple ideas on how to correct this and should be able to post a fix this afternoon when I have a few minutes to try it out. 

Thanks,
Brian

Offline ixbrian

  • Retired Admins
  • Sr. Member
  • *****
  • Posts: 436
Re: Recursive Dependency size.
« Reply #28 on: August 08, 2010, 10:46:44 AM »
The script seems to ignore the gtk2.tcz dep of MPlayer-svn-gtk2.tcz (in 2.x) while deptree.sh does pick it up.

I updated my original post with a new version that fixes this issue.  Thanks for finding this bug.   The problem was with how it detected if the extension was a duplicate.

Thanks again,
Brian

Offline tinypoodle

  • Hero Member
  • *****
  • Posts: 3857
Re: Recursive Dependency size.
« Reply #29 on: September 09, 2010, 12:33:03 PM »
When connection to server is less than ideal, it fails upon some single items in deps, e.g.
Code: [Select]
tc@box:~$ depinfo7.sh VBox-OSE
VBox-OSE.tcz                             size(bytes):   11202560,  10.68 MB
hal_support.tcz                          size(bytes):      40960,   0.04 MB
nc: bad address 'distro.ibiblio.org'
libcap.tcz                               Error, not found
libxslt.tcz                              size(bytes):     303104,   0.29 MB
libxml2.tcz                              size(bytes):     696320,   0.66 MB
nc: bad address 'distro.ibiblio.org'
libIDL.tcz                               Error, not found
SDL.tcz                                  size(bytes):     380928,   0.36 MB
-----------------------snip------------------------------------------

Would there be any way to add something like more "grace time"?
"Software gets slower faster than hardware gets faster." Niklaus Wirth - A Plea for Lean Software (1995)