WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: A script to check dependencies  (Read 15832 times)

Offline polikuo

  • Hero Member
  • *****
  • Posts: 723
A script to check dependencies
« on: December 04, 2016, 01:49:14 PM »
I just make a script to check dependencies
It check all dependencies in a directory, that should come in handy when you're compiling  :)
At the same time, it shows which libraries are absent  :o
The result goes to standard output when the absent one goes to standard error

There're 4 different types of output
With option -f, you'll get a full list of sorted dependencies
with option -l, you'll get a long version of tcz list
with option -s, you'll get a short version of tcz list (similar to chkonboot.sh)
with option -n, you'll get a list of libraries that were not found (it goes to standard error)

The idea is quite simple

1. find all files and run ldd to check all dependencies
2. generate a full list
3. remove the files that are under the directory from the full list
4. generate a "NOTFOUND" list
5. generate a full tcz list
6. remove all redundant

Usage: get-dep-list [-option] your_top_directory
  Available Options (Default -ns)
  -f show full dependencies list
  -h show help
  -l show detected tczs (long)
  -n show dependencies that are not found
  -s show detected tczs (short)



Tested on 7.2 x86
Boot base norestore
tce-load only alsa
As you can see, the libsamplerate.so.0 is missing
After loading the libsamplerate.tcz extension fix the problem



By the way ~
Please add as a dependency, thanks.  ;)
However, the KERNEL module were not listed, extensions like these should be handled with care

Code here:
Code: [Select]
#!/bin/sh
# author polikuo
# version 1.0
# Dec 4 2016

. /etc/init.d/tc-functions
useBusybox

showhelp() {
  echo ${YELLOW}
  echo 'Usage: '"$(basename $0)" [-option] your_top_directory
  echo '  'Available Options \(Default -ns\)
  echo '  '\-f show full dependencies list
  echo '  '\-h show help
  echo '  '\-l show detected tczs \(long\)
  echo '  '\-n show dependencies that are not found
  echo '  '\-s show detected tczs \(short\)
  echo ${NORMAL}
  exit 1
}

unset TCEDIR LIBS NOTFOUND ELF TCZS file DEP DELETE not_needed LIST NN NF option B D F L O N S T DIR SELF REMOVE

DIR=false
for O in $@;
do
  echo $O | grep -Eqv "^-" && {
    cd $O 2> /dev/null && { DIR=true; break; } || { echo ${RED}can\'t cd to $O${NORMAL}; break; }
  }
done

(( $DIR )) || showhelp

D=0
F=false
L=false
N=false
S=false
while getopts fhlns option
do
case "$option" in
    f ) F=true; D=$((D+1));;
    h ) showhelp;;
    l ) L=true; D=$((D+1));;
    n ) N=true; D=$((D+1));;
    s ) S=true; D=$((D+1));;
    * ) showhelp;;
esac
done
[ $D -eq 0 ] && { N=true;S=true; }

# Preparing

TCEDIR=/etc/sysconfig/tcedir
ELF="$(find . -not -type d -exec ldd {} + 2> /dev/null | cut -d '(' -f 1 | tr -d '\t' | sort | uniq)"
SELF="$(for B in `find . -not -type d`; do basename $B; done | grep -Ev "\.$")"
LIBS="$(echo "$ELF" | tr -d ' ' | grep -v "notfound" | cut -d '>' -f 2)"
for REMOVE in $SELF
do
  LIBS="$(echo "$LIBS" | grep -v $REMOVE)"
done
NOTFOUND="$(echo "$ELF" | tr -d ' ' | grep "notfound" | cut -d '=' -f 1)"
TCZS="$(ls -l $LIBS 2> /dev/null | grep /tmp/tcloop/ | cut -d '>' -f 2 | cut -d '/' -f 4 | sort | uniq | sed 's/$/\.tcz/g')"

(( $F )) && echo "$ELF" && echo 1>&0
(( $L )) && echo "$TCZS" && echo 1>&0

# checking if TCZS existed

for file in $TCZS
do
  [ -s "$TCEDIR"/optional/"$file" ] || {
    echo ${MAGENTA}$file not found or size is zero${NORMAL}
    echo If you load it elsewhere, please move it and the dep file into your
    echo ${CYAN}$(ls -l /etc/sysconfig | grep tcedir | awk '{ print $11 "optional/" }')${NORMAL}
    exit 1
  }
done

# checking for repeated

chkrpt() {
  for DEP in $TCZS
  do
    if [ -s "$TCEDIR"/optional/"$DEP".dep ]
    then
      for DELETE in $(cat "$TCEDIR"/optional/"$DEP".dep)
      do
        if ( echo "$TCZS" | grep -q "$DELETE" )
        then
          not_needed="$not_needed"" $DELETE"
        fi
      done
    fi
  done

  LIST="$TCZS"

  for NN in `echo $not_needed | tr ' ' '\n' | sort | uniq`
  do
    LIST=`echo $LIST | sed "s/$NN//g"`
  done

  for DEP in $LIST
  do
    echo ${CYAN}$DEP${NORMAL}
  done
}

(( $S )) && chkrpt && echo 1>&0

# Show NOTFOUND

snf() {
  [ "$(echo "$NOTFOUND" | tr -d ' ')" = '' ] || {
  echo ${RED}"These files need to be checked"${NORMAL} 1>&2
  for NF in $NOTFOUND; do
    echo ${MAGENTA}$NF${NORMAL}
  done | sort | uniq 1>&2
  }
}

(( $N )) && snf

I've found many extensions either having missing dependencies or loading too much extensions
I'm afraid I can't list them all, but I hope this script would help

Feedbacks are welcomed  :D
« Last Edit: December 04, 2016, 01:57:17 PM by polikuo »

Offline andyj

  • Hero Member
  • *****
  • Posts: 1036
Re: A script to check dependencies
« Reply #1 on: December 04, 2016, 10:54:42 PM »
1. Not all dependencies are libraries, just as not all extensions are libraries. Be careful when you say extensions seem to have unneeded dependecies.
2. Watch out for circular (mutual) dependencies. Freetype <-> harfbuzz and cyrus-sasl <-> openldap are two that I know of off the top of my head.
3. Many dependencies are optional. This is particularly true for programs that use plugins / extensions / modules like Apache, PHP, LDAP, rsyslog, Postgresql, etc.
4. This requires that copy2fs.flg or copy2fs.lst not be used. Might be a problem in some cases.

Offline bmarkus

  • Administrator
  • Hero Member
  • *****
  • Posts: 7183
    • My Community Forum
Re: A script to check dependencies
« Reply #2 on: December 05, 2016, 12:47:17 AM »
I'm using a similar script based on ldd and it is extremely useful to create a proper .dep file. andyj's comments are correct on the limitations, it helps a lot but you must fine tune result manually.

Few additional notes:

- With ldd you can't detect libraries which are detected by application at runtime and not linked
- Scripts inside of TCZ need their interpreter like BASH, PERL, PYTHON, etc.
- Required kernel modules need attention; most often forgotten is ipv6-KERNEL.tcz

Knowing these limitations it is really a useful tool! :)
Béla
Ham Radio callsign: HA5DI

"Amateur Radio: The First Technology-Based Social Network."

Offline bmarkus

  • Administrator
  • Hero Member
  • *****
  • Posts: 7183
    • My Community Forum
Re: A script to check dependencies
« Reply #3 on: December 05, 2016, 01:06:15 AM »
Ho can I save the result in a .dep file? Using > formatting color codes are saved too.
Béla
Ham Radio callsign: HA5DI

"Amateur Radio: The First Technology-Based Social Network."

Offline polikuo

  • Hero Member
  • *****
  • Posts: 723
Re: A script to check dependencies
« Reply #4 on: December 05, 2016, 01:10:10 AM »
1. Not all dependencies are libraries, just as not all extensions are libraries. Be careful when you say extensions seem to have unneeded dependecies.
2. Watch out for circular (mutual) dependencies. Freetype <-> harfbuzz and cyrus-sasl <-> openldap are two that I know of off the top of my head.
3. Many dependencies are optional. This is particularly true for programs that use plugins / extensions / modules like Apache, PHP, LDAP, rsyslog, Postgresql, etc.
4. This requires that copy2fs.flg or copy2fs.lst not be used. Might be a problem in some cases.

Thanks for your reply~
I do understand that not all dependencies are libraries, I'm just trying to provide a quick solution for some general extensions that follows this rule

Scripting is never a perfect solution, but it serves as a reference
What I find it helpful is that it reminds you which files are missing

As for "copy2fs" part, my script test if your tcz existed in your TCEDIR
If I can't find the extension, the script exit with error messages

For the "optional" part, I believe the extension manager who made it would pick them wisely
Finally I have a question here
Does ldd link to the "optional" extensons ?

Offline polikuo

  • Hero Member
  • *****
  • Posts: 723
Re: A script to check dependencies
« Reply #5 on: December 05, 2016, 01:14:38 AM »
Ho can I save the result in a .dep file? Using > formatting color codes are saved too.
Oops !
Sorry I have no idea that happens.
I always run it under a terminal emulator
Then copy the deps into the editor
You can remove the "color call" from the script
line 10 ${YELLOW}
line 18 ${NORMAL}
line 106 ${CYAN} ${NORMAL}
line 116 ${RED} ${NORMAL}
line 118 ${MAGENTA} ${NORMAL}
« Last Edit: December 05, 2016, 01:16:59 AM by polikuo »

Offline bmarkus

  • Administrator
  • Hero Member
  • *****
  • Posts: 7183
    • My Community Forum
Re: A script to check dependencies
« Reply #6 on: December 05, 2016, 01:27:38 AM »
Don't ask users to edit the output list, just provide a plain text output. It is better to make it by default and use a -c option to create a colored list if someone need it.
Béla
Ham Radio callsign: HA5DI

"Amateur Radio: The First Technology-Based Social Network."

Offline polikuo

  • Hero Member
  • *****
  • Posts: 723
Re: A script to check dependencies
« Reply #7 on: December 05, 2016, 01:51:14 AM »
Code: [Select]
#!/bin/sh
# author polikuo
# version 1.1
# Dec 4 2016

. /etc/init.d/tc-functions
useBusybox

showhelp() {
  (( $C )) && echo -n ${YELLOW}
  echo 'Usage: '"$(basename $0)" [-option] your_top_directory
  echo '  'Available Options \(Default -ns\)
  echo '  '\-c colorful output \(same as -cns\)
  echo '  '\-f show full dependencies list
  echo '  '\-h show help
  echo '  '\-l show detected tczs \(long\)
  echo '  '\-n show dependencies that are not found
  echo '  '\-s show detected tczs \(short\)
  (( $C )) && echo -n ${NORMAL}
  exit 1
}

unset TCEDIR LIBS NOTFOUND ELF TCZS file DEP DELETE not_needed LIST NN NF option B C D F L O N S T DIR SELF REMOVE

DIR=false
for O in $@;
do
  echo $O | grep -Eqv "^-" && {
    cd $O 2> /dev/null && { DIR=true; break; } || {
      (( $C )) && echo -n ${RED}
      echo can\'t cd to $O
      (( $C )) && echo -n ${NORMAL}
      break
    }
  }
done

(( $DIR )) || showhelp

D=0
C=false
F=false
L=false
N=false
S=false
while getopts cfhlns option
do
case "$option" in
    c ) C=true;;
    f ) F=true; D=$((D+1));;
    h ) showhelp;;
    l ) L=true; D=$((D+1));;
    n ) N=true; D=$((D+1));;
    s ) S=true; D=$((D+1));;
    * ) showhelp;;
esac
done
[ $D -eq 0 ] && { N=true;S=true; }

# Preparing

TCEDIR=/etc/sysconfig/tcedir
ELF="$(find . -not -type d -exec ldd {} + 2> /dev/null | cut -d '(' -f 1 | tr -d '\t' | sort | uniq)"
SELF="$(for B in `find . -not -type d`; do basename $B; done | grep -Ev "\.$")"
LIBS="$(echo "$ELF" | tr -d ' ' | grep -v "notfound" | cut -d '>' -f 2)"
for REMOVE in $SELF
do
  LIBS="$(echo "$LIBS" | grep -v $REMOVE)"
done
NOTFOUND="$(echo "$ELF" | tr -d ' ' | grep "notfound" | cut -d '=' -f 1)"
TCZS="$(ls -l $LIBS 2> /dev/null | grep /tmp/tcloop/ | cut -d '>' -f 2 | cut -d '/' -f 4 | sort | uniq | sed 's/$/\.tcz/g')"

(( $F )) && {
  (( $C )) && echo -n ${GREEN}
  echo "$ELF" && echo 1>&0
  (( $C )) && echo -n ${NORMAL}
}
(( $L )) && {
  (( $C )) && echo -n ${BLUE}
  echo "$TCZS" && echo 1>&0
  (( $C )) && echo -n ${NORMAL}
}

# checking if TCZS existed

for file in $TCZS
do
  [ -s "$TCEDIR"/optional/"$file" ] || {
    (( $C )) && echo -n ${MAGENTA}
    echo $file not found or size is zero
    (( $C )) && echo -n ${NORMAL}
    echo If you load it elsewhere, please move it and the dep file into your
    (( $C )) && echo -n ${CYAN}
    echo $(ls -l /etc/sysconfig | grep tcedir | awk '{ print $11 "optional/" }')
    (( $C )) && echo -n ${NORMAL}
    exit 1
  }
done

# checking for repeated

chkrpt() {
  for DEP in $TCZS
  do
    if [ -s "$TCEDIR"/optional/"$DEP".dep ]
    then
      for DELETE in $(cat "$TCEDIR"/optional/"$DEP".dep)
      do
        if ( echo "$TCZS" | grep -q "$DELETE" )
        then
          not_needed="$not_needed"" $DELETE"
        fi
      done
    fi
  done

  LIST="$TCZS"

  for NN in `echo $not_needed | tr ' ' '\n' | sort | uniq`
  do
    LIST=`echo $LIST | sed "s/$NN//g"`
  done

  for DEP in $LIST
  do
    (( $C )) && echo -n ${CYAN}
    echo $DEP
    (( $C )) && echo -n ${NORMAL}
  done
}

(( $S )) && chkrpt && echo 1>&0

# Show NOTFOUND

snf() {
  [ "$(echo "$NOTFOUND" | tr -d ' ')" = '' ] || {
  (( $C )) && echo -n ${RED} 1>&2
  echo "These files need to be checked" 1>&2
  (( $C )) && echo -n ${NORMAL} 1>&2
  for NF in $NOTFOUND; do
    (( $C )) && echo -n ${MAGENTA}
    echo $NF
    (( $C )) && echo -n ${NORMAL}
  done | sort | uniq 1>&2
  }
}

(( $N )) && snf

Fixed
« Last Edit: December 05, 2016, 01:55:38 AM by polikuo »

Offline Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 14815
Re: A script to check dependencies
« Reply #8 on: December 05, 2016, 04:53:43 AM »
Does ldd link to the "optional" extensons ?

These would typically be found under /usr/local/lib/extension_name/ and could be both libs or executables (libexec).

Offline andyj

  • Hero Member
  • *****
  • Posts: 1036
Re: A script to check dependencies
« Reply #9 on: December 05, 2016, 06:09:35 AM »
I have my own script as well, which is how I learned quickly of the limitations of such scripts. My script will list dependencies to optionally loaded libraries, but as an extention maintainer I recognize the optional from required dependencies. If you are going to try this with extentions you are unfamiliar with proceed cautiously.

Offline polikuo

  • Hero Member
  • *****
  • Posts: 723
Re: A script to check dependencies
« Reply #10 on: December 05, 2016, 06:17:57 AM »
If you are going to try this with extentions you are unfamiliar with proceed cautiously.
I'll keep that in mind
Thanks for your advice

Offline polikuo

  • Hero Member
  • *****
  • Posts: 723
Re: A script to check dependencies
« Reply #11 on: December 05, 2016, 08:45:10 AM »
Code: [Select]
#!/bin/sh
# author polikuo
# version 1.2
# Dec 4 2016

. /etc/init.d/tc-functions
useBusybox

showhelp() {
  (( $C )) && echo -n ${YELLOW}
  echo 'Usage: '"$(basename $0)" [-option] your_top_directory
  echo '  'Available Options \(Default -ns\)
  echo '  '\-c colorful output \(same as -cns\)
  echo '  '\-f show full dependencies list
  echo '  '\-h show help
  echo '  '\-l show detected tczs \(long\)
  echo '  '\-n show dependencies that are not found
  echo '  '\-s show detected tczs \(short\)
  (( $C )) && echo -n ${NORMAL}
  exit 1
}

unset TCEDIR LIBS NOTFOUND ELF TCZS file DEP DELETE not_needed LIST NN NF option B C D F L O N S T DIR SELF REMOVE

DIR=false
for O in $@;
do
  echo $O | grep -Eqv "^-" && {
    cd $O 2> /dev/null && { DIR=true; break; } || {
      (( $C )) && echo -n ${RED}
      echo can\'t cd to $O
      (( $C )) && echo -n ${NORMAL}
      break
    }
  }
done

(( $DIR )) || showhelp

D=0
C=false
F=false
L=false
N=false
S=false
while getopts cfhlns option
do
case "$option" in
    c ) C=true;;
    f ) F=true; D=$((D+1));;
    h ) showhelp;;
    l ) L=true; D=$((D+1));;
    n ) N=true; D=$((D+1));;
    s ) S=true; D=$((D+1));;
    * ) showhelp;;
esac
done
[ $D -eq 0 ] && { N=true;S=true; }

# Preparing

TCEDIR=/etc/sysconfig/tcedir
ELF="$(find . -not -type d -exec ldd {} + 2> /dev/null | cut -d '(' -f 1 | tr -d '\t' | sort | uniq)"
SELF="$(for B in `find . -not -type d`; do basename $B; done | grep -Ev "\.$")"
LIBS="$(echo "$ELF" | tr -d ' ' | grep -v "notfound" | cut -d '>' -f 2)"
NOTFOUND="$(echo "$ELF" | tr -d ' ' | grep "notfound" | cut -d '=' -f 1)"
TCZS="$(ls -l $LIBS 2> /dev/null | grep /tmp/tcloop/ | cut -d '>' -f 2 | cut -d '/' -f 4 | sort | uniq | sed 's/$/\.tcz/g')"

(( $F )) && {
  (( $C )) && echo -n ${GREEN}
  echo "$ELF" && echo 1>&0
  (( $C )) && echo -n ${NORMAL}
}
(( $L )) && {
  (( $C )) && echo -n ${BLUE}
  echo "$TCZS" && echo 1>&0
  (( $C )) && echo -n ${NORMAL}
}

for REMOVE in $SELF
do
  LIBS="$(echo "$LIBS" | grep -v $REMOVE)"
  NOTFOUND="$(echo "$NOTFOUND" | grep -v $REMOVE)"
done

# checking if TCZS existed

for file in $TCZS
do
  [ -s "$TCEDIR"/optional/"$file" ] || {
    (( $C )) && echo -n ${MAGENTA}
    echo $file not found or size is zero
    (( $C )) && echo -n ${NORMAL}
    echo If you load it elsewhere, please move it and the dep file into your
    (( $C )) && echo -n ${CYAN}
    echo $(ls -l /etc/sysconfig | grep tcedir | awk '{ print $11 "optional/" }')
    (( $C )) && echo -n ${NORMAL}
    exit 1
  }
done

# checking for repeated

chkrpt() {
  for DEP in $TCZS
  do
    if [ -s "$TCEDIR"/optional/"$DEP".dep ]
    then
      for DELETE in $(cat "$TCEDIR"/optional/"$DEP".dep)
      do
        if ( echo "$TCZS" | grep -q "$DELETE" )
        then
          not_needed="$not_needed"" $DELETE"
        fi
      done
    fi
  done

  LIST="$TCZS"

  for NN in `echo $not_needed | tr ' ' '\n' | sort | uniq`
  do
    LIST=`echo $LIST | sed "s/$NN//g"`
  done

  for DEP in $LIST
  do
    (( $C )) && echo -n ${CYAN}
    echo $DEP
    (( $C )) && echo -n ${NORMAL}
  done
}

(( $S )) && chkrpt && echo 1>&0

# Show NOTFOUND

snf() {
  [ "$(echo "$NOTFOUND" | tr -d '[\ \t\n]')" = '' ] || {
  (( $C )) && echo -n ${RED} 1>&2
  echo "These files need to be checked" 1>&2
  (( $C )) && echo -n ${NORMAL} 1>&2
  for NF in $NOTFOUND; do
    (( $C )) && echo -n ${MAGENTA}
    echo $NF
    (( $C )) && echo -n ${NORMAL}
  done | sort | uniq 1>&2
  }
}

(( $N )) && snf

I made some stupid mistakes which caused me a lot of trouble
Fix the NOTFOUND function

Offline polikuo

  • Hero Member
  • *****
  • Posts: 723
Re: A script to check dependencies
« Reply #12 on: December 09, 2016, 03:16:07 AM »
Oops, wrong script, my bad :-[

Code: [Select]
#!/bin/sh
# author polikuo
# version 1.3
# Dec 9 2016

. /etc/init.d/tc-functions
useBusybox

showhelp() {
  (( $C )) && echo -n ${YELLOW}
  echo 'Usage: '"$(basename $0)" [-option] your_top_directory
  echo '  'Available Options \(Default -ns\)
  echo '  '\-c colorful output \(same as -cns\)
  echo '  '\-f show full dependencies list
  echo '  '\-h show help
  echo '  '\-l show detected tczs \(long\)
  echo '  '\-n show dependencies that are not found
  echo '  '\-s show detected tczs \(short\)
  (( $C )) && echo -n ${NORMAL}
  exit 1
}

unset TCEDIR LIBS NOTFOUND ELF TCZS file DEP DELETE not_needed LIST NN NF option B C D F L O N S T DIR SELF REMOVE

DIR=false
for O in $@;
do
  echo $O | grep -Eqv "^-" && {
    cd $O 2> /dev/null && { DIR=true; break; } || {
      (( $C )) && echo -n ${RED}
      echo can\'t cd to $O
      (( $C )) && echo -n ${NORMAL}
      break
    }
  }
done

(( $DIR )) || showhelp

D=0
C=false
F=false
L=false
N=false
S=false
while getopts cfhlns option
do
case "$option" in
    c ) C=true;;
    f ) F=true; D=$((D+1));;
    h ) showhelp;;
    l ) L=true; D=$((D+1));;
    n ) N=true; D=$((D+1));;
    s ) S=true; D=$((D+1));;
    * ) showhelp;;
esac
done
[ $D -eq 0 ] && { N=true;S=true; }

# Preparing

TCEDIR=/etc/sysconfig/tcedir
ELF="$(find . -not -type d -exec ldd {} + 2> /dev/null | cut -d '(' -f 1 | tr -d '\t' | sort | uniq)"
SELF="$(for B in `find . -not -type d`; do basename $B; done | grep -Ev "\.$")"
LIBS="$(echo "$ELF" | tr -d ' ' | grep -v "notfound" | cut -d '>' -f 2)"

for REMOVE in $SELF
do
  LIBS="$(echo "$LIBS" | grep -v $REMOVE)"
  NOTFOUND="$(echo "$NOTFOUND" | grep -v $REMOVE)"
done

NOTFOUND="$(echo "$ELF" | tr -d ' ' | grep "notfound" | cut -d '=' -f 1)"
TCZS="$(ls -l $LIBS 2> /dev/null | grep /tmp/tcloop/ | cut -d '>' -f 2 | cut -d '/' -f 4 | sort | uniq | sed 's/$/\.tcz/g')"

(( $F )) && {
  (( $C )) && echo -n ${GREEN}
  echo "$ELF" && echo 1>&0
  (( $C )) && echo -n ${NORMAL}
}
(( $L )) && {
  (( $C )) && echo -n ${BLUE}
  echo "$TCZS" && echo 1>&0
  (( $C )) && echo -n ${NORMAL}
}

# checking if TCZS existed

for file in $TCZS
do
  [ -s "$TCEDIR"/optional/"$file" ] || {
    (( $C )) && echo -n ${MAGENTA}
    echo $file not found or size is zero
    (( $C )) && echo -n ${NORMAL}
    echo If you load it elsewhere, please move it and the dep file into your
    (( $C )) && echo -n ${CYAN}
    echo $(ls -l /etc/sysconfig | grep tcedir | awk '{ print $11 "optional/" }')
    (( $C )) && echo -n ${NORMAL}
    exit 1
  }
done

# checking for repeated

chkrpt() {
  for DEP in $TCZS
  do
    if [ -s "$TCEDIR"/optional/"$DEP".dep ]
    then
      for DELETE in $(cat "$TCEDIR"/optional/"$DEP".dep)
      do
        if ( echo "$TCZS" | grep -q "$DELETE" )
        then
          not_needed="$not_needed"" $DELETE"
        fi
      done
    fi
  done

  LIST="$TCZS"

  for NN in `echo $not_needed | tr ' ' '\n' | sort | uniq`
  do
    LIST=`echo $LIST | sed "s/$NN//g"`
  done

  for DEP in $LIST
  do
    (( $C )) && echo -n ${CYAN}
    echo $DEP
    (( $C )) && echo -n ${NORMAL}
  done
}

(( $S )) && chkrpt && echo 1>&0

# Show NOTFOUND

snf() {
  [ "$(echo "$NOTFOUND" | tr -d '[\ \t\n]')" = '' ] || {
  (( $C )) && echo -n ${RED} 1>&2
  echo "These files need to be checked" 1>&2
  (( $C )) && echo -n ${NORMAL} 1>&2
  for NF in $NOTFOUND; do
    (( $C )) && echo -n ${MAGENTA}
    echo $NF
    (( $C )) && echo -n ${NORMAL}
  done | sort | uniq 1>&2
  }
}

(( $N )) && snf

Offline polikuo

  • Hero Member
  • *****
  • Posts: 723
Re: A script to check dependencies
« Reply #13 on: January 05, 2017, 01:45:11 PM »
In case anybody is still interested in this.
Tested on x86_64.
I practically re-write the script.  :P
bug fix, add new features, check copy2fs  8)

Code: [Select]
tc@box:~ $ script [-option] your_top_directory [append your list here ...]
### for example ###
tc@box:~ $ script -s /tmp/tcloop/gnome-session `cat /etc/sysconfig/tcedir/optional/gnome-session.tcz.dep`
adwaita-icon-theme.tcz
cantarell-fonts.tcz
desktop-file-utils.tcz
epiphany.tcz
gdm-gir.tcz
gedit-gir.tcz
geocode-glib-gir.tcz
gnome-backgrounds.tcz
gnome-terminal.tcz
gvfs.tcz
mousetweaks.tcz
nautilus-gir.tcz
telepathy-mission-control.tcz
udev-extra.tcz
udisks-gir.tcz
vte-gir.tcz
yelp.tcz

As you can see, the output list seems to be shorter than original.
However, if you load the list, you'll get the same result.
Why is that happening ?

Take a look at these 2 extensions  :)
  gnome-desktop-gir.tcz
  gnome-shell-gir.tcz

They are both present in gnome-session.tcz.dep
Yet, if you trace the dep files manually, you'll get:
gnome-session
  gdm-gir
    gnome-shell-gir
      mutter-gir
        gnome-desktop-gir

gnome-desktop-gir and gnome-shell-gir are both dependencies of gdm-gir
The chkonboot.sh would't fix this because the script only cat the dep file once
BTW, the new list works as expected.  ;D

My script fix this bug by looping, this will takes a while if you're scanning big extensions, but the result is quite satisfying.
As for the kernel modules, such as alsa-modules-4.2.9-tinycore64.tcz,
KERNEL is auto-renamed to fit your current kernel when scanning, and is auto-renamed to KERNEL when outputting.

Code here:
Code: [Select]
#!/bin/sh
# author polikuo
# version 1.4
# Jan 6 2017

. /etc/init.d/tc-functions
useBusybox
unset A APPEND C COPY2FS D DELETE DETECTED_TCZS DIR EXT F FILE KERNELVER LDD_OUT LIBS LIST LO N NEWLINE NF NN NOTFOUND NOT_NEEDED R RPT RS S SCAN SELF TCEDIR TCZS option

showhelp() {
( $C ) && echo -n ${YELLOW} 1>&2
cat << END_HELP 1>&2
Usage: $(basename $0) [-option] your_top_directory [append your list here ...]
  Available Options (Default -ns)
  -c colorful output (same as -cns)
  -d show only detected tczs, without custom extensions (cyan)
  -f show full detected dependencies with ldd (white)
  -h show this helpinfo (yellow)
  -n show dependencies that are not found (red)
  -r show recursively scanned tczs (full)
  -s show detected tczs (short)
END_HELP
( $C ) && echo -n ${NORMAL} 1>&2
exit 1
}

# check if TCZS existed
chkexist() {
  for FILE in $@
  do
    [ -s "$TCEDIR"/optional/"$FILE" ] || {
      ( $C ) && echo -n ${MAGENTA}
      echo $FILE not found or size is zero
      echo If you load it elsewhere, please move it and the dep file into your
      echo "$TCEDIR""/optional/"
      ( $C ) && echo -n ${NORMAL}
      exit 1
    }
  done
}

# check repeated
chkrpt() {
  for RPT in $TCZS
  do
    if [ -s "$TCEDIR"/optional/"$RPT".dep ]
    then
      for DELETE in $(cat "$TCEDIR"/optional/"$RPT".dep)
      do
        DELETE=${DELETE##*/}
        DELETE=${DELETE%.tcz}
        DELETE=${DELETE//KERNEL/$KERNELVER}
        DELETE=${DELETE}".tcz"
        if ( echo "$TCZS" | grep -q "$DELETE" )
        then
          NOT_NEEDED="$NOT_NEEDED"" $DELETE"
        fi
      done
    fi
  done

  LIST="$TCZS"

  for NN in $(echo $NOT_NEEDED | tr ' ' '\n' | sort -u)
  do
    LIST="$(echo "$LIST" | sed "/^$NN$/d")"
  done

  ( $C ) && echo -n ${CYAN}
  echo "${LIST//$KERNELVER.tcz/KERNEL.tcz}"
  ( $C ) && echo -n ${NORMAL}
}

# recursively scan dependencies
rscan() {
  cd $TCEDIR/optional
  echo -e "$@" | awk '
  function awk_scan(name, depfile, line) {
    if (name) {
      sub(/KERNEL/, KERNELVER, name)
      depfile=name".dep"
      printf "%s\n", name
      while (getline line < depfile > 0)
        awk_scan(line)
      close(depfile)
    }
  }
  BEGIN {KERNELVER="'"$KERNELVER"'"; IDX=0;}
  {awk_scan($1)}
  '
}

# Options
A=false
C=false
D=false
F=false
N=false
R=false
S=false
while getopts cdfhnrs option 2> /dev/null
do
  case "$option" in
    c ) C=true;;
    d ) D=true; A=true;;
    f ) F=true; A=true;;
    h ) showhelp;;
    n ) N=true; A=true;;
    r ) R=true; A=true;;
    s ) S=true; A=true;;
    * ) showhelp;;
  esac
done
( $A ) || { N=true;S=true; }
shift $((OPTIND-1))

# directory to scan
DIR=false
[ -n "$1" ] || showhelp
cd $1 && DIR=true
( $DIR ) || showhelp
shift

# additional custom extensions
NEWLINE="
"
KERNELVER="$(uname -r)"
[ "$#" -ne 0 ] && {
  for EXT in $@
  do
    EXT=${EXT##*/}
    EXT=${EXT%.tcz}
    EXT=${EXT//KERNEL/$KERNELVER}
    EXT=${EXT}".tcz"
    APPEND="$EXT""$NEWLINE""$APPEND"
  done
}

# copy2fs
COPY2FS=false
TCEDIR="$(realpath /etc/sysconfig/tcedir)"
[ -f $TCEDIR/copy2fs.lst ] && {
  ( $C ) && echo -n ${RED} 1>&2
  echo WARNING! copy2fs.lst found in $TCEDIR, please disable it and reboot 1>&2
  ( $C ) && echo -n ${NORMAL} 1>&2
  COPY2FS=true
}
[ -f $TCEDIR/copy2fs.flg ] && {
  ( $C ) && echo -n ${RED} 1>&2
  echo WARNING! copy2fs.flg found in $TCEDIR, please disable it and reboot 1>&2
  ( $C ) && echo -n ${NORMAL} 1>&2
  COPY2FS=true
}
( $COPY2FS ) && exit 1

# main
LDD_OUT="$(find . -not -type d -exec ldd {} + 2> /dev/null)"
LO="$(echo "$LDD_OUT" | cut -d '(' -f 1 | tr -d '[[:blank:]]')"
SELF="$(find . -not -type d -exec sh -c 'basename "$1"' _ {} \;)"
LIBS="$(echo "$LO" | grep '/' | cut -d '>' -f 2 | sort -u)"
( $N ) && NOTFOUND="$(echo "$LO" | grep "notfound$" | sort -u)"

for REMOVE in $SELF
do
  ( $D ) || ( $R ) || ( $S ) && LIBS="$(echo "$LIBS" | grep -v "/$REMOVE$")"
  ( $N ) && NOTFOUND="$(echo "$NOTFOUND" | grep -v "^$REMOVE=>notfound$")"
done

DETECTED_TCZS="$(realpath $LIBS 2> /dev/null | grep '^/tmp/tcloop/' | cut -d '/' -f 4 | sort -u | sed 's/$/\.tcz/g')"
SCAN="$APPEND""$DETECTED_TCZS"
( $R ) || ( $S ) && { TCZS="$(for RS in $SCAN; do rscan $RS;done | sort -u)";chkexist "$TCZS"; }
NOTFOUND="$(echo "$NOTFOUND" | cut -d = -f 1)"

# output
( $D ) && {
  chkexist "$DETECTED_TCZS"
  ( $C ) && echo -n ${CYAN}
  echo "${DETECTED_TCZS//$KERNELVER.tcz/KERNEL.tcz}"
  ( $C ) && echo -n ${NORMAL}
}

( $F ) && {
  echo "$LDD_OUT"
}

( $N ) && [ -n "$NOTFOUND" ] && {
  ( $C ) && echo -n ${RED} 1>&2
  echo These dependencies are missing 1>&2
  echo "$NOTFOUND" 1>&2
  ( $C ) && echo -n ${NORMAL} 1>&2
}

( $R ) && {
  ( $C ) && echo -n ${GREEN}
  echo "${TCZS//$KERNELVER.tcz/KERNEL.tcz}"
  ( $C ) && echo -n ${NORMAL}
}

( $S ) && chkrpt

unset A APPEND C COPY2FS D DELETE DETECTED_TCZS DIR EXT F FILE KERNELVER LDD_OUT LIBS LIST LO N NEWLINE NF NN NOTFOUND NOT_NEEDED R RPT RS S SCAN SELF TCEDIR TCZS option

Feedbacks are welcomed  :D

Offline patrikg

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 713
Re: A script to check dependencies
« Reply #14 on: January 06, 2017, 04:29:14 AM »
My comment: Have you seen:
Halma on this forum have the following script thread
http://forum.tinycorelinux.net/index.php/topic,17454.0.html