Tool to check to see if an extension update will break existing apps/libs.

[Jason W]

This tool will help extension makers find what may break due to library solib changes in extensions they are planning to update.  With the sheer size of the repo, there is no efficient way to check and see if an update will break things due to library changes.  We could predict this much easier in the early days when the repo was small, but not anymore.  This tool has been tested and should help provide information to allow extension makers to plan extension updates with the potentially broken existing ones in mind.  The library linking information is produced on my server, and though it is not kept in real time with the repo, I will try to keep it updated and uploaded daily so it should be useful.

Extension makers, please use this tool and see if your updates will break existing apps, and if so, either coordinate a planned rebuild of dependent apps or hold off on updating, as we do not need to break things first and then try to fix them later.  I do not blame our hard working extension makers on any previous updates that broke things, as there was no way to forsee it then.  But I do ask our valued contributors to check their extensions with this tool before sending them in, and let me know if you have any concerns.

Code: [Select]

#!/bin/sh

. /etc/init.d/tc-functions

> /tmp/ldderrors


FILEDIR=/tmp/lddout
[ -d "$FILEDIR" ] && rm -r "$FILEDIR"
[ -f /tmp/lddout.tar.gz ] && rm /tmp/lddout.tar.gz
wget -O /tmp/lddout.tar.gz http://distro.ibiblio.org/tinycorelinux/4.x/x86/tcz/src/lddout.tar.gz

tar xf /tmp/lddout.tar.gz -C /tmp


for I in `ls *.tcz`; do

if [ -d "$I".tmp ]; then
sudo umount "$I".tmp
else
mkdir "$I".tmp
fi
sudo busybox mount -o loop "$I" "$I".tmp

echo "${YELLOW}Checking the repo for possible breakage from the update of extension: ${GREEN}"$I":${NORMAL}"

find "$I".tmp/lib "$I".tmp/usr/lib "$I".tmp/usr/local/lib -maxdepth 1 -name *.so* | cut -f1 -d: > "$I".liblist

LIST=`cat "$I".liblist | sort | uniq`

for F in $LIST; do

FULLNAME=`basename "$F"`
SHORTNAME=`echo ${FULLNAME%*.so.*}`
echo " "
echo "${YELLOW}Checking file: ${GREEN}"$FULLNAME":${NORMAL}"


for D in `grep "$SHORTNAME".so "$FILEDIR"/*.ldd`; do
FILE1=`echo "$D" | cut -f1 -d:`
FILE=`basename "$FILE1" .ldd`
FOUND=`echo "$D" | cut -f2 -d:`
  if [ ! -z "$FOUND" ]; then
     if [ "$FOUND" != "$FULLNAME" ] && [ "$(expr length "$FOUND")" == "$(expr length "$FULLNAME")" ] && ! echo "$LIST" | grep "$FOUND$" >/dev/null; then
        echo " "
echo "${GREEN}"$I":  ${RED} WARNING!:${YELLOW}  This extension may break ${MAGENTA}"$FILE"${YELLOW}, which needs ${GREEN}"$FOUND"${YELLOW}, but ${GREEN}"$I"${YELLOW} contains ${GREEN}"$FULLNAME"${NORMAL}"     
        echo ""$I":   This extension may break "$FILE", which needs "$FOUND", but "$I" contains "$FULLNAME"" >> /tmp/ldderrors
        echo " " >> /tmp/ldderrors
     fi
  fi
done


done

sudo umount -d "$I".tmp
rmdir "$I".tmp
rm "$I".liblist
echo " "
echo "${GREEN}###################################${NORMAL}"
echo "${GREEN}###################################${NORMAL}"
echo " "
done
echo "${YELLOW}Please read ${GREEN}/tmp/ldderrors${YELLOW} to see the total of possible broken extensions.${NORMAL}"




[Jason W]

And here is the routine to generate ldd files:

Code: [Select]

#!/bin/sh

OUTPUTDIR=lddout


[ -d "$OUTPUTDIR" ] && rm -r "$OUTPUTDIR"
mkdir "$OUTPUTDIR"

for I in `ls *.tcz`; do

if mount | grep "$I".tmp; then
umount "$I".tmp
fi

[ -d "$I".tmp ] || mkdir "$I".tmp

busybox mount -o loop "$I" "$I".tmp


> "$OUTPUTDIR"/"$I".ldd
find "$I".tmp -not -type d | xargs file | grep "ELF" | grep "executable" | cut -f1 -d: | xargs ldd | cut -f1 -d" " | grep -v "linux-gate.so.1" | grep -v "/lib/ld-linux.so.2" | sed 's/[ \t] *//' | sort | uniq >> "$OUTPUTDIR"/"$I".ldd
find "$I".tmp -not -type d | xargs file | grep "ELF" | grep "shared object" | cut -f1 -d: | xargs ldd | cut -f1 -d" " | grep -v "linux-gate.so.1" | grep -v "/lib/ld-linux.so.2" | sed 's/[ \t] *//' | sort | uniq >> "$OUTPUTDIR"/"$I".ldd


umount -d "$I".tmp
rm -r "$I".tmp
done



[Rich]

Hi Jason W

And here is the routine to generate ldd files:

Outstanding piece of work. Maybe it should be included as part of the submitqc.tcz extension as this script
eliminates much of the tedium of tracking down dependencies. One very minor note, I had to use sudo to run
the script due to its mount command. Should I be running as root when creating a package?

[Jason W]

Running as root versus not when creating packages has proponents on each side.  I know with one extension I made, xscreensaver I think, the directory ownership was tc:staff for all directories and that caused issues with something else that needed them root.  Directories like /usr/local, that is, since loading the extension imposes the extension's directory owner/perms on the existing directories.   I personally run as root to preserve permissions mainly.