WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Tool to check to see if an extension update will break existing apps/libs.  (Read 30702 times)

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
This tool will help extension makers find what may break due to library solib changes in extensions they are planning to update.  With the sheer size of the repo, there is no efficient way to check and see if an update will break things due to library changes.  We could predict this much easier in the early days when the repo was small, but not anymore.  This tool has been tested and should help provide information to allow extension makers to plan extension updates with the potentially broken existing ones in mind.  The library linking information is produced on my server, and though it is not kept in real time with the repo, I will try to keep it updated and uploaded daily so it should be useful.

Extension makers, please use this tool and see if your updates will break existing apps, and if so, either coordinate a planned rebuild of dependent apps or hold off on updating, as we do not need to break things first and then try to fix them later.  I do not blame our hard working extension makers on any previous updates that broke things, as there was no way to forsee it then.  But I do ask our valued contributors to check their extensions with this tool before sending them in, and let me know if you have any concerns.

Code: [Select]
#!/bin/sh

. /etc/init.d/tc-functions

> /tmp/ldderrors


FILEDIR=/tmp/lddout
[ -d "$FILEDIR" ] && rm -r "$FILEDIR"
[ -f /tmp/lddout.tar.gz ] && rm /tmp/lddout.tar.gz
wget -O /tmp/lddout.tar.gz http://distro.ibiblio.org/tinycorelinux/4.x/x86/tcz/src/lddout.tar.gz

tar xf /tmp/lddout.tar.gz -C /tmp


for I in `ls *.tcz`; do

if [ -d "$I".tmp ]; then
sudo umount "$I".tmp
else
mkdir "$I".tmp
fi
sudo busybox mount -o loop "$I" "$I".tmp

echo "${YELLOW}Checking the repo for possible breakage from the update of extension: ${GREEN}"$I":${NORMAL}"

find "$I".tmp/lib "$I".tmp/usr/lib "$I".tmp/usr/local/lib -maxdepth 1 -name *.so* | cut -f1 -d: > "$I".liblist

LIST=`cat "$I".liblist | sort | uniq`

for F in $LIST; do

FULLNAME=`basename "$F"`
SHORTNAME=`echo ${FULLNAME%*.so.*}`
echo " "
echo "${YELLOW}Checking file: ${GREEN}"$FULLNAME":${NORMAL}"


for D in `grep "$SHORTNAME".so "$FILEDIR"/*.ldd`; do
FILE1=`echo "$D" | cut -f1 -d:`
FILE=`basename "$FILE1" .ldd`
FOUND=`echo "$D" | cut -f2 -d:`
  if [ ! -z "$FOUND" ]; then
     if [ "$FOUND" != "$FULLNAME" ] && [ "$(expr length "$FOUND")" == "$(expr length "$FULLNAME")" ] && ! echo "$LIST" | grep "$FOUND$" >/dev/null; then
        echo " "
echo "${GREEN}"$I":  ${RED} WARNING!:${YELLOW}  This extension may break ${MAGENTA}"$FILE"${YELLOW}, which needs ${GREEN}"$FOUND"${YELLOW}, but ${GREEN}"$I"${YELLOW} contains ${GREEN}"$FULLNAME"${NORMAL}"     
        echo ""$I":   This extension may break "$FILE", which needs "$FOUND", but "$I" contains "$FULLNAME"" >> /tmp/ldderrors
        echo " " >> /tmp/ldderrors
     fi
  fi
done


done

sudo umount -d "$I".tmp
rmdir "$I".tmp
rm "$I".liblist
echo " "
echo "${GREEN}###################################${NORMAL}"
echo "${GREEN}###################################${NORMAL}"
echo " "
done
echo "${YELLOW}Please read ${GREEN}/tmp/ldderrors${YELLOW} to see the total of possible broken extensions.${NORMAL}"


« Last Edit: December 05, 2011, 05:11:36 PM by Jason W »

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: Tool to check to see if an extension update will break existing apps/libs.
« Reply #1 on: December 02, 2011, 02:31:19 PM »
And here is the routine to generate ldd files:

Code: [Select]
#!/bin/sh

OUTPUTDIR=lddout


[ -d "$OUTPUTDIR" ] && rm -r "$OUTPUTDIR"
mkdir "$OUTPUTDIR"

for I in `ls *.tcz`; do

if mount | grep "$I".tmp; then
umount "$I".tmp
fi

[ -d "$I".tmp ] || mkdir "$I".tmp

busybox mount -o loop "$I" "$I".tmp


> "$OUTPUTDIR"/"$I".ldd
find "$I".tmp -not -type d | xargs file | grep "ELF" | grep "executable" | cut -f1 -d: | xargs ldd | cut -f1 -d" " | grep -v "linux-gate.so.1" | grep -v "/lib/ld-linux.so.2" | sed 's/[ \t] *//' | sort | uniq >> "$OUTPUTDIR"/"$I".ldd
find "$I".tmp -not -type d | xargs file | grep "ELF" | grep "shared object" | cut -f1 -d: | xargs ldd | cut -f1 -d" " | grep -v "linux-gate.so.1" | grep -v "/lib/ld-linux.so.2" | sed 's/[ \t] *//' | sort | uniq >> "$OUTPUTDIR"/"$I".ldd


umount -d "$I".tmp
rm -r "$I".tmp
done


Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11516
Re: Tool to check to see if an extension update will break existing apps/libs.
« Reply #2 on: February 12, 2012, 01:31:48 PM »
Hi Jason W
Quote
And here is the routine to generate ldd files:
Outstanding piece of work. Maybe it should be included as part of the submitqc.tcz extension as this script
eliminates much of the tedium of tracking down dependencies. One very minor note, I had to use sudo to run
the script due to its mount command. Should I be running as root when creating a package?

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: Tool to check to see if an extension update will break existing apps/libs.
« Reply #3 on: February 12, 2012, 03:03:40 PM »
Running as root versus not when creating packages has proponents on each side.  I know with one extension I made, xscreensaver I think, the directory ownership was tc:staff for all directories and that caused issues with something else that needed them root.  Directories like /usr/local, that is, since loading the extension imposes the extension's directory owner/perms on the existing directories.   I personally run as root to preserve permissions mainly.