WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Broken library linking in the repo  (Read 2654 times)

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Broken library linking in the repo
« on: November 08, 2011, 11:49:19 AM »
I have a pretty simple routine almost in place that will check for broken library linking  in what is already existing.  And perhaps if it can be feasible, maybe a way to check to see if library extension updates are going to break any apps in the repo to know ahead of time if an update will cause an issue.  It will take a long time to run, but maybe something that can be done from time to time.

Best case is for updates to not break existing apps without planning for it ahead of time, but I know that is not always feasible to predict.

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: Broken library linking in the repo
« Reply #1 on: November 08, 2011, 10:41:05 PM »
I was hoping to find a much smaller list of broken extensions in regards to library dependencies.  I am very glad folks are working hard to stay current on their extensions, but it is not so good when updates break existing extensions.   I wish to be able to offer a reasonably performing method of checking if one's library extension update will break things in the repo in terms of library linking.  Still would take a few minutes one a P4, but perhaps could add stability and save us some headaches in the long run.  I will check into possible methods of allowing extension makers to do this with a tool that would operate in a timely manner.

Below is a list of broken extensions due to missing library linking.

Code: [Select]
amsn.tcz     Needs but not offered:      libImlib.so.11
apache2-mod-php5.tcz     Needs but not offered:      libMagickCore.so.4
apache2-mod-php5.tcz     Needs but not offered:      libMagickWand.so.4
audacious-plugins.tcz     Needs but not offered:      libavutil.so.49
cloog-ppl.tcz     Needs but not offered:      libppl.so.7
cloog-ppl.tcz     Needs but not offered:      libppl_c.so.2
cloog.tcz     Needs but not offered:      libppl.so.7
cloog.tcz     Needs but not offered:      libppl_c.so.2
cloog.tcz     Needs but not offered:      libppl.so.7
cloog.tcz     Needs but not offered:      libppl_c.so.2
compiz-gtk-window-decorator.tcz     Needs but not offered:      libgnome-desktop-2.so.11
deadbeef.tcz     Needs but not offered:      libpulsecommon-0.9.22.so
dvdauthor.tcz     Needs but not offered:      libMagickCore.so.4
e_tclock.tcz     Needs but not offered:      libecore-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_con-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_evas-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_file-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_imf-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_imf_evas-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_input-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_ipc-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_job-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_txt-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libecore_x-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libedbus-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libedje-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libefreet-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libefreet_mime-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libehal-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libeina-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libembryo-ver-svn-05.so.0
e_tclock.tcz     Needs but not offered:      libevas-ver-svn-05.so.0
ebook-tools.tcz     Needs but not offered:      libzip.so.1
ebook-tools.tcz     Needs but not offered:      libzip.so.1
efm_addons.tcz     Needs but not offered:      libecore-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libecore_con-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libecore_evas-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libecore_file-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libecore_imf-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libecore_imf_evas-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libecore_input-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libecore_input_evas-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libecore_ipc-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libecore_x-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libedbus-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libedje-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libefreet-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libefreet_mime-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libehal-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libeina-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libembryo-ver-pre-svn-05.so.0
efm_addons.tcz     Needs but not offered:      libevas-ver-pre-svn-05.so.0
freeradius.tcz     Needs but not offered:      libmysqlclient_r.so.16
gnash.tcz     Needs but not offered:      libavutil.so.49
gnome-media.tcz     Needs but not offered:      libpulsecommon-UNKNOWN.UNKNOWN.UNKNOWN.so
gnome-settings-daemon.tcz     Needs but not offered:      libpulsecommon-0.9.22.so
graphviz.tcz     Needs but not offered:      libruby.so.1.9
inkscape.tcz     Needs but not offered:      libMagick++.so.4
inkscape.tcz     Needs but not offered:      libMagickCore.so.4
iproute2.tcz     Needs but not offered:      libdb-4.8.so
kdebindings-dev.tcz     Needs but not offered:      libcppparser.so
kdelibs-3.5.10.tcz     Needs but not offered:      libpcre.so.3
kdelibs-3.5.10.tcz     Needs but not offered:      libpcreposix.so.3
libcanberra.tcz     Needs but not offered:      libpulsecommon-UNKNOWN.UNKNOWN.UNKNOWN.so
lighttpd-mysql.tcz     Needs but not offered:      libmysqlclient.so.16
mercurial-1.2.1.tcz     Needs but not offered:      libpython2.5.so.1.0
mkvtoolnix.tcz     Needs but not offered:      libebml.so.0
mkvtoolnix.tcz     Needs but not offered:      libmatroska.so.0
moc-svn.tcz     Needs but not offered:      libdb-4.8.so
moc-svn.tcz     Needs but not offered:      libdb-4.8.so
nautilus-sendto.tcz     Needs but not offered:      libcamel-1.2.so.14
nautilus-sendto.tcz     Needs but not offered:      libebook-1.2.so.9
nautilus-sendto.tcz     Needs but not offered:      libedataserver-1.2.so.13
ogre.tcz     Needs but not offered:      libboost_thread.so.1.44.0
ogre.tcz     Needs but not offered:      libboost_thread.so.1.44.0
openoffice3.tcz     Needs but not offered:      libdb-4.8.so
php5.tcz     Needs but not offered:      libmysqlclient.so.16
qt-4.x-sql.tcz     Needs but not offered:      libmysqlclient_r.so.16
rasqal.tcz     Needs but not offered:      libmpfr.so.1
redland.tcz     Needs but not offered:      libdb-4.8.so
redland.tcz     Needs but not offered:      libdb-4.8.so
redland.tcz     Needs but not offered:      libmysqlclient.so.16
robot-player.tcz     Needs but not offered:      libboost_thread.so.1.44.0
robot-player.tcz     Needs but not offered:      libboost_thread.so.1.44.0
rrdtool.tcz     Needs but not offered:      libruby.so.1.9
rtorrent.tcz     Needs but not offered:      libtorrent.so.11
rtorrent.tcz     Needs but not offered:      libxmlrpc.so.3
rtorrent.tcz     Needs but not offered:      libxmlrpc_server.so.3
seed.tcz     Needs but not offered:      libgirepository-1.0.so.0
seed.tcz     Needs but not offered:      libmpfr.so.1
squeeze.tcz     Needs but not offered:      libexo-0.3.so.0
squeeze.tcz     Needs but not offered:      libexo-hal-0.3.so.0
squeeze.tcz     Needs but not offered:      libexo-0.3.so.0
stage.tcz     Needs but not offered:      libfltk_gl.so.1.1
thunar-plugins.tcz     Needs but not offered:      libexo-0.3.so.0
thunar-plugins.tcz     Needs but not offered:      libthunarx-1.so.2
thunarx-python.tcz     Needs but not offered:      libthunarx-1.so.2
totem.tcz     Needs but not offered:      libgdata.so.6
xarchiver.tcz     Needs but not offered:      libnss3.so.1d
xarchiver.tcz     Needs but not offered:      librpm.so.0
xarchiver.tcz     Needs but not offered:      librpmio.so.0
xfce4-goodies-panel-plugins.tcz     Needs but not offered:      libexo-0.3.so.0
xfce4-goodies-panel-plugins.tcz     Needs but not offered:      libxfce4panel.so.1
xfce4-goodies-panel-plugins.tcz     Needs but not offered:      libxfce4panel.so.1
xfce4-mixer.tcz     Needs but not offered:      libxfce4panel.so.1
xfce4-power-manager.tcz     Needs but not offered:      libxfce4panel.so.1
xfce4-weather-plugin.tcz     Needs but not offered:      libxfce4panel.so.1
xvidcap.tcz     Needs but not offered:      libavutil.so.49


Offline bmarkus

  • Administrator
  • Hero Member
  • *****
  • Posts: 7183
    • My Community Forum
Re: Broken library linking in the repo
« Reply #2 on: November 09, 2011, 02:07:45 AM »
A slightly different but related topic. What is about a tool to create .dep file automatically?
Béla
Ham Radio callsign: HA5DI

"Amateur Radio: The First Technology-Based Social Network."

Offline bmarkus

  • Administrator
  • Hero Member
  • *****
  • Posts: 7183
    • My Community Forum
Re: Broken library linking in the repo
« Reply #3 on: November 09, 2011, 02:23:59 AM »
Regarding moc-svn.tcz it is better to drop and forget.

- version number unknown
- having unstable versions is against generic TC repo policy
- it is broken
- extension creator is gone, logged in last time July 2010

Béla
Ham Radio callsign: HA5DI

"Amateur Radio: The First Technology-Based Social Network."

Offline Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 14819
Re: Broken library linking in the repo
« Reply #4 on: November 09, 2011, 02:53:40 AM »
cloog is the tc-3.x version of the cloog-ppl extension - I guess, since the name changed in the tc-4.x repo, it did not overwrite the old version - I've deleted cloog from the tc-4.x repo

As regards libcloog from cloog-ppl:
Code: [Select]
$ ldd /usr/local/lib/libcloog.so
        linux-gate.so.1 =>  (0xb783b000)
        libgmp.so.10 => /usr/local/lib/libgmp.so.10 (0xb77c1000)
        libppl_c.so.4 => /usr/local/lib/libppl_c.so.4 (0xb73b2000)
        libppl.so.9 => /usr/local/lib/libppl.so.9 (0xb72de000)
        libgmpxx.so.4 => /usr/local/lib/libgmpxx.so.4 (0xb72da000)
        libc.so.6 => /lib/libc.so.6 (0xb71ce000)
        libpwl.so.5 => /usr/local/lib/libpwl.so.5 (0xb71ca000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7108000)
        libm.so.6 => /lib/libm.so.6 (0xb70e5000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xb70cf000)
        /lib/ld-linux.so.2 (0xb783c000)

..so things look OK now.

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: Broken library linking in the repo
« Reply #5 on: November 09, 2011, 07:42:01 AM »
Ok, I will delete moc-svn.  I normally think of openssl, maybe icu, and a couple others when I think of stuff that may break things with an upgrade, but evidently the list is a bit longer. 

I could make an automatic dep file maker pretty simply, but would require rsync to fetch the list files, as rsync does it in a few seconds while wget takes quite a bit longer. 

But to be able to make a routine that will check to see if an extension will break something would require maintaining ldd output files for each extension.  They are small, only a few or a few hundred bytes each.  Making them for everything all at once takes a few hours on my old server, but making one at a time with a simple command takes a few seconds. 

In a way I hesitate to push for yet another extension related file to maintain, though it's creation on extension making/auditing and space taken up would be trivial, and it only takes three commands, two of which are mounting and umounting the extension.   

I will get my routines in place and see what I would need.  A generator for needed deps library wise will be relatively simple with what we have now, and I will ponder the feasibility of the ldd stuff, though it is valuable information to know.  But perhaps not needed daily enough to warrant changes, we'll see.


Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: Broken library linking in the repo
« Reply #6 on: November 09, 2011, 08:21:33 AM »
Here is an example ldd file for fontconfig.tcz, to give an idea on what one would look like.

Code: [Select]
/lib/ld-linux.so.2
libc.so.6
libexpat.so.1
libfontconfig.so.1
libfreetype.so.6
libz.so.1
linux-gate.so.1

(Thread split as this is separate topic.)
« Last Edit: November 09, 2011, 09:40:07 AM by Jason W »

Offline gerald_clark

  • TinyCore Moderator
  • Hero Member
  • *****
  • Posts: 4254
Re: Broken library linking in the repo
« Reply #7 on: November 09, 2011, 10:27:52 AM »
What do you do if more than one extension provides the same module?

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: Broken library linking in the repo
« Reply #8 on: November 09, 2011, 11:19:33 AM »
Good point, and that does get into the realm of alternate dependencies if two or more extensions contain the same solib.  As for determining what needs to be in the dep file, that would of course be at the judgement of the extension maker. 

This script piece will generate the ldd file, I still am working on a routine to suggest dependencies.  This takes .39 seconds to make an ldd file for gtk2:

Code: [Select]
#!/bin/sh


for I in `ls *.tcz`; do

[ -f "$I".ldd ] && rm "$I".ldd

if [ -d "$I".tmp ]; then
umount "$I".tmp
else
mkdir "$I".tmp
fi
busybox mount -o loop "$I" "$I".tmp

find "$I".tmp -not -type d | xargs file | grep "ELF" | \
grep "executable" | cut -f1 -d: | xargs ldd | cut -f1 -d" " | \
sed 's/[ \t] *//' >> "$I".lddtmp

find "$I".tmp -not -type d | xargs file | grep "ELF" | \
grep "shared object" | cut -f1 -d: | xargs ldd | cut -f1 -d" " | \
sed 's/[ \t] *//' >> "$I".lddtmp

cat "$I".lddtmp | sort | uniq > "$I".ldd
rm "$I".lddtmp

umount -d "$I".tmp
rm -r "$I".tmp
done



Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: Broken library linking in the repo
« Reply #9 on: November 09, 2011, 11:35:12 AM »
And here is a very rough prototype of a dependency suggester I have run on my server, very slow but somewhat functional.


for I in `cat lddout/gtk2.tcz.ldd`; do for E in `ls tinycorelinux/4.x/x86/tcz/*.list`; do grep -q -m1 ""$I"$""$E" && echo "`basename "$E" .list`  Provides:    "$I""; done; done

Offline gerald_clark

  • TinyCore Moderator
  • Hero Member
  • *****
  • Posts: 4254
Re: Broken library linking in the repo
« Reply #10 on: November 09, 2011, 12:04:42 PM »
How is that going to work with nested dependencies?
Won't that make a linear dependency list that will be slower with multiple extensions?

Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: Broken library linking in the repo
« Reply #11 on: November 09, 2011, 12:19:24 PM »
This or any other routine will not make a ready to use dep file, but simply points out the library links.  And then the resulting info sorted, some tossed out, and then made recursive.   Appbrowser has the "provides" button which instantly shows which extension contains a certain library.

Both a dependency maker and a breakage detector has to sort through a huge amount of lines and is very slow.  Mostly I am concerned with checking for breakage due to library linking, which can be done periodically.  And the usefulness of an .ldd file for that purpose.  Though the dep file generating is an exercise.


Offline Jason W

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 9730
Re: Broken library linking in the repo
« Reply #12 on: November 09, 2011, 03:05:22 PM »
Ok, I have found a much, much faster command than posted above to suggest dependencies from ldd output.  But the usefulness of the resulting data basically relegates it so a scripting exercise, so I really won't pursue that one further.

But, more importantly, a means of quickly checking whether an extension update will break an existing app is taking shape and may be worth pursuing.