Tiny Core Linux
Tiny Core Extensions => TCE Q&A Forum => Topic started by: jls on April 01, 2010, 06:07:25 AM
-
???
-
Some extensions are dependencies of dependencies. Or even dependencies of dependencies of dependencies.
In the old system, dependencies of dependencies were all listed in the dep file application which needed them.
With the new system, they are listed in the dep file of the dependency which needs them, so they do not need to be listed in the dep file of the application.
To convert them, dep files need to be made for all dependencies with dependencies. Then dependencies of dependencies can be left out of the dep file of the application.
-
ldd is not dependable here, as it shows the full dep chain. Use readelf to see the direct dependencies of binaries and libs:
readelf -d sylpheed | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libgtk-x11-2.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libgdk-x11-2.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libgdk_pixbuf-2.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libpangocairo-1.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libcairo.so.2]
0x0000000000000001 (NEEDED) Shared library: [libpango-1.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libgobject-2.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libgmodule-2.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libglib-2.0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libsylpheed-plugin-0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libsylph-0.so.0]
0x0000000000000001 (NEEDED) Shared library: [libcrypto.so.0.9.8]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
vs.
ldd sylpheed
linux-vdso.so.1 => (0x00007fff64fff000)
libgtk-x11-2.0.so.0 => /usr/lib/libgtk-x11-2.0.so.0 (0x00007f5733d52000)
libgdk-x11-2.0.so.0 => /usr/lib/libgdk-x11-2.0.so.0 (0x00007f5733ab3000)
libgdk_pixbuf-2.0.so.0 => /usr/lib/libgdk_pixbuf-2.0.so.0 (0x00007f573389a000)
libpangocairo-1.0.so.0 => /usr/lib/libpangocairo-1.0.so.0 (0x00007f573368e000)
libcairo.so.2 => /usr/lib/libcairo.so.2 (0x00007f5733413000)
libpango-1.0.so.0 => /usr/lib/libpango-1.0.so.0 (0x00007f57331c9000)
libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0x00007f5732f86000)
libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0x00007f5732d83000)
libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x00007f5732a99000)
libsylpheed-plugin-0.so.0 => /usr/lib/libsylpheed-plugin-0.so.0 (0x00007f5732894000)
libsylph-0.so.0 => /usr/lib/libsylph-0.so.0 (0x00007f5732625000)
libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 (0x00007f57322af000)
libc.so.6 => /lib/libc.so.6 (0x00007f5731f6a000)
libXinerama.so.1 => /usr/X11R7/lib/libXinerama.so.1 (0x00007f5731d68000)
libXrandr.so.2 => /usr/X11R7/lib/libXrandr.so.2 (0x00007f5731b60000)
libXcursor.so.1 => /usr/X11R7/lib/libXcursor.so.1 (0x00007f5731956000)
libstdc++.so.6 => /usr/lib/../lib/libstdc++.so.6 (0x00007f5731653000)
libgcc_s.so.1 => /usr/lib/../lib/libgcc_s.so.1 (0x00007f5731446000)
libXcomposite.so.1 => /usr/X11R7/lib/libXcomposite.so.1 (0x00007f5731244000)
libXext.so.6 => /usr/X11R7/lib/libXext.so.6 (0x00007f5731031000)
libXdamage.so.1 => /usr/X11R7/lib/libXdamage.so.1 (0x00007f5730e2f000)
libXfixes.so.3 => /usr/X11R7/lib/libXfixes.so.3 (0x00007f5730c2a000)
libatk-1.0.so.0 => /usr/lib/../lib/libatk-1.0.so.0 (0x00007f5730a0c000)
libpixman-1.so.0 => /usr/X11R7/lib/libpixman-1.so.0 (0x00007f57307b5000)
libpng12.so.0 => /usr/lib/../lib/libpng12.so.0 (0x00007f5730592000)
libXrender.so.1 => /usr/X11R7/lib/libXrender.so.1 (0x00007f5730389000)
libX11.so.6 => /usr/X11R7/lib/libX11.so.6 (0x00007f573004e000)
libxcb.so.1 => /usr/X11R7/lib/libxcb.so.1 (0x00007f572fe34000)
libXau.so.6 => /usr/X11R7/lib/libXau.so.6 (0x00007f572fc32000)
libXdmcp.so.6 => /usr/X11R7/lib/libXdmcp.so.6 (0x00007f572fa2d000)
libgio-2.0.so.0 => /usr/lib/../lib/libgio-2.0.so.0 (0x00007f572f77a000)
libresolv.so.2 => /lib/libresolv.so.2 (0x00007f572f566000)
libpangoft2-1.0.so.0 => /usr/lib/../lib/libpangoft2-1.0.so.0 (0x00007f572f338000)
libfontconfig.so.1 => /usr/lib/../lib/libfontconfig.so.1 (0x00007f572f0fa000)
libfreetype.so.6 => /usr/lib/../lib/libfreetype.so.6 (0x00007f572ee6b000)
libz.so.1 => /lib/libz.so.1 (0x00007f572ec55000)
libexpat.so.1 => /usr/lib/../lib/libexpat.so.1 (0x00007f572ea28000)
libdl.so.2 => /lib/libdl.so.2 (0x00007f572e824000)
libm.so.6 => /lib/libm.so.6 (0x00007f572e5a5000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00007f572e38e000)
libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0x00007f572e146000)
/lib/ld-linux-x86-64.so.2 (0x00007f573430d000)
-
I'm asking 4 a script 2 convert an old .dep into an new .dep which is recursive
-
we were using a script during the testing of recursive deps, but I think it was modified - maybe it's better Jason/Robert suggest the best way to handle this
-
While script published by Jason does a great job, it is using a strict matemathical algorithm without knowing anything about the real dependencies. Result is a set of minimalist dep files where it is guaranteed that all necessary will be loaded but not guranteed that all real or direct dependencies kept.
Best way is to adjust dep files manually by the package creator knowing real dependencies. There are either only few and easy to handle dependencies or typical cases easy to recognize. For example many GUI based on GTK+ If you see gtk2.tcz you can remove its dependencies
atk.tcz
cairo.tcz
pango.tcz
glib2.tcz
fontconfig.tcz
expat2.tcz
libxml2.tcz
graphics-libs-1.tcz
pixman.tcz
Xlibs_supp
python.tcz , dbus-glib are similar.
These cases can be scripted to make life easier, but at the end you have to check and correct result yourself.
-
Hi bmarkus,
not guaranteed that all real or direct dependencies kept.
if every package creator knowing real dependencies, then would it be guaranteed to have minimal real dependency so that recursive works?
-
Hi bmarkus,
not guaranteed that all real or direct dependencies kept.
if every package creator knowing real dependencies, then would it be guaranteed to have minimal real dependency so that recursive works?
Of course nothing is guaranteed. There are missing and unneeded dependencies for sure in current 'old' deps due to human mistakes including myself. There will be bugs in the future as well. I'm saying only that package creator knows dependencies and can make better dep lists than a script processing dep files only.
-
If all needed dependencies are already listed in a dep file, none of them will get erased if they are not already satisfied at a lower level in the dep chain. At least I have not seen a case using the script I use to convert my dep files once I debugged it. Of course if you are missing dependencies in your dep file, then simply add them before converting it to recursion or add it after once checking it over.
I am not against folks converting their dep files by hand. But I have seen so very few submissions of converted existing dep files. I am going to post the tool I use here for those who want it, even if there is disagreement with it's use. As with any tool, simply verify the results.
The recursive routine used in the script was taken directly from tce-load. Run in the same directory as your dep files are.
#!/bin/sh
rsync -av distro.ibiblio.org::distros/tinycorelinux/2.x/tcz/*.dep /tmp/repodeplist
# routine taken from tce-load
recursive_scan() {
echo ${1} >> /tmp/deplist
deps=""
if [ -f /tmp/repodeplist/${1}.dep ]; then
deps=`cat /tmp/repodeplist/${1}.dep`
for d in $deps; do
recursive_scan $d
done
fi
}
for F in `ls *.dep`; do
for I in `cat "$F"`; do
recursive_scan "$I"
for E in `cat "$F"`; do
if cat /tmp/deplist | grep -v "$I" | grep "$E" >/dev/null; then
echo "$E"
sed -i "/^"$E"/d" "$F"
fi
done
rm /tmp/deplist
done
done
-
Original hal.tcz.dep is
expat2.tcz
glib2.tcz
dbus.tcz
dbus-glib.tcz
libpci.tcz
pci-utils.tcz
libusb.tcz
usb-utils.tcz
udevadm.tcz
acl.tcz
which is converted by the script to
dbus-glib.tcz
udevadm.tcz
I would say the correct is:
dbus-glib.tcz
pci-utils.tcz
usb-utils.tcz
udevadm.tcz
-
pci-utils.tcz and usb-utils.tcz are satisfied by udevadm.tcz so the result of the script is correct in terms of the definition of recursion. Brutally efficient, but technically correct. If there is a technical flaw in the script I definitely want to know about it as I don't want to post a flawed utility. What you are talking about is a matter of preference as to the number of redundancies in the dep file. Personally, I prefer none as I make no distinction whether a dependency is an app or lib. .
You are free to include redundancies in your dep files. But as I understand it, not including redundancies is not incorrect.
-
As the udevadm extension is unnecessary on >=2.9, and recursive deps require at least 2.10, I think it wouldn't be needed anymore.
edit: quick fix
-
pci-utils.tcz and usb-utils.tcz are satisfied by udevadm.tcz so the result of the script is correct in terms of the definition of recursion. Brutally efficient, but technically correct. If there is a technical flaw in the script I definitely want to know about it as I don't want to post a flawed utility. What you are talking about is a matter of preference as to the number of redundancies in the dep file. Personally, I prefer none as I make no distinction whether a dependency is an app or lib. .
You are free to include redundancies in your dep files. But as I understand it, not including redundancies is not incorrect.
Jason,
script is really great and in most cases gives the proper result. What my 6th sense is saying that there will be or there are deps removed unnecesarily due to multiple dependencies which are not causing unresolved dep on the top.
Script is like machine online translation. Gives a funny result, but very useful if you do not speak target language or can be used as a basis for translation and enhance manually.
To be pragmatic, my proposal is to convert the whole repo and correct bugs on the fly. It is much faster and efficient than to wait for extension builders - including myself - to submit corrected lists. Of course, everybody is free to send dep updates later on. In such case no need to indicate it .info file as all extensions will be recursive. The only extra work is to exclude dep file already made recursive.
BTW I like the way how you are converting repo with scripts when big changes occure, like single type, squash, etc :-*
-
As the udevadm extension is unnecessary on >=2.9, and recursive deps require at least 2.10, I think it wouldn't be needed anymore.
Thanks for mentioning! Somehow I missed it. Tested, indeed HAL works fine on 2.10 with udevadm from the base. I will make necessary corrections in dep and info files.
-
My main purpose with a script is to of course save time, and I admit it does produce some funny looking results. Like epdfview, the gtk2 pdf viewer, has now only poppler.tcz as a dependency. Gtk2 is often buried under gnome libs or even simpler ones like poppler. But hopefully if it is technically correct then the extensions will load and run with no issue.
How about this, I can script convert and upload folks extensions as they give the go ahead. I will backup the existing dep files on the server, and then convert folks dep files as they say it is ok. I can start with bmarkus' and we can verify to make sure it went ok.
-
would be nice 2 automatize the conversion of the tcbuild scritps too, the one I use have inside the deps list
-
My main purpose with a script is to of course save time, and I admit it does produce some funny looking results. Like epdfview, the gtk2 pdf viewer, has now only poppler.tcz as a dependency. Gtk2 is often buried under gnome libs or even simpler ones like poppler. But hopefully if it is technically correct then the extensions will load and run with no issue.
How about this, I can script convert and upload folks extensions as they give the go ahead. I will backup the existing dep files on the server, and then convert folks dep files as they say it is ok. I can start with bmarkus' and we can verify to make sure it went ok.
For me OK
-
Bmarkus, your deps have been converted. I will be away for a few hours but I have a tarball previous state of all that was converted and can revert in a few seconds. Let me know if there is any issue in the result. Thanks.
-
I'm testing the new deps. This is really a fun :) gpredict.tcz works, but have missing icons and background image, will look for cause. Main issue is the strange curl.tcz.dep file
BTW, it is not a problem of the conversion script but the original curl.tcz.dep which has libldap.tcz as a dependency. Logically it depend on openssl only.
-
Here was the original dep file"
goocanvas.tcz
curl.tcz
openssl-0.9.8m.tcz
gtk2.tcz
atk.tcz
cairo.tcz
pango.tcz
glib2.tcz
fontconfig.tcz
expat2.tcz
libxml2.tcz
graphics-libs-1.tcz
pixman.tcz
Xlibs_support.tcz
Sounds like shared-mime-info was missing as an entry in the original dep file. I am not on my home machine to test the origial dep file when used with "base norestore", but I am curious if the same behavior would be seen.
And I had added libldap to the curl dep file before conversion, it was not a result of conversion. I had forgot to take it out after disabling ldap.
-
Sounds like shared-mime-info was missing as an entry in the original dep file.
That's right, and there is another missing dependency. It has nothing to do with recursion or conversion, just a bug in early extension. Will fix it later.
-
Jason:
following dep files are tested and correct as they are converted:
aircrack-ng.tcz
alpine.tcz
asterisk.tcz
bind.tcz
fetchmail-locale.tcz
fetchmail.tcz
filezilla-locale.tcz
gnutls.tcz
goocanvas-locale.tcz
goocanvas.tcz
gpicview-locale.tcz
gpicview.tcz
GraphicsMagick.tcz
No conversion bug found, only bugs in original dep files. Corrections will be posted. Remaining in progress.
-
Thanks.
If anyone wants me to convert their dep files let me know. For TC 3.x, extensions will all have recursive deps, so any moved from 2.x will be converted in the process.
-
is there any advantage in terms of speed in loading tcz with converted deps?
-
There is less processing of entries in the dep files, so there is more efficiency.
-
I think it's a bit slower when downloading. Having to download all .dep files (of the extension and deps) instead of one being the main point.
-
I thought he was meaning was there a speed advantage when using TC 2.10 with converted dep files versus the old simple ones, but you also have a point.
-
I think the deps conversion should be a tc tem decision, 4 me it's ok since I use 2.10
-
I will go ahead and convert the repo's dep files as I think with the recent testing the method is reliable enough to use in production. I will keep backups of the originals in case of any issue.