Tiny Core Linux
dCore Import Debian Packages to Mountable SCE extensions => dCore X86 => Topic started by: netnomad on November 08, 2013, 10:53:27 AM
-
hi roberts, hi jason w,
with microcore i configured a very lean base-system with ondemand-packages that cover almost every task i could need during the year with my micro-tiny-multi-core-stick.
this lean base system can be expanded with some really heavy packages like libreoffice, multimedia-stuff and virtualboxes,
but usually i load only few packages that are really needed.
my dCore-configuration offers only few packages although my mydesktop.sce weights already 250mb.
now i want to configure a similar lean configuration with dCore,
but i experienced that additional, separated sce-packages depend on so many duplicated packages.
in the moment it seems to be problematic to reach my goal with this approach.
please consider following idea:
importsce -f base
# base.sce offers a base-system
importsce -b base -f desktop
# desktop.sce includes all further packages based on the packages and their dependencies
# that are provided by base
importsce -b base,desktop -f multimedia
# multimedia.sce is imported based on the packages
# that are provided by base and desktop
with such an approach dCore could offer a resembling small footprint like the tcz-packages of the other cores.
thank you for improving this exciting project, for your time and your commitment.
-
hi friends,
jason w contributed a great tool called sce-merge:
http://forum.tinycorelinux.net/index.php/topic,16199.msg95963.html#msg95963
as i´m looking for a solution with low hardware-requirements,
i think that a modular approach could help.
with the current situation my configuration is getting soon so fat that it could get unusable :(
thank you for your help and hints.
-
hi roberts, hi jason w,
did you miss to read my post?
thank you for your reflections about this topic.
-
Hi netnomad,
The use of a list file to import that creates one sce with no duplicate files is the closes thing to what you are talking about that we are able to support.
-
hi jason w,
thank you for your reply.
yes, you are right that i could do everything in one sce and then there are no duplicates in that sce.
i'm looking for a lean, fast and stable system that is so multifuctional that i can use it for all my every day tasks.
so i want to start with a lean basic system and load the seldom used application only when they are needed.
my basic programm-set includes fluxbox, iceweasel, icedove and some basic cli-programs :)
in some cases i want to load these fat programs, only when i need them:
libreoffice, gimp, xsane, gtkam, isomaster, minitube, linphone, skype, virtualbox, several additional browsers.
all programs in one sce extend the resources of my hardware and my patience (to boot) ;)
on the other side it is not very handy to package additional sces,
cause every separate package needs so many duplicate libraries, especially for X :(
in my proposed concept these duplicates could be avoided and
the advantages of the tcz-approach could improve the dCore-concept in a lean way.
so please be so kind and consider the suggestions in my post.
thank you for your time and attention :)
-
I see how what you are wanting could be done, simply download and merge what is not already installed by a running sce. Simply add on another layer. The problem is that would create a web of dependent sces and once one of them is taken from sceboot.lst, the system is broken. That, and one would have to know which packages are smaller than others so they could start with the smaller ones first. And then those sce's would have to be loaded in exactly the same order as they were imported or there would be missing libraries or files possible.
The only advantage I see is saving HD or flash media storage space. What is your goal, a lean and clean system where nothing is duplicated in RAM or simply smaller storage media space requirements? Nothing is duplicated in the running system, loading one sce that contains an xorg and then another that also contains it does not require any more RAM.
-
The only advantage I see is saving HD or flash media storage space. What is your goal, a lean and clean system where nothing is duplicated in RAM or simply smaller storage media space requirements? Nothing is duplicated in the running system, loading one sce that contains an xorg and then another that also contains it does not require any more RAM.
hi jason w,
most of the time i need just X, a browser, a mail-agent, mc and ssh.
for this scenario i need a fast booting, lean system that is easy to update, to clone or to modify.
i think there is a difference between 10s boottime or around 40-60s,
around 300mb of ram or almost all my 1gb that has to swap and so on.
if i need libreoffice, gimp or multimedia than it's o.k. to wait another 10s to load the extension,
then it's o.k. that more ram is needed for that single package,
but the unneeded packages should not bloat my system.
the main reason is that i want to avoid to put everything in one sce
that could be rarely needed once in the next 12 months ;)
That, and one would have to know which packages are smaller than others so they could start with the smaller ones first. And then those sce's would have to be loaded in exactly the same order as they were imported or there would be missing libraries or files possible.
i think that the users that are interested in this option will learn to understand the way how to use it :)
by the way some distributions use a similiar approach...
first a core-package is loaded, then a package with the x-windows-infrastructure,
afterwards you can choose between different packages that include windowsmanagers or desktop-environments,
then there is a choice of different browser-packages,
later you can choose between a small office-package like abiword or the big libreoffice-suite,
and at the very end eventually a development-tool-set is loaded.
with this approach the user can decide between a non-X-base system, different desktop-environments, browsers, office-solutions and the option of development without the disadvantages of an full-blown livesystem that includes all choices that could be needed some time.
i think that it's not so difficult to start with a base.sce, then package in the next stage something like desktop.sce based on base.sce and then package something like multimedia.sce or office.sce based on these two.
perhaps it won't be the most common way to use dCore, but it could be a suitable option.
thank you for all your considerations, ideas and help.
-
hi jason w,
i want to give you some facts:
i boot with syslinux the actual dCore 5.14.02.11 and my basic micocore 4.7.7:
###
dCore 5.14.02.11:
bootime displayed in the boot-messages 36,69s
real boottime from syslinux to fluxbox: 1:18 min
used ram 538000 from 1gb
loaded sces:
/dev/loop0 14M 14M 0 100% /tmp/tcloop/original-modules-3.8.13-tinycore
/dev/loop1 362M 362M 0 100% /tmp/tcloop/mydesktop
mydesktop includes:
Xprogs
wbar
xorg-all
grep
bash
screen
mc
ssh
iptables
fluxbox
fail2ban
laptop-mode-tools
pm-utils
iceweasel
icedove
cups
cups-driver-gutenprint
hplip-cups
jpilot
gnupg
rsync
epdfview
gthumb
xvnc4viewer
gimp
importupdatecheck
alsa-base
alsa-utils
alsamixergui
cdparanoia
faad
lame
vlc
nmap
lvm2
cryptsetup
xsane
after loading icecat in dCore 5.14.02.11 and doing some actions my ram changed to
used ram 762800 from 1gb
after loading thunderbird
used ram 917000 from 1gb
###
microcore 4.7.7
bootime displayed in the boot-messages 3,79s
real boottime from syslinux to fluxbox: 0:29min
used ram 107300 from 1gb
loaded tczs:
/dev/loop0 288.0K 288.0K 0 100% /tmp/tcloop/fltk-1.1.10
/dev/loop1 3.2M 3.2M 0 100% /tmp/tcloop/Xlibs
/dev/loop2 128.0K 128.0K 0 100% /tmp/tcloop/Xprogs
/dev/loop3 364.0K 364.0K 0 100% /tmp/tcloop/Xvesa
/dev/loop4 1.3M 1.3M 0 100% /tmp/tcloop/libiconv
/dev/loop5 76.0K 76.0K 0 100% /tmp/tcloop/expat2
/dev/loop6 132.0K 132.0K 0 100% /tmp/tcloop/fontconfig
/dev/loop7 656.0K 656.0K 0 100% /tmp/tcloop/Xorg-7.6-lib
/dev/loop8 668.0K 668.0K 0 100% /tmp/tcloop/fluxbox
/dev/loop9 36.0K 36.0K 0 100% /tmp/tcloop/wbar
/dev/loop10 8.0K 8.0K 0 100% /tmp/tcloop/915resolution
/dev/loop11 140.0K 140.0K 0 100% /tmp/tcloop/kmaps
/dev/loop12 12.0K 12.0K 0 100% /tmp/tcloop/actkbd
/dev/loop13 552.0K 552.0K 0 100% /tmp/tcloop/myfonts
/dev/loop14 1.4M 1.4M 0 100% /tmp/tcloop/openssl-1.0.0
/dev/loop15 840.0K 840.0K 0 100% /tmp/tcloop/libssl-0.9.8
/dev/loop16 60.0K 60.0K 0 100% /tmp/tcloop/libssh2
/dev/loop17 12.0K 12.0K 0 100% /tmp/tcloop/libffi
/dev/loop18 1.4M 1.4M 0 100% /tmp/tcloop/glib2
/dev/loop19 396.0K 396.0K 0 100% /tmp/tcloop/slang
/dev/loop20 1.1M 1.1M 0 100% /tmp/tcloop/mc
/dev/loop21 12.0K 12.0K 0 100% /tmp/tcloop/ncurses-common
/dev/loop22 148.0K 148.0K 0 100% /tmp/tcloop/ncurses
/dev/loop23 392.0K 392.0K 0 100% /tmp/tcloop/bash
/dev/loop24 380.0K 380.0K 0 100% /tmp/tcloop/screen
/dev/loop25 304.0K 304.0K 0 100% /tmp/tcloop/gcc_libs
/dev/loop26 876.0K 876.0K 0 100% /tmp/tcloop/openssh
/dev/loop27 316.0K 316.0K 0 100% /tmp/tcloop/netfilter-3.0.21-tinycore
/dev/loop28 448.0K 448.0K 0 100% /tmp/tcloop/iptables
/dev/loop29 28.0K 28.0K 0 100% /tmp/tcloop/bzip2-lib
/dev/loop30 328.0K 328.0K 0 100% /tmp/tcloop/sqlite3
/dev/loop31 8.6M 8.6M 0 100% /tmp/tcloop/python
/dev/loop32 92.0K 92.0K 0 100% /tmp/tcloop/fail2ban
/dev/loop33 28.0K 28.0K 0 100% /tmp/tcloop/cpufrequtils
/dev/loop34 28.0K 28.0K 0 100% /tmp/tcloop/acpid
/dev/loop35 80.0K 80.0K 0 100% /tmp/tcloop/cpufreqd
/dev/loop36 60.0K 60.0K 0 100% /tmp/tcloop/laptop-mode-tools
/dev/loop37 164.0K 164.0K 0 100% /tmp/tcloop/libx86
/dev/loop38 24.0K 24.0K 0 100% /tmp/tcloop/libpci
/dev/loop39 316.0K 316.0K 0 100% /tmp/tcloop/suspend-utils
/dev/loop40 4.0K 4.0K 0 100% /tmp/tcloop/mirrors
/dev/loop41 28.0K 28.0K 0 100% /tmp/tcloop/ipager
/dev/loop42 12.0K 12.0K 0 100% /tmp/tcloop/ntpclient
/dev/loop43 88.0K 88.0K 0 100% /tmp/tcloop/pdnsd
/dev/loop44 220.0K 220.0K 0 100% /tmp/tcloop/libevent
/dev/loop45 36.0K 36.0K 0 100% /tmp/tcloop/redsocks
/dev/loop46 16.0K 16.0K 0 100% /tmp/tcloop/xautolock-2.2
/dev/loop47 8.0K 8.0K 0 100% /tmp/tcloop/xtrlock
/dev/loop48 4.0K 4.0K 0 100% /tmp/tcloop/xautolock_meta
installed tczs:
915resolution
acpid
actkbd
atk
bash
bzip2-lib
cairo
cpufreqd
cpufrequtils
expat2
fail2ban
fltk-1.1.10
fluxbox
fontconfig
gcc_libs
gdk-pixbuf2
glib2
graphics-libs-1
gtk2
icecat
ipager
iptables
kmaps
laptop-mode-tools
libasound
libevent
libffi
libiconv
liblzma
libpci
libssh2
libssl-0.9.8
libx86
libxcb
libxml2
mc
mirrors
myfonts
ncurses
ncurses-common
netfilter-3.0.21-tinycore
nspr
ntpclient
openssh
openssl-1.0.0
pango
pdnsd
pixman
python
redsocks
screen
shared-mime-info
slang
sqlite3
suspend-utils
wbar
xautolock-2.2
xautolock_meta
Xlibs
Xorg-7.6-lib
Xprogs
xtrlock
Xvesa
after loading icecat in 4.7.7 and doing some actions my ram changed to
used ram 362300 from 1gb
after loading thunderbird
used ram 524500 from 1gb
-
I am open to options that are obtainable without costing performance for everyone else. But to make dCore modular is not as simple as it sounds and would basically require a rewrite from scratch. We did try at first the modular approach, but the performance of the import routine was slow and Debian packages have circular dependencies that lock up our dependency routine.
dCore certainly uses some more resources than standard Core due to the size and dependencies of Debian packages, but please use the "cache-clear" command before comparing RAM between the two systems. I am sure the results would be more comparable after clearing the memory cache, use cache-clear and post the results again.
As for the boot time, I assure you that if we used the modular approach to dCore that it would be a much longer boot, SCE cut down that time very significantly.
And you are comparing two different package sets. The dCore install has the heavyweight VLC , xsane, gthumb with it's large dependency list, epdfview, gimp (another large one) and xorg-all, the full Xorg package. It is not a fair comparison, the only somewhat large package in your tcz installation is the icecat browser.
Choose a similar package list for both tcz and sce, use cache-clear once booted, and I am very interested in those results in RAM occupation, boot time, and total package size.
-
I am open to options that are obtainable without costing performance for everyone else.
what if we could specify just one package as base for another one. And with no recursive deps.
e.g. i start with bare dCore. and i make xorg-all. It probaby has many libraries in that other packages could reuse.
now i make , e.g. i3 package (the window manager), by doing something like
importsce i3 --base xorg-all which will work like import i3 , but will only include files not provided by xorg-all sce.
However, doing
importsce something --base i3
should fail with "i3 is not a standard sce, can't be used as dependency".
For simplicity, only one package can be specified as argument for --base , and it has to be standard sce package (doesn't depend on any other sce).
So you would make one basic package, and base arbitrary number of packages on it. But there would be only one level of hierarchy and no circular deps.
For faster package generation, classic sce's could be accompanied by list of deb's that it was made from.
-----------------------------
I use dCore on dell c400 laptop and generation of SCE's takes a while, even though i mostly need i3, geeqie, geany, audacious, dwb browser, elinks and few cli tools.
If i want to add an app to it, rebuild of this one big sce is slow. Making separate sce is also fairly slow and reuses lots of files of original sce.
I would happily switch to clasic tinycore, but i cannot get dwb to build on it with new webkit, and other browsers and not very lightweight in comparison. I would probably have to stick with core 4.x. dCore has just way more software available more easily.
-
First, I appreciate any and all interest in dCore and I don't want to come across as defensive or dismissive in the presence of new ideas or requests. I also think it would be useful if there was an option that reduced duplication among sce packages.
How it is now is basically like PC-BSD, all dependencies are in the package file which is quite large, but only what is not already installed is merged into the live directory tree.
I have some ideas in this direction I may can prototype, I will think more on it.
-
I admit to being in over my head here, but my understanding is that it takes a lot of effort on the part of developers to maintain the Tiny Core tcz repository, not only to package apps as tcz's but to make sure all the dependencies are in sync etc.
I thought an advantage of dCore is that less of that is required. Since each app is packaged separately with its own set of dependencies (regardless of what's packaged with other apps) there's a lot more flexibility in obtaining apps from other repositories and having them work in TC. Not only a lot more apps but less effort needed from developers.
Is this wrong?
-
Thane - that is correct. However I have made some adjustments to importsce and the related files that will allow the use of existing sce's as dependencies. Like the size listing option, it will not affect those that want to stay with the current self contained sce. This option would make expanding one's sce collection much more efficient, of course the current routine will remain the same.
Say you have a huge mydesktop.sce that contains all of the deps of gthumb. Use the following command to make use of mydesktop.sce as a dependency.
importsce -d gthumb
With the -d option, you will be prompted with a select menu listing existing sce's in your sce directory. They don't have to be mounted. Select mydesktop as a dep, then proceed. Multiple dependencies are possible.
The result is that gthumb.sce will contain only the gthumb package if all the required deps exist in mydesktop, that is checked for by the list of debs that mydesktop.sce is made of against what gthumb requires. A gthumb.sce.dep file is created, containing mydesktop as the entry. If loadsce sees a sce.dep file when loading, it will check the needed debs versus what the sce in the dep file consists of, alerting the user if the required debs are no longer in the mydesktop.sce. Just in case one updates mydesktop.sce and removes packages on re-creation of it. That way, no mysterious brokenness, missing required packages will be echoed.
I have the code in place and will test more this evening.
-
hi jason w,
great news... i'm looking forward and help testing :)
thank you for your commitment.
-
The result is that gthumb.sce will contain only the gthumb package if all the required deps exist in mydesktop, that is checked for by the list of debs that mydesktop.sce is made of against what gthumb requires.
What will happen if there are some dependencies missing? Will they be included into gthumb, or will a rebuild of mydesktop.sce will be required?
-
Ok, I have a rough working model in place. Here are the basics. Use the "importsce -d" option to make use of an existing sce as a dependency. More than one can be chosen, a select menu will appear when the importsce -d option is used. Lets say your import and install xorg-all. Then you want to install libgtk2.0-0 and use xorg-all as a dep, use the command "importsce -d libgtk2.0-0". Choose xorg-all as a dependency sce when prompted. The rest of the routine is as usual.
Pay attention to the /usr/local/sce/$PKGNAME directory. Some new files are created that can be used by a tool to check for broken dependencies in the case of issues. For now there is no tool. But if you use a mydesktop.sce as a dependency for firefox, then you later rebuild mydesktop without ligtk2.0-0. Firefox will no longer work since it was counting on gtk2 from mydesktop.sce. Any sce that you feel may be broken, just re-import it and use your previous dependency sce. In this case, a re-import of firefox will include ligtk2.0-0 in firefox.sce that previously only had the firefox package inside as it's deps were met before but now they are not..
loadsce makes use of dep files now. Lets say that you import libgtk2.0-0 and use xorg-all as a dep. Later, you import leafpad and only specify libgtk2.0-0 as a dep If ligtk2.0-0 is the only entry in leafpad.sce.dep, then xorg-all.sce will also be loaded since it is in the dep file of libgtk2.0-0. But you can also specify multiple dependency sce's, in the case having both xorg-all and libgtk2.0-0 in leafpad.sce.dep would have the same effect.
I have uploaded a testing dCore.gz to
http://tinycorelinux.net/5.x/x86/release_candidates/
I have tried it out on some packages, and I think things are polished enough for wider testing.
-
loadsce needs some more work, I will aim to tend to it tonight but the basic concept is in place.
We can test this out and see how we like it.
-
hi jason w,
i tried your new rc and did
import -d galculator
Choose the sce you want use as a dependency. You can choose more than one, and enter q for quit when you have completed your selection.
1. libreoffice4.1
2. mydesktop
3. original-modules-3.8.13-tinycore
Enter selection ( 1 - 3 ) or (q)uit: q
mydesktop
original-modules-3.8.13-tinycore
cat: can't open '/tmp/tcloop/mydesktop/usr/local/sce/mydesktop/mydesktop.md5sum': No such file or directory
cat: can't open '/tmp/tcloop/original-modules-3.8.13-tinycore/usr/local/sce/original-modules-3.8.13-tinycore/original-modules-3.8.13-tinycore.md5sum': No such file or directory
Proceed to create galculator.sce? (y/N):
...
===> a dependency-file is created, but the galculator.sce has the same size as without the -d-option.
then i did another
tc@box:~$ importsce -s -f .TCE/sce/mydesktop
after the preparation of the merging i get multiple error messages like:
grep: /tmp/.scedebs: No such file or directory
...
===> and a corrupted small mydesktop.sce is merged with missing files...
thank you for your help and work for the community :)
-
Mydesktop.sce will have to be imported with the new routine, as the md5sum file you mentioned as missing is only created in that manner by the new routine. galculator is the same size because that md5 file is used to determine what packages are contained in mydesktop.sce. If using old sce's, import is assuming nothing is installed in the old sce and therefore of no use as a dependency, galculator.sce will then be full size as before.
I will look into the issue in re-creating mydesktop file list sce's.
-
There is a simple but grievous error in debGetDeps. I am testing that fix along with the finishing touches to loadsce and plan to post another cut tonight.
-
Ok, I think for the most part all should be good now, I reposted in the release candidates directory.
Here is my setup. I have a base.sce. Then a desktop-basic that depends on base. Then a multimedia that depends on desktop-basic, which in turn depends on base. Then a desktop-full which depends on desktop-basic which depends on base. No overlap.
Below are the file lists of my working setup. Please test and report any errors.
base:
ssh
rsync
screen
locales-all
grep
gzip
git
file
findutils
cpio
bc
bash
acpid
bzip2
zip
xz-utils
wget
tar
sed
pciutils
lzma
p7zip-full
desktop-basic:
Xprogs
icewm
xorg-all
graphics-3.8.13-tinycore
leafpad
emelfm2
xscreensaver
firefox
multimedia:
wodim
xine-ui
wine
vlc
timidity
stella
mplayer
mpg123
lame
faac
avidemux
desktop-full:
xfce4
fluxbox
acroread
epdfview
flwm_topside
galculator
geany
gimp
gthumb
gtk-gnutella
mc
mtpaint
-
I have a problem with generation of subsequent dep sce's.
Steps:
- make xorg-all normal sce.
- make libgtk-3-0 sce as dependant on xorg-all
- make deadbeef sce as dependant on libgtk-3-0
when specifying only libgtk-3-0 only that sce is listed as dep.
when specifying both xorg-all and libgtk-3-0 i get dep listing as
libgtk-3-0
xorg-all
xorg-all
in either case an nearly empty package gets made for deadbeef with only .debs files
squashfs-root
squashfs-root/usr
squashfs-root/usr/local
squashfs-root/usr/local/sce
squashfs-root/usr/local/sce/deadbeef
squashfs-root/usr/local/sce/deadbeef/deadbeef.debdeps
squashfs-root/usr/local/sce/deadbeef/libgtk-3-0.debs
squashfs-root/usr/local/sce/deadbeef/xorg-all.debs
Perhaps i should be running on bare system installation to have them generate properly, although previous two sce's seem to have been generated correctly - i was generating this packages with one big sce loaded from previous dCore installation that already provided contents of those three packages (and more).
-
hi jason w,
3 hours ago i took your last cut.
at my reboot the module original-modules-KERNEL of my sceboot.lst wasn´t found anymore.
sceboot.lst:
original-modules-KERNEL
mydesktop
thank you for your help.
-
Ok, evidently that was broken in the changes, I will fix it.
-
Yoshi,
I created the same sce's on my box, and all works as expected. xorg-all was first imported, then libgtk-3-0 was imported with xorg-all listed as a dep. Then deadbeef was imported with libgtk-3-0 listed as a dep. There is no need to specify xorg-all in the dep file of deadbeef since the dep files are recursive.
Please try again with a clean boot on as basic of a system as possible, though it really should not matter as long as the sce's are created with the current dCore.gz and rebooting in case the same sce is already loaded and running.
'
There are 517 files in my deadbeef.sce, and it starts and runs as expected.
-
It works better. Now sce's generate correctly. Could be i needed to be on bare installation.
There is another problem i spotted.
On a bare system (that generated a few sce's but none were loaded) I ran : importsce wireless .
After seeing list of packages i pressed ctrl+c.
I attempted to importsce bzip2
This pulled all the firmware package dependencies into bzip2.sce . Re-importing bzip2 makes it generate properly, without wireless deps.
Few things that could use improvement.
- usage info for importsce
- option to just list packages that will be used into generating an sce, before confirming your choice
- maybe some integrated readme on most common things (esp extra repositories)
- some kind of simple search for debian packages
-
I uploaded a new dCore.gz, please test it. I put in a safety measure for when ctrl-c is pressed during import.
Now that hopefully the bugs are down to a minimum, I will work on some of the points mentioned.
-
hi jason w,
original-modules-KERNEL boots fine now.
but the reimport of my old mydesktop.sce results in multiple line of
....
grep: /tmp/.scedebs: No such file or directory
...
before the start of the merging process.
thank you for your help.
-
Btw is there a way to list those core provided packages or comfortably search them ? There are some core provided packages or metapackages and i am not sure how to find them if i don't know their names. It seems that core packages are used when debian has no packages under the same name.
Also, wifi package has no dependencies. It fails to work due to lack of wireless-tools, even if wireless interfaces are available. Providing wireless-tools makes it work.
One more suggetion that could help less advanced users - simple script to make extension out of directory. I have a b43 driven wireless card and it requires obtaining firmware on my own, with b43-fwcutter. It's not provided in any sce.
Problem is that the easy way of putting it into mydata.tgz makes it available after the b43 driver loads. Unload+load of the driver crashes the kernel (requires more modules to be unloaded/loaded afaik), which makes even more of a problem for less savvy users.
I made an sce with generated firmware manually and set it to load as the first extension, before wireless modules, solving my problem. Most users might have a hard time resolving this. I suspect this situation might occur in different circumstances.
-
Uploaded new cut.
Hopefully fixed the /tmp/.scedebs grep echo which is harmless but nevertheless needs to be quieted.
Added a --help to importsce. Use -h or --help to access it.
Before final confirmation of making an sce, the list of packages that will be included only in that sce and not what is already in it's deps are shown.
Added wireless-tools to the deps of wifi.
As for readmes, we can always use help in documentation. I try to get to it when I can, maybe as things settle more documentation can appear.
Added a tool "searchdeb" that will search the Packages files of the main and any extra mirror, listing possible package matches along with the mirror they will be pulled from.
I will find a way to list custom or meta packages soon.
To make an sce out of a single directory, it is simple. If the directory is firmware it's directory from the top is firmware/lib/firmware/, do:
mksquashfs firmware myfirmware.sce.
-
hi jason w,
i can confirm that the last cut works fine for me.
thank you for your help.
-
Great.
I honed searchdeb to include a more exhaustive list of packages. But I would rather importsce to give the more streamlined results as it currently does. Searchdeb to find all possible matches, then use the results as a guide on what characters to enter into importsce.
Also, a searchprebuilt tool is available. Uploaded new cut to release candidates. If no more major bugs, I will soon commit to git.
-
hi jason w,
a really good job :)
these new options improved dCore so much!
new opportunities are waiting for us :)
searchdeb and searchprebuilt: are these tools for the user or are they just working inside of importsce?
thank you for your work.
-
Thanks!
Both search tools are basically importsce with everything removed except from what is needed to search debs or prebuilt. They are made for easy and exhaustive searching for the user. I thought it would keep it simpler to separate the search only functions and not add any more complexity to importsce.
Hopefully things will just work at this point, lets give it a week of use and if no issues arise I will add to git and this should make for another release.
Location of the current testing dCore.gz:
http://www.tinycorelinux.net/5.x/x86/release_candidates/dCore.gz
http://www.tinycorelinux.net/5.x/x86/release_candidates/dCore.gz.md5.txt
EDIT: Reposted with some bugfixes 2.28.14 7:30pm.
-
Ok, another cut after going over with a fine tooth comb.
Also, a new tool "checkmissingdebs" that is available as an extension for in the case you use, say, mydesktop as a dependency but then later re-import it with fewer packages than originally and then apps that depend on it are broken. This tool will point out what is missing that is needed by dependent apps, though a simple re-import of a broken sce in this case will fix it by bringing in any needed deps not provided by the dependency sce's.
"importsce checkmissingdeps" to import the tool.
"checkmissingdebs firefox" to tell which Debian packages are needed by firefox but are not loaded in the system and will tell you which dependency SCE's they were originally provided by.
-
hi jason w,
my tests give really good results.
only little issues:
tc@box:~$ checkmissingdebs gimp
No package information is available for gimp. Exiting..
tc@box:~$ checkmissingdebs mc
No package information is available for mc. Exiting..
tc@box:~$ checkmissingdebs 000-boot
No package information is available for 000-boot. Exiting..
during the
importsce -d -s -f .TCE/sce/000-base
i get error messages about a missing /tmp/.scedebs and /tmp/.scedeps
just to inform you ;)
thank you for your good work.
-
Thanks, I also saw the missing /tmp files a time or two on the latest cut. I will peek in at it tonight.
The missing package information message means that there are no lists of dependency packages inside the sce. I will make it check sces that have no depedencies using their md5sum file and echo an "No missing packages" type of message. Though I don't know how contents of a loaded sce would not get installed, that would at least tell us if the general import and loadsce routine is working. This checkmissingdebs tool had actually exposed a few bugs in import/loadsce due to these recent changes that I would otherwise not have found, at least not right away.
-
hi jason w,
i tested the last cut...
tc@box:~/.MNT/sdb1/live/dcore-testing/boot$ md5sum dCore.gz
6fb5dbec60195fd1cc17da51c3276b67 dCore.gz
it seems fine for me except of the error messages about the missing /tmp/.scedebs.
so i help me with
tc@box:~$ cd tmp
tc@box:~/tmp$ touch .scedebs
tc@box:~/tmp$ touch .scedeps
thank you for your help.
-
Hi netnomad,
It is tonight, and I just took a peek at it. :-) I honed checkmissingdebs, and hopefully fixed the /tmp file existence error messages. Harmless, but indeed should not be there. Also some other minor cosmetic bugfixes. I am satisfied that we are at the stage of going over it with a fine tooth comb.
Uploaded a new dCore.gz to the release candidate area, and also updated the checkmissingdebs package.
Please test and report any issues. Thanks.
-
hi jason w,
sorry, but the files /tmp/.scedebs and /tmp/.scedeps are still missing...
... i just want to inform you and i don't want to harrass you with cosmetic stuff ;)
but i'm still not sure whether i understand the usage of checkmissingdebs:
could you give an example that shows me any result?
my packages don't show any results:
tc@box:~$ checkmissingdebs mc
No package information is available for mc. Exiting..
tc@box:~$ checkmissingdebs 000-base
No package information is available for 000-base. Exiting..
thank you for your help.
-
Checkmissingdebs is really a way to protect the user from himself. Say he imports base.sce from a package list, then uses it as a dependency for firefox. Then later he re-imports base with a shorter file list than before, leaving out libgtk2.0-0 or any package that firefox is dependent upon. On rebooting, he sees firefox does not work.
He can run "checkmissingdebs firefox" and it will tell him that at the time firefox was imported using base as a dep, base contained libgtk2.0-0 but now it does not. He can then re-import firefox and all is well.
Any missing dependencies from re-importing dependency packages that result in a broken app that depends on it can be identified. Say firefox depended on desktop.sce which depends on base.sce, a missing package in base or desktop or even firefox will show up when checking firefox.
I redid checkmissingdebs to have more clear messages. /tmp/.debinstalled is the list of packages (debs and prebuilt, not the sce's) that are installed in the system. And that is the file that is used by checkmissingdebs. To test functionality without re-importing sce's that are used as dependencies just delete entries from that list, emulating their not being installed in the system.
I will look into the echoing of missing /tmp files, I thought I covered the bases but something is supposed to be redirecting to null output that is not.
-
Importupdatecheck updated to reflect current changes.
-
that might be a stupid question to ask, but is there a package that will provide user with basic system?
something like libc+coreutils+util-linux+zlib+bzip2 (and probably more) ?
we could use some kind of repository of metapackages that would reference certain commonly encountered sets, as now it's mostly trial and error and i often end up with sce's duplicating certain libraries, despite my best efforts not to.
i would really like as little things reused as possible between sce's. obviously best way might be 1:1 deb-sce conversion, but it would be very mundane task.
-
If we ask 10 people what makes up a basic system, we would get 10 different answers. Same with a desktop, network, or ultimedia collection. Most folks here recognize the base and busybox as a basic system. I know what you are saying though. I would rather have a thread where folks posted package lists that make up their preference of a base, small desktop, large desktop, office, multimedia, etc. One can just copy and paste that list and import it.
As for duplicating files and libraries, that does not happen when using the dependency option which is the whole point of the recent changes. Regardless of the size of an sce, if it is used as a dependency then there will be no duplicate packages between it and sce's that are imported using it id as a dep. If you have a base.sce and use it as a dep for a desktop.sce, then any package dependencies needed by desktop.sce that are in base.sce will not be included in desktop.sce since they exist in base.sce. And if you import multimedia.sce using desktop.sce, then any package dependencies needed by multimedia that are in either base.sce or desktop.sce will not be included in multimedia. And so on and so forth, use multimedia.sce as a dep and any packages existing in base, desktop, and multimedia will not be included in the resulting sce. This goes for meta sce's as well as actual packages.
If libraries are being duplicated between sce's that are imported using the "-d" option, that is a bug and please report the specifics.
Thanks.
-
Added the ability to deal with circular dependencies, a safety measure. Please test.
-
Regardless of the size of an sce, if it is used as a dependency then there will be no duplicate packages between it and sce's that are imported using it id as a dep. If you have a base.sce and use it as a dep for a desktop.sce, then any package dependencies needed by desktop.sce that are in base.sce will not be included in desktop.sce since they exist in base.sce.
That i understand, and that works just fine. My problem is that so far no matter how i try, multiple packages will still share quite a lot of low level system packages that compose undefined base set, that i have to figure out by trial and error.
So far that set entails coreutils, zlib1g, bzip2, util-linux, gcc-base, libc6-bin, e2fsprogs and i am not sure what else. I keep seeing packages duplicating those debs, and to minimize duplication i have to redo my bottom level sce's. Finding that common denominator is mostly done through experimentation.
I would just like to have something resembling a system set of debootstrap wrapped up in one sce, but i am not sure if that is defined as package set. That would greatly shrink other sce's i am deriving from it.
-
Have you re-imported all your sce's from scratch? Because no matter how low level a package is, it should not be duplicated by dependent sce's..
-
Have you re-imported all your sce's from scratch? Because no matter how low level a package is, it should not be duplicated by dependent sce's..
no i am digressing about something else. there is no duplication between sce and its dependent packages.
i thought about that there are lots of low level packages introduced into each package's dependencies (exluding those you provide in dependencies). i wondered if there is no defined set for that base system in debian or a dcore metapackage of sorts for it.
-
I see you are wanting for there to be a meta "base" package that would eliminate lower level packages from being included in higher level apps.
What I recommend is to install your favorite sets of apps using the dependency option, then go through those apps and find which libs you don't want duplicated across the higher level apps. Say both firefox and leafpad and emelfm2 contain libc6. Then libc6 is one you would want in your base.
Ok, I will look at CorePlus and perhaps base some meat package sets on it's selection.
-
If no more bug reports, I will keep testing here and if I don't find anything I will make a release for x86 and Allwinner.
-
hi jason w,
sounds good :) and i like it!
perhaps you want to touch these two missing files?
tc@box:~/tmp$ touch .scedebs
tc@box:~/tmp$ touch .scedeps
or is it too quick and dirty?
thank you for your help and commitment.
-
Oh, yeah, those 2 files. I will find where they are being echoed and fix it.
-
I think I found it. Uploaded a new cut to release candidates.
-
Fixed a bug, uploaded new dCore.gz to release candidates.
-
hi jason w,
all works flawless for me.
this is an excellent milestone and gives a path to a highly usable dCore-system for every-day-tasks and highly specified applications.
thank you, keep on hacking :)
-
Great, it is working here too. I have some other ideas, but at this point I think we have a well tested set of changes that qualifies for a release. I will put it up tomorrow.