Tiny Core Linux
Tiny Core Extensions => TCE Talk => Topic started by: julianb on December 05, 2009, 03:17:15 PM
-
one of my desires in using tinycorelinux is to have an ultra-quick boot. on the machine I installed it on (eee pc 900a, install on 4gb ssd) the fresh install boot-time was quick, probably less than 20 seconds. after i installed applications, the boot became much slower. (maybe 40 seconds) i'm still a newb with this tinycore stuff.
so anyway, i'm interested in making the machine keep extensions sitting on disk, ready to load, but have them only load when i open up an app that uses them.
neat thing about tiny core linux, if startup and shutdown are ultra-quick, there isn't a lot of need for a sleep mode. minimizing energy used at startup and shutdown is also desirable for me, because i am setting these machines up to be used in an off-grid environment where power is from solar panels and a solar charged battery bank.
-
A trick I use, is to put only the extensions that are absolutely necessary in/tce, and the rest that I need as occasion arises, I put in /tce/optional. Importantly, you need to take care of dependencies as well, so all those extensions that are in tce, must be there with their dependencies, and all the rest you can also move to /tce/optional.
I built a custom wbar config file and load this instead of the one that is contructed at every boot. I use the boot time code 'noicons' to disable the starting of default wbar. The custom wbar config stuff, I put in a separate home folder which I call /.mywbar. I copy all the extensions' icons to this folder. I write brief scripts that will load the extension on invoking the wbar icon press. The loading first checks to see if the extension is already loaded.
#!/bin/sh
[ ! -f /usr/local/tce.installed/OSS ] && tce-load -i /mnt/sda7/tce/optional/OSS.tczm
[ ! -f /usr/local/tce.installed/flash10 ] && tce-load -i /mnt/sda7/tce/optional/flash10.tcz
[ ! -f /usr/local/tce.installed/opera10 ] && tce-load -i /mnt/sda7/tce/optional/opera10.tcz
opera &
Here is a typical wbar config file entry:
i: /home/tc/.mywbar/opera10.png
t: Opera
c: exec /home/tc/.mywbar/opera.sh
(Note that the Opera example Flash does not work, I need to debug it.)
Using this, I have turbo-charged my boot, and in most cases, no noticeable penalty when invoking apps for the 1st time. There may be half a seconbd lag the fist time I call Opera which is a big app, but in most of the others there is almost nothing.
The stuff I have in /tce are jwm and the wireless network drivers, along with their dependencies.
To update extensions that are in /tce/optional, use jpeters' update extension script. Find it in
http://forum.tinycorelinux.net/index.php?topic=1698.0
Using the above, my boot times are essentially the same as a raw tc.
-
Having them load only when used means the system must be able to detect when they are needed, but how would you communicate to the system you want to use them (ones without an icon or menu entry)?
For your situation, I'd suggest adding lines to /opt/bootlocal.sh to load the extensions. It is executed in the background as TC boots. With loop mounting there is no memory overhead, so you can load up to 255 extensions if desired and it won't slow down your boot.
Something like:
#!/bin/sh
# Wait for init to finish
wait $PPID
# Load extensions
for e in /mnt/sda1/extensions/*; do
su -c "tce-load -i $e" tc
done
if your extensions are stored in /mnt/sda1/extensions/.
-
julianb said something I was looking for from the start:
"so anyway, i'm interested in making the machine keep extensions sitting on disk, ready to load,
but have them only load when i open up an app that uses them."
that TC doesn't do it simply yet ( especially the loading on demand )
-download once, use many times and on demand.
appbrowser loads/install but doesn't launch an app.
What julianb is talking about I guess, is simply something like 'clicking on the app's icon' will load-launch or just launch the app (as jur did in his scripts)
and this is THE moment when the system reacts to the user demand; no guessing no detection.
-
so anyway, i'm interested in making the machine keep extensions sitting on disk, ready to load,
but have them only load when i open up an app that uses them.
Since no resources (except loop devices) are consumed by a loaded extension, there is no benefit to waiting until the app is used to load it. Loading them in the background at startup becomes the simplest solution.
-
danielibarnes, thanks for responding. When I first installed the system, i didn't realize that it might be beneficial to AVOID using the "install" option in the application browser, for each app I wanted.
What i'm gathering from your post is taht you recommend following the "third mode of installation ( TCE/Mount ) " mentioned at the tinycore linux core concepts page ( http://www.tinycorelinux.com/concepts.html ) because that keeps your system capable of booting essentially equally quickly with extensions as with no extensions.
is that right?
julian
-
If you want to change to "mount" just edit or delete the file tcz2ram.lst in yout tce directory.
No need to re-download anything.
-
you recommend following the "third mode of installation ( TCE/Mount ) "
Yes. "TCE/Install" will consume memory when installed because extensions are copied to memory instead of mounted. This is really only useful if the extensions are stored on a medium which might need to be removed, like an CD-ROM or USB drive. If they are stored on a hard drive (like your SSD) then "TCE/Mount" is the best solution.
just edit or delete the file tcz2ram.lst in yout tce directory.
You'll want to rename your tce directory as well because the startup will automatically load all extensions in any top-level directory named tce.
-
For your situation, I'd suggest adding lines to /opt/bootlocal.sh to load the extensions. It is executed in the background as TC boots. With loop mounting there is no memory overhead, so you can load up to 255 extensions if desired and it won't slow down your boot.
I'm not sure you are correct. If you have a lot of extensions especially big ones (say 10M typically, such as Thunderbird, Opera, wine-gl and so on, then there is a lot of symlinks to be made and it takes time. I know because I already tried something very similar to what you suggest. There is no getting away from that. It is either wait a long time till all the extensions are loaded at once, or spread that loading time around into small almost imperceptible little bits.
-
If you have a lot of extensions ... then there is a lot of symlinks to be made and it takes time.
Definitely true, but at least the system doesn't pause during startup to load extensions. If you find that it slows the system down even when running in the background, then you can try to nice the process:
su -c "nice tce-load -i $e" tc
-
Nice - I'll have to look into that.
But running stuff in the background is no magic bullet. The processor still has to do the same work. Even if you have multiple processors there is still only one disk and only one data bus so it of necessity is a serial process. Putting stuff into the background merely means it will be processed when higher priority stuff is done.
The problem with booting is that all the stuff effectively has the same priority if you are waiting for access to the OS.
-
About on demand loading, to create the icon/menu entry one would need to mount, read, and umount the extension, to copy the icon and other necessary info. This wouldn't really make sense, since doing the work at once is faster in total.
-
It is possible to build hyper fast-booting hyper-performant setups using Tiny Core
This is the way I do it
1. first I create a loop-mountable ext2 filesystem, that will receive the resources from extensions I do not want to keep in RAM. Then I loopmount it in the core file system.
2. Then I start with temporarily installing all extensiont I want in RAM.
3. Next step is moving all files/directories that I do not want to be in RAM in the definitive system to the loopmount, leaving behind a symlink to the file. For performance and debug reasons I leave every file needed to boot to the desktop in RAM + ROX filer + Leafpad, so I can debug when there is something wrong with the loopmount during development.
4. Last step is to remaster all what is left in the root filesystem, including the symlinks to the loopmount, into a big initrd.
5. I do not compress the remaster result to tinycore.gz but leave the file uncompressed - this boots faster. The uncompressed remastered initrd is currently 65 M.
Of course I have a build script that does the hard work. I can rebuild a system in a couple of minutes.
During boot, I loopmount (Read-Only) the file system with the resources in the bootlocal script.
Advantages
- system has no work during boot time, everything is prepared
- only one loopmount covering all apps (actually I have two loopmounts, the second one being persistent home)
- boot is essentally one sequential read of a big file - very fast - you don't hear your disk heads rattling
- extremely reponsive system as all essential files are pre-loaded in RAM
- Basic Tinycore "engine" is untouched - you still have all the features, can use Appbrowser etc. etc.
- System is pristine after boot as writes to the root file system are discarded after boot and the resources loopmount is Read-Only.
Disadvantages
- more maintenance, especially tweaking and hand-optimizing the build script
- only way to de-install an app is to rebuild the system
- higher RAM consumption but this is a trade-off with reponsivity. My system uses 105M when booted to the ROX desktop. This is not a setup for memory-challenged systems (it's already very usable with 512 M, but 1G recommended)
Results
My setup consists of a 100% pure NTFS Windows XP system without any Linux partitions, booting TC 2.6 with GRUB4DOS. For maximum performance it is best to defragment your Windows filesystem when your system is definitive.
My TC build has Xorg, SLIM logon manager (it's a multi-user setup), JWM, ROX Filer and pinboard, OSS sound, Leafpad, Xarchiver, Firefox browser with Flash, Thunderbird e-mail client, WINE, MPlayer, XMMS stuffed with codecs and a DVD playback module, SuperTux and a couple of other games, epdfview PDF Reader, galculator, gpicview applet to display photos and some stuff I probably forgot. So it's not really a "tiny" Tiny Core system.
My boot time from first kernel message to the logon screen is about 12 seconds on my seven year old 2.4 Ghz Pentium IV system with 512M RAM. After logging on the desktop appears quasi-immediately. Firefox starts in 3 seconds. Most other apps start instananeously. On my son's system (five year old - 1G RAM - single user setup) boot time is ten seconds.
It is my daily system since last Spring. I need to boot Windows only a couple of times per month. It is very stable but memory contention is a problem when loading extremely heavy Web pages (I have no swap file and my system has only 512M). It works so well that even my wife has switched to it instead of Windows (I have given the system an XP-like theme for her).
-
Of course I have a build script that does the hard work.
Hmmm... Any chance you could post that in the programming/scripting area?
-
For fast boot why not just use "suspend" (hibernate) ?
-
About on demand loading, to create the icon/menu entry one would need to mount, read, and umount the extension, to copy the icon and other necessary info. This wouldn't really make sense, since doing the work at once is faster in total.
Perhaps you misunderstood... the copying of icons is a once-only thing, then re-used every time. Mounting extensions is a once-only thing, at run time.
The advantage of my method is I don't need to remaster anything at all, dead simple scripts as shown in my earlier post, updating extensions or tc itself is as simple as before, and my booting time is essentially the same as with a completely raw tc. For example, booting from usb, 8 seconds from when the first boot message appears to the ready desktop. I have rox, opera with flash, OSS, thunderbird, vlc, mtpaint, couple of games, pcmanfm, wine, beaver, geany... in fact the booting time is completely unaffected by the number of extensions.
-
Hmmm... Any chance you could post that in the programming/scripting area?
Build script not fit for public consumption at this time. Contains too much stuff specific to my setup. Also loads non-published extensions. Useless in the current form. So needs clean-up before publishing.
Will think of a way to do a kind of proof-of concept if sufficient people show interest.
-
Jur, I really like your system, as it seems simple and relatively easy for others to immitate. Let me see if I understand it.
1) Your /tce directory only includes the stuff that you are pretty sure you want to use every time you boot up your computer. In my case that would be 915resolution.tcz, OSS.tczm, mc.tcz, opera.tcz, a couple of wireless.tczl files, and a few dependencies.
2) Everything else is invoked on an as-needed basis from your custom wbar. (Can flash for Opera be invoked in that way, or should it be loaded with Opera in the standard boot?
3) Your custom wbar is invoked . . . how? By a line in bootlocal.sh?
Would you be willing to post more examples of the custom wbar entries? The example of Opera may be something I can use as a model for all others, but I'd like to confirm that with a couple of other examples.
Thanks for sharing a neat idea.
-
My /tce folder contains 915resolution, jwm and its dependencies, mc and its dependencies, the wireless files, and acpid - so only the items needed during boot.
OSS and flash10 I have also now moved to /optional, since OSS is needed only for vlc player or flash10, so it is only mounted when either of these apps are mounted. I still have to include flash10's dependencies because it doesn't have a list, only getflash10 has a list but not all of those are needed, so I am just sorting that out.
So here is the custom wbar config file:
# The Bar && Font && Font size (11)
i: /usr/share/wbar/osxbarback.png
t: /usr/share/fonts/luxisr/11
c:
i: /usr/share/wbar/aterm.png
t: Aterm
c: exec aterm
i: /usr/share/wbar/cpanel.png
t: Panel
c: exec cpanel
i: /usr/share/wbar/appbrowser.png
t: Apps
c: exec appbrowser
i: /home/tc/.mywbar/beaver2.png
t: beaver2
c: exec /home/tc/.mywbar/beaver2.sh
i: /home/tc/.mywbar/gtkfind.png
t: gtkfind
c: exec /home/tc/.mywbar/gtkfind.sh
i: /home/tc/.mywbar/mtpaint.png
t: mtpaint
c: exec /home/tc/.mywbar/mtpaint.sh
i: /home/tc/.mywbar/opera10.png
t: Opera
c: exec /home/tc/.mywbar/opera.sh
i: /home/tc/.mywbar/rox-filer.png
t: rox-filer
c: exec /home/tc/.mywbar/rox-filer.sh
i: /home/tc/.mywbar/thunderbird.png
t: Thunderbird
c: exec /home/tc/.mywbar/thunderbird.sh
i: /home/tc/.mywbar/vlc.png
t: vlc
c: exec /home/tc/.mywbar/vlc.sh
i: /home/tc/.mywbar/xmahjongg.png
t: mahjongg
c: exec /home/tc/.mywbar/xmahjongg.sh
I use this command in .xsession to invoke wbar:
wbar -nofont -vbar -pos left -jumpf -0.1 -bpress -config /home/tc/.mywbar/tce.wbar
Here is the wine invoker:
#!/bin/sh
[ ! -f /usr/local/tce.installed/wine-gl ] && tce-load -i /mnt/sdb1/tce/optional/wine-gl.tczl
wine $1 &
beaver has the same looking code.
To get windows apps to use wine even before it is mounted, I open rox, navigate to the directory containing the win app, then set the default run application as the file in .mywbar - so it will mount it first if not already mounted and then run the app the normal way.
Even regularly run apps such as opera resides in /optional.
To update extensions in /optional, I use jpeters' script which he kindly modded so it included extensions in /optional.
-
I'm curious why you don't just keep everything in your tce_dir and load a sparse symlinked group. By leaving /opt/.tce_dir set to your main tce_dir, update/mount/install/download functions all work as usual. "tce-load -i app" ( I use a "load" script ), loads any additional extension from the tce_dir that you need on the fly, complete with it's icon and menu.
-
Thanks, jur. I'm going to give your setup a try--maybe shave a few seconds off my boot time.
And jpeters, I'm confident you also have a good way of speeding system load, but I don't yet understand it. If everything is in the tce directory, everything loads. Right? That's what I do now, except for stuff in my optional directory, which I load with appbrowser as needed. I don't think I'm comprehending how you suggest modifying things.
-
Thanks, jur. I'm going to give your setup a try--maybe shave a few seconds off my boot time.
And jpeters, I'm confident you also have a good way of speeding system load, but I don't yet understand it. If everything is in the tce directory, everything loads. Right? That's what I do now, except for stuff in my optional directory, which I load with appbrowser as needed. I don't think I'm comprehending how you suggest modifying things.
You can check out the load-group script, which creates a symlinked tce-group from a load.grp file placed in your tce_dir. Tce_group is added to your bootcodes (there's a script to add and remove it from your menu.lst file) I also included a script to change /opt/.tce_dir back to the main /tce_dir so that update/install/mount functions act upon the main directory instead of the symlinks. There's also a "load" script, which performs "load -i app" for on-the-fly mounts from the tce_dir.
-
For fast boot why not just use "suspend" (hibernate) ?
Stand-by (suspend to RAM) might be good for some folks, but if the machine boots in ten seconds it's barely needed. For off-grid solar powered computing, (which is where my tiny core linux machines will be used) using suspend to RAM requires too much power. I also don't know how to make tinycorelinux suspend to RAM.
If there's a hibernate (suspend to hard drive) feature for tiny core linux, I don't know how to use it AND I don't see how that would be faster than an ordinary shutdown / startup on a well-set-up tinycore system.
-
For fast boot why not just use "suspend" (hibernate) ?
Stand-by (suspend to RAM) might be good for some folks, but if the machine boots in ten seconds it's barely needed. For off-grid solar powered computing, (which is where my tiny core linux machines will be used) using suspend to RAM requires too much power. I also don't know how to make tinycorelinux suspend to RAM.
If there's a hibernate (suspend to hard drive) feature for tiny core linux, I don't know how to use it AND I don't see how that would be faster than an ordinary shutdown / startup on a well-set-up tinycore system.
I could not agree more. These are needed for Windows and other OS that take minutes to boot.
If these are stock 990a 4GB SSDs they have the slowest SSDs I have encountered. So writing state to them would be quite slow.
julianb, I hope you tried mount style extensions before you embark on scripting. IMO scripting solutions presented so far are better suited for static extension collections.
You could also try adding the boot code of elevator=noop to see if that helps boot and run time.
-
julianb, I hope you tried mount style extensions before you embark on scripting. IMO scripting solutions presented so far are better suited for static extension collections.
You may recall that I made a suggestion for improving boot times in this thread:
http://forum.tinycorelinux.net/index.php?topic=3367.0
I have verified for myself that it makes a massive difference; if not using Xorg, I have essentially the same boot time as with a raw tcl.
So by adding an installation option to extensions, the user could opt for having the lightning-fast boot that tcl offers while still having all the extensions they want. Based on the trivial amount of scripting I have to do when adding another app, I imagine this would be very easy to implement on tcl.
Of course it would add new user confusion but that can be alleviated by explanation.
Since distros are often compared on the basis of booting times, and fast boot times make headlines, I think this is a very worthwhile feature to have as standard in tcl.
-
julianb, I hope you tried mount style extensions before you embark on scripting. IMO scripting solutions presented so far are better suited for static extension collections.
You may recall that I made a suggestion for improving boot times in this thread:
http://forum.tinycorelinux.net/index.php?topic=3367.0
I have verified for myself that it makes a massive difference; if not using Xorg, I have essentially the same boot time as with a raw tcl.
So by adding an installation option to extensions, the user could opt for having the lightning-fast boot that tcl offers while still having all the extensions they want. Based on the trivial amount of scripting I have to do when adding another app, I imagine this would be very easy to implement on tcl.
Of course it would add new user confusion but that can be alleviated by explanation.
Since distros are often compared on the basis of booting times, and fast boot times make headlines, I think this is a very worthwhile feature to have as standard in tcl.
Thank you Jur for initiating this thread. It has brought out several good ideas as to how to manage booting with extensions.
I have been using the script driven wbar idea for a long time to achieve dynamic extension management. By dynamic I mean installing and uninstalling extensions. I went down this path mainly because I was booting from a floppy disk which was painfully slow. I have since moved on to using a netbook so I don't have the slow boot anymore but I still use my dynamic extension management scripts for Samba and VNC . I use two color wabr icons to denote install or uninstall actions.
I posted my ideas a long time ago here:
http://forum.tinycorelinux.net/index.php?topic=1705.0
http://forum.tinycorelinux.net/index.php?topic=1693.0
The above has been changed of course since I posted the information to keep pace with tinycore changes.
Anyway, I agree with your approach albeit for different reasons and would also like to see more discussion on the future of extension management for tinycore.
Edit: Just to be clear my current install scripts use mount not copy to the file system.
-
I stand by what I wrote. I think better promotion of "mount" style extension loading over "install" (copy) goes along way towards better performance. I am going to do so for 2.7.
Customized scripting solutions are welcomed in the scripting section of the board.
-
hp netbooks generally have ultrafast boot (HP QuickWeb)
-
one of my desires in using tinycorelinux is to have an ultra-quick boot. on the machine I installed it on (eee pc 900a, install on 4gb ssd) the fresh install boot-time was quick, probably less than 20 seconds. after i installed applications, the boot became much slower. (maybe 40 seconds) i'm still a newb with this tinycore stuff.
so anyway, i'm interested in making the machine keep extensions sitting on disk, ready to load, but have them only load when i open up an app that uses them.
neat thing about tiny core linux, if startup and shutdown are ultra-quick, there isn't a lot of need for a sleep mode. minimizing energy used at startup and shutdown is also desirable for me, because i am setting these machines up to be used in an off-grid environment where power is from
Really liked your set up.. I know thread is bit old but I was searching for similar sort of set up and hope it works.