Tiny Core Linux
Tiny Core Base => TCB Talk => Topic started by: tnussb on September 12, 2011, 03:03:38 PM
-
First of all: I really love the modularity and the nomadic approach of TinyCore, but what I'm really missing is a kind of bootstrap hook for scripted customization.
usage of bootstrap hook:
- check for bootcode scripturl
- if bootcode is set download the script from given url using wget
- execute downloaded script as user tc at the end of the boot sequence
benefits of bootstrap hook:
- no more remastering necessary (pure nomadic!)
- installation and configuration of any system can be automated
- just a few script lines necessary to integrate this mechanism (small pain, great gain)
- completely diskless operation of customized tc instances
- customized appliances boil down to a single script (great for sharing setups!)
Isn't it worth integrating this kind of bootstrap hook?
-
Hi tnussb
You can set up a script that runs the output of cat /proc/cmdline through grep to search for any
custom boot code you care to use, and then act on it if found. There several ways you can get the
script to execute. You can call it from /opt/bootlocal.sh. I believe if you place a script in /home/tc/.X.d
it will execute automatically after booting.
-
Scripts in .X.d run when X starts, not at boot.
There already exist many ways to execute custom scripts or extensions when booting from drives or network.
-
Well, thanx for your replies, but I'm not really looking for a technical advice (sorry if the post misleads you).
I've already implemented this kind of mechanism by my own (it's quite easy), but what I'm really looking for is to get this mechanism integrated into the base distribution.
If it will be integrated in a future base distribution everyone could benefit from it without remastering by their own and without remastering every new upcoming version of tcl.
-
You can set up a script that runs the output of cat /proc/cmdline through grep to search for any
custom boot code you care to use
To counterfeit a phrase: "Don't ignore what's in core." ;D
Function getbootparam is defined in /etc/init.d/tc-functions which is sourced early on in the boot process.
For instance,
getbootparam tz
yields (on my system)
GMT+5
Last I tried it, it did a similarly fine job of handling my "custom" boot codes. I have to admit I didn't try it right now (on TC4.0rc1) as I just in the last few days removed all of my custom boot codes - I found I wasn't using them any more.
Edit: I just saw your post, tnussb, that came in while I was typing. So, while ability to parse custom boot codes from the command line is in base, the functionality you are looking for to act upon it is not. But perhaps a mod to bootsync.sh, saved in you backup, would do the job?
-
I'm not really looking for a technical advice
I've already implemented this kind of mechanism by my own
what I'm really looking for is to get this mechanism integrated into the base distribution.
I've seen this pattern so much that someone should address it in an FAQ or something.
Feel free to post ideas, but don't be offended if someone offers a way to do it without modifying the base. That's what "extensible" is - adding functionality without modifying the base.
Gerald proposed changes to base for loading extensions via nfs that were eventually integrated. You may get some insight from reading that thread.
-
Hi Lee
To counterfeit a phrase:
Don't you mean "To coin a phrase:"?
Anyway, that's much better than what I suggested..
-
Don't you mean "To coin a phrase:"?
Well, I didn't coin it myself - I shamelessly stole it from roberts' sig line.
It sort of like a cross "code reusability" and plagiarism. :)
-
I've seen this pattern so much that someone should address it in an FAQ or something.
Exactly because this pattern is required "so much" it shouldn't be just documented somewhere but become part of the base distro! This way anyone can benefit from it without remastering by their own.
For clarifications: it's meant to get totally rid off remastering. Just start from a standard distribution and supply an URL of a script as boot parameter which performs all required setup.
Actually I'm using this mechanism to boot my private cloud of dozens diskless tcl instances (with varying configurations) in a pure nomadic way. A local proxy is used to buffer TCEs and a webserver is used to serve the configs and the master control GUI for the kvm slave hosts ...
-
Actually, I meant the pattern of "I have an idea that would be good for the base." There are lots of ideas, but not all of them are good for the base. It would be helpful to have a guide for contributors to evaluate their customizations.
When you say "remaster" I presume you are booting your machines with CD. Did you investigate PXE booting?
-
Well, was I wrong to start this discussion to evaluate if it is "good for the base"? Would be a mail or private message to the maintainers better? If so, please tell me.
IMHO the idea of a post-boot hook by using a script downloaded from an url is the most open and generic approach for automated setups I can think of. If it's a bad idea, please tell me (and of course, I'm not offended by a rejection of the idea or anything else, I just want to discuss this topic).
PXE booting only helps in distributing the kernel and it's boot parameters, but will not serve as a generic non intrusive post-boot hook. It's about a generic/standard way of implementing this post-boot hook I'm talking here. The required changes to the base would be only around 5-10 script lines with no penalities (if the execution time for an 'if-parameter-set' script line is not counted).
More clarifications:
The idea behind is to drive the nomadic approach within a (local/private) cloud to the max. Each cloud node behaves like a diskless/persistence-less thin client. It always boots from a tiny standard cd image (shared by dozens or hundreds of instances), fetches its specified setup script and runs it to finish its installation.
It's just a matter of supplying another script url as boot parameter to transform (for example) a mysql instance into a webserver. With a local proxy fetching/installing all required TCEs on-the-fly during booting is still fast enough and reduces the management effort of persistence hdd images to exactly zero.
This way the main management effort for a node boils down to exactly two settings: the cd image (which is the same for all nodes except for testing a new standard distro) and the url of the installation-on-the-fly script
Of course, data drives (for example: content of database/webserver) have to be managed, too, but this management is completely separated from the management of the worker nodes (type of setups). If a worker node of a kind finished booting it (for example a generic webserver), it "talks back" to the master controller to get a specific job.
-
Well, was I wrong to start this discussion to evaluate if it is "good for the base"?
No, this is the right forum for the discussion. I didn't mean to imply otherwise.
PXE booting only helps in distributing the kernel and it's boot parameters, but will not serve as a generic non intrusive post-boot hook.
PXELINUX does more than load the kernel. It can load multiple cpio archives, and you can put your modifications in a cpio archive to be loaded in this way.
-
tftplist and httplist boot codes can be used to load regular or custom extensions.
PXE boot is not required to use these options.
-
tftplist and httplist boot codes can be used to load regular or custom extensions.
I read about this already. While these options are nice for standard installations they fail if any additionally step is required to finish the installation of an extension (for example: loading a kmap after kmaps.tcz is installed). Just specifying a lists of extensions to load is often not enough and/or not flexible enough compared to an external script which will be executed via bootsync.sh or (?) bootlocal.sh.
Daniel: Of course, PXE booting can do more than just starting/distributing a kernel. I'm aware of that and for my use case it's enough.
My main point in general is flexibility and ease-of-use. Suppose you want to share a setup with other tc users on the internet (for education of newbies, knowledge transfer or similiar). While anyone could type in three lines after booting (wget + chmod + execute the script) its cumbersome to do it - at least when there is no copy&paste available. A single boot parameter is faster, its easier to automate and you haven't to tell everybody how to go through the hassle of setting up the pxe environment. It's about boiling down a complete appliance to an url (and a standard distro).
Am I really that far from reality, that there is no need for such a simple mechanism? Wouldn't HowTos for specific setups become much simpler and automatically testable with new distributions?
It's also about configuration management, you know? Actually if you have fine-tuned a setup by hand using persistence and/or a local installation it could be a quite time consuming job to upgrade to a new distribution or re-setup after a desaster. After fine-tuning by hand there is no single place that tells you what you have changed and where. Having your complete setup in a single file is much more comfortable and mimicks the RedHat kickstart way.
-
@tnussb: Am I missing the point, but what stops you to put your script as a startup script (e.g. '/usr/local/tce.installed/my-startup') into a "home-grown" extension (e.g. 'my-startup.tcz')? You would then have to just ensure that this extension gets installed at system boot time (e.g. either via PXE or by becoming part of the ISO image, but you seem to dislike re-mastering). Therefore at system boot the script gets executed and all else follows from there (and everybody should be a happy camper ...).
I guess the smarts will have to be in either this first script or if it is just a generic one to negotiate with some "master" to possible load another more specific script. In any case all this is not really something a typical desktop user will need to do. And users that require this functionality would have the scripting knowledge to create such a solution for themselves.
So, what is it that current TC (or MC) is really lacking to support your vision?
EDIT: Just to be clear the 'my-startup.tcz' extension could be so generic (if it is just another step in the bootstrap process) that it pretty much never requires another update. In which case more smarts will need to be in the next script in the chain. But at least this way any ISO remastering would be a purely "mechanic" exercise (i.e. achievable via the most simplistic scripting).
-
@Maro: You are missing at least my point ;) It's not about the techniques of customization. There are various approaches and I'm experienced enough to realize them. My point is that for a real "nomadic distribution" there is still something missing, namely the explained hook mechanism via a boot parameter. It's about plug-and-play without any required customization - a one-stop-solution. Whatever is required to be performed during an installation/setup (what you normally type in) comes from an single external hosted script.
Suppose you want to guide a complete newbie to setup a special installation. The simplest way is to point him to download a standard distro and let him type in a web url of a kickstart-like file on the bootscreen. This way, appliances don't require any "large" downloads (customized distros) or require an unexperienced user to type in some cryptic lines. Just enter a single line at the boot prompt. That's all.
When it comes to automation of hundreds of nodes a single boot parameter rocks, too.
A special tc extension is fine, but requires still more "setup" work (and must be maintained in some way, of course).
-
Some time ago, as an exercise of curiosity when I wanted to see how early I could gain control of the boot process (without remastering), I created an extension called onload.tcz which simply checks for the existence an external script named onload.sh (which is -not- included in the extension) and sources it if found (otherwise exiting silently). Essentially a hook into the extension loading process.
It runs automatically at whatever point in the extension loading sequence you place it - at the beginning of onboot.lst it runs early, at the end of onboot.lst, or as a dep of another extension, it runs a little later. Loaded manually it runs when ever you want - but its kind of pointless then. If you wanted it to download additional script from the net, you might have to wait for the network to be available.
-
The way I see it:
- security nightmare. Think of some website somewhere telling newbs to boot "fetchscript=1.2.3.4/script.sh"
- how is adding a small edit like that to the iso less complex than setting up a web server?
I get the feel you have a special situation (hundreds of nodes?) and for some reason do not want to do any customization to the standard isos.
What I read of Red Hat Kickstart says it's NFS only, on the local network. Is that any different to loading extensions from NFS, again from the lan?
-
I get the feel you have a special situation (hundreds of nodes?) and for some reason do not want to do any customization to the standard isos.
That feeling get you completely wrong, because I have already realized it. In a matter of sense it works quite similar to the httplist bootcode, but instead of fetching just a list of tc extensions to install it fetches a script and executes it. This way not only extension installation can be performed automatically, but also all required configuration steps (whatever they are).
My intention was to contribute the idea of an ease-of-use automated setup mechanism. If it's not wanted, it's fine. I don't bother.
And btw, If you call that a "security nightmare" you can call almost everything a security nightmare (prefactored appliances, special distros, any package you install from the net without checking its sourcecode etc.).
Kickstart: By "like kickstart" I meant, that you have a single file which contains all the necessary installation and setup steps, so you can recreate a system from scratch using just that file at any time. This increases maintainability alot.
-
What I read of Red Hat Kickstart says it's NFS only
Kickstart can do http as well. In fact, the primary reason I use Tiny Core is to serve files via lighttpd for kickstart-based CentOS and Red Hat installations.
Other distributions already offer what tnussb is suggesting (Red Hat kickstart, SUSE AutoYaST, HP-UX Ignite, etc.), so perhaps someone can investigate how they accomplish this and craft something similar for Tiny Core.
A properly generalized solution will require more than three lines, but I think it would be useful.
-
I've re-read the thread, and I still fail to see how doing a scripted customization once every node boot is better compared to doing it just once per version update, and then distributing that ready-to-go and tested image. Doing the work only once would fullfill most of the points mentioned, as far as I see.
-
daniel: At least someone gets my point :)
curaga: It doesn't matter if you do it each time during boot (for diskless thin clients) or do it just once and distribute an image. The point is: even in the case of an image its much more comfortable (faster, easier to maintain, etc.pp.) if you can do it scripted without any manual interaction.
Systems like http://susestudio.com/ (http://susestudio.com/) depends heavily on such a bootstrap script hook. Within a web gui you select your system's options and configure whatever is necessary and then an image is "automagically" produced for you. Such a hook opens up endless possibility. It's all about automation and flexibility ...
-
The point is: even in the case of an image its much more comfortable (faster, easier to maintain, etc.pp.) if you can do it scripted without any manual interaction.
That's quite possible, we have several such scripts in the programming area, and writing your own is about the same work as writing one to run on boot.
The remasters of svolli also come like that: both as images, and as automatic scripts.
I haven't used Suse Studio personally, but if it's anything like the Nimblex creator or rom-o-matic, it wouldn't require such a hook.