Suggestion: bootstrap hook for script (boot parameter + auto execution)

[tnussb]

First of all: I really love the modularity and the nomadic approach of TinyCore, but what I'm really missing is a kind of bootstrap hook for scripted customization.

usage of bootstrap hook:

• check for bootcode scripturl
• if bootcode is set download the script from given url using wget
• execute downloaded script as user tc at the end of the boot sequence

benefits of bootstrap hook:

• no more remastering necessary (pure nomadic!)
• installation and configuration of any system can be automated
• just a few script lines necessary to integrate this mechanism (small pain, great gain)
• completely diskless operation of customized tc instances
• customized appliances boil down to a single script (great for sharing setups!)

Isn't it worth integrating this kind of bootstrap hook?

[Rich]

Hi tnussb
You can set up a script that runs the output of   cat /proc/cmdline   through   grep   to search for any
custom boot code you care to use, and then act on it if found. There several ways you can get the
script to execute. You can call it from   /opt/bootlocal.sh. I believe if you place a script in  /home/tc/.X.d
it will execute automatically after booting.

[gerald_clark]

Scripts in .X.d run when X starts, not at boot.
There already exist many ways to execute custom scripts or extensions when booting from drives or network.

[tnussb]

Well, thanx for your replies, but I'm not really looking for a technical advice (sorry if the post misleads you).

I've already implemented this kind of mechanism by my own (it's quite easy), but what I'm really looking for is to get this mechanism integrated into the base distribution.

If it will be integrated in a future base distribution everyone could benefit from it without remastering by their own and without remastering every new upcoming version of tcl.

[Lee]

You can set up a script that runs the output of   cat /proc/cmdline   through   grep   to search for any
custom boot code you care to use

To counterfeit a phrase: "Don't ignore what's in core." 

Function getbootparam is defined in /etc/init.d/tc-functions which is sourced early on in the boot process.

For instance,

Code: [Select]

getbootparam tz
yields (on my system)

Code: [Select]

GMT+5
Last I tried it, it did a similarly fine job of handling my "custom" boot codes.  I have to admit I didn't try it right now (on TC4.0rc1) as I just in the last few days removed all of my custom boot codes - I found I wasn't using them any more.


Edit: I just saw your post, tnussb, that came in while I was typing.  So, while ability to parse custom boot codes from the command line is in base, the functionality you are looking for to act upon it is not.  But perhaps a mod to bootsync.sh, saved in you backup, would do the job?

[danielibarnes]

I'm not really looking for a technical advice

I've already implemented this kind of mechanism by my own

what I'm really looking for is to get this mechanism integrated into the base distribution.

I've seen this pattern so much that someone should address it in an FAQ or something.

Feel free to post ideas, but don't be offended if someone offers a way to do it without modifying the base. That's what "extensible" is - adding functionality without modifying the base.

Gerald proposed changes to base for loading extensions via nfs that were eventually integrated. You may get some insight from reading that thread.

[Rich]

Hi Lee

To counterfeit a phrase:

Don't you mean  "To coin a phrase:"?

Anyway, that's much better than what I suggested..

[Lee]

Don't you mean  "To coin a phrase:"?

Well, I didn't coin it myself - I shamelessly stole it from roberts' sig line.

It sort of like a cross  "code reusability"  and plagiarism. 

[tnussb]

I've seen this pattern so much that someone should address it in an FAQ or something.

Exactly because this pattern is required "so much" it shouldn't be just documented somewhere but become part of the base distro! This way anyone can benefit from it without remastering by their own.

For clarifications: it's meant to get totally rid off remastering. Just start from a standard distribution and supply an URL of a script as boot parameter which performs all required setup.

Actually I'm using this mechanism to boot my private cloud of dozens diskless tcl instances (with varying configurations) in a pure nomadic way. A local proxy is used to buffer TCEs and a webserver is used to serve the configs and the master control GUI for the kvm slave hosts ...

[danielibarnes]

Actually, I meant the pattern of "I have an idea that would be good for the base." There are lots of ideas, but not all of them are good for the base. It would be helpful to have a guide for contributors to evaluate their customizations.

When you say "remaster" I presume you are booting your machines with CD. Did you investigate PXE booting?

[tnussb]

Well, was I wrong to start this discussion to evaluate if it is "good for the base"? Would be a mail or private message to the maintainers better? If so, please tell me.

IMHO the idea of a post-boot hook by using a script downloaded from an url is the most open and generic approach for automated setups I can think of. If it's a bad idea, please tell me (and of course, I'm not offended by a rejection of the idea or anything else, I just want to discuss this topic). 

PXE booting only helps in distributing the kernel and it's boot parameters, but will not serve as a generic non intrusive post-boot hook. It's about a generic/standard way of implementing this post-boot hook I'm talking here. The required changes to the base would be only around 5-10 script lines with no penalities (if the execution time for an 'if-parameter-set' script line is not counted).

More clarifications:

The idea behind is to drive the nomadic approach within a (local/private) cloud to the max. Each cloud node behaves like a diskless/persistence-less thin client. It always boots from a tiny standard cd image (shared by dozens or hundreds of instances), fetches its specified setup script and runs it to finish its installation.

It's just a matter of supplying another script url as boot parameter to transform (for example) a mysql instance into a webserver. With a local proxy fetching/installing all required TCEs on-the-fly during booting is still fast enough and reduces the management effort of persistence hdd images to exactly zero.

This way the main management effort for a node boils down to exactly two settings: the cd image (which is the same for all nodes except for testing a new standard distro) and the url of the installation-on-the-fly script

Of course, data drives (for example: content of database/webserver) have to be managed, too, but this management is completely separated from the management of the worker nodes (type of setups). If a worker node of a kind finished booting it (for example a generic webserver), it "talks back" to the master controller to get a specific job.

[danielibarnes]

Well, was I wrong to start this discussion to evaluate if it is "good for the base"?

No, this is the right forum for the discussion. I didn't mean to imply otherwise.

PXE booting only helps in distributing the kernel and it's boot parameters, but will not serve as a generic non intrusive post-boot hook.

PXELINUX does more than load the kernel. It can load multiple cpio archives, and you can put your modifications in a cpio archive to be loaded in this way.

[gerald_clark]

tftplist and httplist boot codes can be used to load regular or custom extensions.
PXE boot is not required to use these options.

[tnussb]

tftplist and httplist boot codes can be used to load regular or custom extensions.

I read about this already. While these options are nice for standard installations they fail if any additionally step is required to finish the installation of an extension (for example: loading a kmap after kmaps.tcz is installed). Just specifying a lists of extensions to load is often not enough and/or not flexible enough compared to an external script which will be executed via bootsync.sh or (?)  bootlocal.sh.

Daniel: Of course, PXE booting can do more than just starting/distributing a kernel. I'm aware of that and for my use case it's enough.

My main point in general is flexibility and ease-of-use. Suppose you want to share a setup with other tc users on the internet (for education of newbies, knowledge transfer or similiar). While anyone could type in three lines after booting (wget + chmod + execute the script) its cumbersome to do it - at least when there is no copy&paste available. A single boot parameter is faster, its easier to automate and you haven't to tell everybody how to go through the hassle of setting up the pxe environment. It's about boiling down a complete appliance to an url (and a standard distro).

Am I really that far from reality, that there is no need for such a simple mechanism? Wouldn't HowTos for specific setups become much simpler and automatically testable with new distributions?

It's also about configuration management, you know? Actually if you have fine-tuned a setup by hand using persistence and/or a local installation it could be a quite time consuming job to upgrade to a new distribution or re-setup after a desaster. After fine-tuning by hand there is no single place that tells you what you have changed and where. Having your complete setup in a single file is much more comfortable and mimicks the RedHat kickstart way.

[maro]

@tnussb: Am I missing the point, but what stops you to put your script as a startup script (e.g. '/usr/local/tce.installed/my-startup') into a "home-grown" extension (e.g. 'my-startup.tcz')? You would then have to just ensure that this extension gets installed at system boot time (e.g. either via PXE or by becoming part of the ISO image, but you seem to dislike re-mastering). Therefore at system boot the script gets executed and all else follows from there (and everybody should be a happy camper ...).

I guess the smarts will have to be in either this first script or if it is just a generic one to negotiate with some "master" to possible load another more specific script. In any case all this is not really something a typical desktop user will need to do. And users that require this functionality would have the scripting knowledge to create such a solution for themselves.

So, what is it that current TC (or MC) is really lacking to support your vision?

EDIT: Just to be clear the 'my-startup.tcz' extension could be so generic (if it is just another step in the bootstrap process) that it pretty much never requires another update. In which case more smarts will need to be in the next script in the chain. But at least this way any ISO remastering would be a purely "mechanic" exercise (i.e. achievable via the most simplistic scripting).