Starting, intitializing system services

[bmarkus]

I have a problem with the startup mechanism. How to start a service at the end of the boot procedure?

Script in /usr/local/tce.installed runs too early, possibly ealier than another required package loaded, so not possible to check the environment, do conditional initialization, run scripts, etc.

bootlocal.sh is a single script,  changing it would cause problems for other applications and also easily would lead to concurent change by different extensions.

One solution would be to change tc-config and to run startup scripts in a directory at the end of the boot.  This is a well working solution in ordinary LINUX, see /etc/rc.d

[curaga]

Having things in a single script instead of a directory is called BSD-style init, and some linux distros use it too. Our way could also be described as such, with tc-config being the main init script, and bootlocal.sh meant for the user, at the end of the boot.

Is there some specific problem with bootlocal.sh?

edit: for only starting an extension service, you could create an extension named zzsomething.tce with the startup script in tce.installed. Since the extensions are loaded alphabetically, this would load last.

[bmarkus]

Having things in a single script instead of a directory is called BSD-style init, and some linux distros use it too. Our way could also be described as such, with tc-config being the main init script, and bootlocal.sh meant for the user, at the end of the boot.

Yes, I know SLACKWARE 

This is fine to offer bootlocal.sh for the user. But it is not a system wise solution. Root problem is the modularity. If you have a custom distro with a fix package collection you can adjust tc-config with no problem.

Is there some specific problem with bootlocal.sh?

Sure. For example hald do not starts if usb.ids and pci.ids files are not available, but they are loaded later than hal loaded and its initialization script started, This is just one example, for sure there are more. Adding it to bootlocal.sh would require to add bootlocal.sh to the extension which is not the right solution as it is for the user. A clever script can be used to insert necessary parts to bootlocal.sh but then other similar applications have to follow rules, avoid concurrent changes, etc. It has to be hidden for the user, must work automatically and safely 

edit: for only starting an extension service, you could create an extension named zzsomething.tce with the startup script in tce.installed. Since the extensions are loaded alphabetically, this would load last.

Do not know it is sequential or done parallel. If parallel,  not 100% sure initialization will be finished in time.

[curaga]

It is sequential among the extension type. I don't remember right now though if tcz were processed after tce or the other way around.

[bmarkus]

OK. For sure there is solution in the current environment, but still I have a bit bad feeling. First time using TC/MC   

[helander]

Hi,

I share your feeling about TC/MC possibly missing something here. According to quick study of tc-config, this is basically what happens.

   1. All extensions with .core in its name are loaded and their install scripts are run. For each extension loaded
       its install script is executed immediately and before the next .core extension is loaded.  The order of the loading
       is based on the output of an "ls"-command.

  2. All other (non- .core) extensions are loaded but their install scripts are not run in association with the load, but later
     (see 3). The order of the load becomes insignificant unless you have content for the same node in the filesystem
     in multiple extensions - but you should really avoid having this.

  3 . The install scripts for the non-.core extension are executed. The order is based on the output of a "find" command.


One solution to the controlled ordering problem could be to do something like this

   - For extensions that needs to have its initialization done in an ordered way, place
     the init logic in a script under /usr/local/bin (or maybe somewhere else)
   - Install an extension that in its init scripts runs the scripts above in the proper order.
     If some of the extensions in the order are missing, just skip over to the next one.

On the downside, you will always have to have this additional extension loaded in order to have the
individual extensions to be initialized. However it will give control over the order. The ordering extension could
also look at some boot parameter that provides ordering info, e.g. "extorder=ext1,ext2,ext3, ...."


/Lars

[bmarkus]

Lets see first what Robert is saying.

[helander]

Lets see first what Robert is saying.

Sounds like a good idea.

Meanwhile, another solution could be to add two bootcodes (must be managed by tc-config):

      "firstinit=<name of extension who's install script should run first among all loaded (.core excluded)>"
      "lastinit=<name of extension who's install script should run last among all loaded (.core excluded)>"

In the install script of firstinit, one could unset the -x flag of the install script of the extensions that
will be executed by by lastinit.

Another variant:
   Boot option initorder lists the install script ordering for a set of extensions. tc-config will make sure that the
   install script execution of these scripts will happen in the specified order. tc-config will set a variable INITORDER
   to the content of the boot option and you could add a .core extension that will either override this value or modify
   it when the install script of the .core extension is executed. This way you have the possibility to control the order
   via boot code, an extension or a combination.

[danielibarnes]

How to start a service at the end of the boot procedure?

I had this problem recently. I wanted an extension to be able to modify /etc/inittab and have init reload it in the extension init script. Because tc-config is run by init, you cannot do this within an extension. My solution was to make an extension (reinit.tce) that had only one file, /usr/local/tce.installed/reinit, listed below:

cat <<EOF >> /opt/bootlocal.sh
sed /tty.::/s/^#//' -i /etc/inittab
wait \$PPID
sleep 1
kill -SIGHUP 1
EOF

The are other ways to do it, but this worked great for me. The key here is "wait $PPID" (must be escaped in cat <<). This waits until tc-config has finished before continuing, and it doesn't harm the existing bootlocal.sh (at least in my case). I didn't just add the text to my bootlocal.sh and make a backup because I didn't want a backup file.

Adding complexity to TC (more boot codes) is likely not the answer. It is possible to achieve the desired effect within the current framework, so an easier way of doing what is already possible is likely the best solution. /etc/rc.d is not really an answer either, because those entries are for services which persist and thus have a logical ordering. Extensions are just applications, which by design are transient, and so dependency analysis becomes complicated.

Daniel

[bmarkus]

Hi,

agree that bootcodes are not the solution as it must be typed by the user, while system much startup automatically without bothering the user. In this early phase most of TC users are somehow geeks, but as system will be grown up will see more ordinary users.

Also the question is not how X or Y can solve it on his own, but to offer a system wise mechanism where components from different authors can be put together without manual fine tuning and adjustement.

[roberts]

The startup scripts do run at the end of the booting process, except for .core. elements, and before anything in /opt/bootlocal.sh. However, currently in alpha order regardless if tce or tcz.

Curaga, is right when he mentioned adding a prefix of zz, and that is how I had originally designed it years ago,  B4TC. Back then my offering was more turnkey and a complete desktop. Here much less so. Thus modifying extension names is not very desireable, even though workable.

I don't really want an additional boot code, or having the user made aware of and/or maintaing an 'order list'

The current system uses /usr/local/tce.installed for a dual purpose. To indicate which extensions are installed and if executable, the startup script to execute during boot in alpha order.

What I can offer is an init style prefix, e.g., 00-99 and such scripts be placed in /usr/local/tce.init/ directory. This would then execute the scripts by prefix category (00 - 99) as many Linux/Unix systems typically have in /etc/rc.d directories. Note, busybox does not have runlevels, and using /usr/local/tce.init is also consistent with other tce system components and is PPI (persistent /usr/local/) friendly.

Still with TC/MC being only a core, means much additionally functionality is community contributed. Therefore assiging an appropriate prefix will take coordination between the extension contributors. Perhaps a guideline can be established. And once implemented, extensions requiring this will need to make mention in their .info files specifically which version of TC/MC is required, e.g., version 2.3 or later.

For the hundreds of existing extensions those that currently have a startup script, I can programmatically move them to the new init directory with a common prefix of 55 during the boot process. Those that need adjusting can be moved to earlier or later by adjusting their prefix and resubmitting to the repository.

[helander]

I agree as well that bootcodes are far from ideal.

Another approach would be to factor out todays tc-config into a set of extensions. The code left in tc-config would then only
deal with finding and mounting the tce directory. tc-functions or similar may contain functions of general
use during startup, e.g, finding and mounting volumes based on uuid, label, directory, file; that could be used
by the factored out extensions.
In order to get full control, it should somehow be possible to control at least what extension that is loaded and started first.
This could for example be controlled by having a specific name or name pattern of the extension, or have it located in a specific sub-directory of the tce directory. This approach does provide two things in general:

      - Replaces some bootcodes and instead control behavior by what extensions are in the tce directory.
      - You could take full control of everything that happens during startup (for some this could important).

To make things easy and at the same time apply control like maintain ordering is basically a contradiction, because
you somehow need to understand the ordering requirement and then it is the question of how you best could
specify it. In many cases this could probably be "part" of each extension, by something similar to today's .dep files, but in
other cases it could be outside the knowledge of the extension creator what start order scenarios  it needs to be part
of.


/Lars

[danielibarnes]

Is there a need for 00-99 levels? I think it seems there are only three levels needed for extension initialization: first, or "pre-init"; normal; and last, or "post-init." Can anyone think of use cases for more?

Also, each level is just a "best effort." Script order cannot be guaranteed, of course. If two initialization scripts are labelled 99 to run last, then only one of them can actually be last, so the script must be written with this in mind. I like the "post-init" phase because it does not imply order. It only implies that it will execute after normal extensions are loaded and does not try to guarantee it will be the dead-last extension script executed. It might even be advantageous to split a complicated init script into more than one part and optionally include a script for pre- and post-initialization phases in addition to the normal phase.

I like the idea of separating the dual purpose /usr/local/tce.installed/, though. That is kind of simplification I figured was possible.

My personal preference for general use in the current environment is to put any commands with special requirements into a script somewhere, like /opt/bin/ and have the script in /usr/local/tce.installed/ call it and background it. The script will then wait for the conditions it requires become true and then execute similar to the way I "wait $PPID" in my example.

[roberts]

The categories that I mentioned are based loosely on the "runlevels" and  Start/Kill links as was used in the Red Hat to manage the order of init script processings. See http://www.linuxjournal.com/article/4445

There they used 00-60. We don't have runlevels, and therefore don't need to limit to 60. There is no need to make extra links. Just deploy the concept of prefixed number groups.

Of course there could be many scripts within the same category, and they would, as they do now, run by alphabetical order. I was not suggesting to number order each script, just to group by category.

Still if this were to be adopted there would need to be agreement on some meaning to the various categories. Whether these prefix categories are numbers or text or as few as three or as many as 99 makes little difference. It is the concept of category of init scripts within an init style directory. 

The current design has served us well, so would the effort be worth the result?

[bmarkus]

To be pragmatic, I will apply the most simple solution for HAL using the zzz500-hal name to execute it at the end of the chain. In fact, introducing categories or a separate directory wouldn't give too much more benefits, if we can introduce and accept some rules, for example use name prefixes for example

000...999 and zzz000...zzz999

for extensions. If there are initializations depending on others fitting the same category, authors can agree on name, or simply anybody can change the order renaming extension without involving creator.

Not the most beatiful solution but works. Introduction of a more advanced mechanism can be postponed.