Tiny Core Linux
Tiny Core Base => TCB Talk => Topic started by: jur on November 09, 2010, 12:48:58 PM
-
I have the following stuff in .X.d:
in a file called mystuff:
( loadapp volumeicon && volumeicon & ) &
loadapp hwmon-2.6.33.3-tinycore.tcz &
loadapp pci-hotplug-2.6.33.3-tinycore.tcz &
#( loadapp acpid.tcz && sudo /usr/local/etc/init.d/acpid start ) &
# loadapp filesystems-2.6.33.3-tinycore.tcz &
loadapp hicolor-icon-theme.tcz &
loadapp v4l-dvb-2.6.33.3-tinycore.tcz &
loadapp gtk2_prefs &
loadapp zenity &
updates &
amixer sset 'Master' 64 >/dev/null &
amixer sset 'Headphone' 55 >/dev/null &
amixer sset 'Speaker' 55 >/dev/null &
In a file called desktop:
conky &
tint2 &
in a file called iptables:
loadapp iptables
sudo basic-firewall
loadapp script:
#!/bin/sh
[ -z "$1" ] && echo "Usage: loadapp <extension>[.tcz]" && exit 1
TCE_DIR=`cat /opt/.tce_dir`
THIS_APP=$(basename $1 .tcz)
if [ ! -f "$TCE_DIR"/optional/"$THIS_APP".tcz ]; then
popup Downloading "$THIS_APP"... &
tce-load -w "$THIS_APP"
killall popup
fi
if [ ! -f /usr/local/tce.installed/"$THIS_APP" ]; then
tce-load -i"$NOICON" "$TCE_DIR"/optional/"$THIS_APP"
fi
exit 0
My problem is, all the stuff is not being loaded reliably. Especially conky and tint2 sometimes do not start. I start these manually afterwards with no problem. And most of the time it works. I have noticed this behaviour a long time already , from the beginning of introduction of .X.d but because I can't find a reliable way of reproducing it, I have held off reporting.
conky and tint2 used to be loaded in .X.d using loadapp but were not reliable. I have tried improving things by moving them to OnBoot. That hasn't improved things so I moved these two items to their own file, again with no improvement.
-
What a mess.
.X.d is for starting X applications.
Put your extensions in onboot.lst.
.X.d scripts will run every time X is started, not once per system load.
-
I started using iptables last night; now conky and tint2 reliably do not start.
-
Taking heed of gerald_clark's unhelpful remark, I moved mystuff out of .X.d.
Conky and tint2 in combination with iptables do not load.
-
jur: You might find it to be strongly worded what gerald_clark wrote, but my initial thought on seeing your "mystuff" was fairly similar.
First off you are way to liberal with using '&' (i.e. sending a process to the background). There is always the risk of race conditions when doing that to the degree you are (mis-)using it. Your last two paragraphs in your OP are almost a perfect description of how race conditions are observed by a user (i.e. unreliable, hard to reproduce, ...). In my view you let an awful lot of tasks basically started all at once without paying attention to which event depends on what other events as precondition.
So lets start from first principles: Be very clear for each process from where in the startup sequence it should be started. That means '/opt/bootsync.sh' for system wide things that need to be executed (once per startup as 'root') before '/opt/bootlocal.sh' (which runs asychronously). As gerald_clark has already pointed out: '~/.X.d' scripts are run every time the X server starts, so ensure for those that should run only once that this is the case.
I think you should start from scratch and create a script for each task (e.g. to run 'conky') that contains all preconditions in that script (e.g. to download the extension(s), if not already downloaded, then to install, if not already installed, and finally to execute if not already running).
Another "eyesore" in my view is how you use 'killall' in your "loadapp". As all things are for you running basically in parallel the first process that reaches a 'killall' does so for ALL the popup processes (and not just for the one that should be terminated). Here is a code snippet (albeit untested) that should do the "proper" thing: popup Downloading "$THIS_APP"... &
POPUP_PID=$!
tce-load -w "$THIS_APP"
kill $POPUP_PID
Now I personally don't see a real need for having all those "popups" anyway. If you want it for monitoring purposes I believe adding messages to a unique, temporary log-files (e.g. "/tmp/loadapp.$$.log") to be much better.
I believe that you will have to resolve all your race conditions first. Only after that should you focus on any remaining issues. My hunch is that there won't be any issues left (providing you do it right).
-
Boiling it down to what actually happens... loadapp does not download stuff and does not killall popups at boot... all the extensions being loaded are already present so ignore that.
So, what I effectively have is, assuming stuff is done alphabetically:
conky &
tint2 &
tce-load -i iptables
sudo basic-firewall
and that does not seem to work.
(Note that tint2 and conky HAVE to be backgrounded.)
-
I couldn't see how X apps like conky & tint2 versus iptables (which has the firewall modules as a dependency) and a firewall script could be related.
-
Please split things into separate scripts which contain sufficient checks and can be run (and tested) independently:
So (as one example) create a '~/.X.d/conky' with:#!/bin/sh
which conky > /dev/null || ! date '+%T: conky not found" >> /tmp/conky_$$.log || exit 2
conky &
I assume that running conky & from an interactive shell works as expected.
-
.X.d scripts are not preceded by #!/bin/sh
regarding my "liberal" use of '&', I recall that roberts wrote that stuff called in .X.d have to be backgrounded, that's the only reason I did it (except for conky and tint2).
(Why check for the presence of conky? it is loaded OnBoot.)
-
(Why check for the presence of conky? it is loaded OnBoot.)
Because it is good programming practice.
The scripts in .X.d have no knowledge of what is or is not in .onboot.lst
-
Regarding loading my apps in .X.d, I like to be at my desktop as quickly as possible at boot. So what's wrong with loading extensions which are not immediately needed a bit later (ie during .X.d)? bootlocal.sh can't be used. And there is no other startup provision.
-
It has been very clearly (IMHO) stated by gerald_clark and then by maro (quoting gerald_clark) what is wrong.
You might actually lose time on each subsequent start of X, left alone potential unpredictable consequences.
Wondering what your most optimistic estimation of gaining time at boot is, and if it ever could sum up to the time invested of hacking boot sequence (I'd highly doubt it).
-
So, what I effectively have is, assuming stuff is done alphabetically:
I wouldn't assume that. Create one script which does what you want in the order you want to be safe. Use functions to separate code like "loadapp".
conky &
tint2 &
tce-load -i iptables
sudo basic-firewall
and that does not seem to work.
I made a script called ~/.X.d/stuff with exactly the above and it worked every time. Can you confirm this duplicates your procedure? I did not reboot in between attempts, so I simply executed "startx" then exited back to the prompt. There are only a couple of things on which I might comment:
- basic-firewall includes a "read" command. I don't know if reading from the console might interfere with something. You might try sudo basic-firewall < /dev/null
- you only need to start iptables once, but it does no harm to attempt it every time because the basic-firewall script resets to a known state when executed.
-
It has been very clearly (IMHO) stated by gerald_clark and then by maro (quoting gerald_clark) what is wrong.
You might actually lose time on each subsequent start of X, left alone potential unpredictable consequences.
Bull.
1. gerald_clark stated the obvious. Wasn't helpful at all.
2. maro didn't quote gerald_clark.
3. Backgrounding was stated by roberts as being necessary for commands in .X.d - I was merely doing what he said should be done.
The only reason I started this thread and what you are losing sight of, is that conky and tint2 do not start reliable. These are both X apps. Both are started according to their recommended use, and in .X.d which it is meant for.
I found that these 2 are not reliable, and stated this here, and produced a set of commands that reproduce this condition. The problem is not iptables nor is it the so-called mess that gerald_clark so unkindly referred to. The problem is conky and tint2.
And I do not lose time on each subsequent start of X. My loadapp script checks before doing stuff.
-
Cannot reproduce. Works as expected. The order of launching the two mentioned X programs, conky and tint2, had no bearing. The order by default is not alpha and should not matter.
Tested three ways:
1. the order as per the find clause (see find .X.d/ -type f).
2. wedging in a | sort to ensure an alpha ordering (see find .X.d/ -type f | sort ).
3. wedging in a | sort -r to ensure a reverse alpha sort (see find .X.d/ -type f | sort -r).
No failures were observed both when exiting X and startx as well as rebooting.
These tests were conducted with the recommended and supported methods of using Tiny Core.
That is not trying to background load multiple extensions. Much effort has been put forth to ensure the proper loading of extension. Using .xsession is not recommended. The backgrounding to launch loaded X programs within xsession is.
.X.d/tint2
tint2 &
.X.d/conky
conky &
If further information can be supplied, sans any custom scripts or background extension loading within .xsession/.X.d, that can document such a claim, I will gladly look further.
-
OK, thanks for checking.
What I will do is to pare the stuff down to the barest bones that will reproduce the result. That will take a day or so.
-
jur, most likely danielibarnes described the solution already. "sudo basic-firewall" ends with a "Press enter" prompt (that you cannot see in your scenario). Any script that comes afterwards -- will not be executed.
To prevent the sequence of scripts from getting stuck, you may wish to try danielibarnes' suggestion: "sudo basic-firewall < /dev/null".
-
Hm, I really should add a "noprompt" option for the non-interactive starts of basic-firewall.
edit: Done.
-
if I put
conky &
tint2 & and tce-load -i iptables
sudo basic-firewall in two separate files named desktop and iptables respectively, then conky and tint2 don't load.
If I add '&' to sudo basic-firewall then they do load.
If I put all 4 statements in a single file, then they load. It works better to put conky and tint2 before iptables, but still both ways work with and without adding & to basic-firewall.
Above tried with the modded iptables.
[edit]If I add < /dev/null instead of & to basic-firewall it also works, as does noprompt.
-
@roberts:
Is it required to background tasks in .X.d by appending '&' to statements?
-
You do not need to background them if they background themselves,
or if you need them to exit normally before continuing to the next step.
However, If any of these scripts do not detach, or pause waiting for input,
.xsession will not finish and you will be confused because something is not going
to be running correctly.
If you have a complex startup, you can enclose the script in parenthesis and background it.
EX:
(
step 1
step 2
step 3
) &
This will run the script in background, but ensure that step 3 does not run until steps 1 and 2 have
completed.
-
@roberts:
Is it required to background tasks in .X.d by appending '&' to statements?
gerald_clark is correct here. But typically most X programs do.
[You do not need to background them if they background themselves,
or if you need them to exit normally before continuing to the next step.
The running of non-X within .xsession .X/.d seems to be overly complicating matters.
It is like let's do away with SOP of bootlocal.sh, bootsync.sh, .profile, e.g., the typical places to start non-X programs. In fact the .info file for iptables correctly suggests to use:
From bootlocal.sh (to start on every boot):
/usr/local/sbin/basic-firewall noprompt
-
I may seem complicated but it's driven by the decision not to load extensions at boot time. I like to have the desktop up and running ASAP. So my OnBoot list only contains those extensions which are essential to get the desktop up and running, such as Xorg, the wm and so on. Those other extensions are not OnDemand either, it's just not necessary to load them immediately.
So how would you load extensions later? There is no provision - I like to keep .profile and .xsession default, because previous times when I did actually insert stuff in there, it complicated matters when switching to new releases when from time to time there are changes in those files. bootlocal.sh runs as root so is not suitable either.
So it seems to me there needs to be a startup file which is processed as user in which one can place various startup commands and scripts.
Or am I missing something?
-
So how would you load extensions later? There is no provision
On-demand does just that.
-
Perhaps I didn't make myself clear enough... extension I have in mind are eg hwmon-2.6.33.3-tinycore. I need that for normal running but I don't need it to get the desktop running. There are quite few of these. Clearly they are not ondemand extensions.
-
jur: I really think you are "missing something" (which has come up in this forum multiple times). The simple way for 'root' to run something as user 'tc' is via su -c COMMAND tc
As a proof of concept I've done:
tc@box:~$ echo 'su -c "tce-load -w conky && tce-load -i conky" tc' >> /opt/bootlocal.sh
tc@box:~$ echo 'su -c "tce-load -w tint2 && tce-load -i tint2" tc' >> /opt/bootlocal.sh
tc@box:~$ echo '( while [ ! $( which conky 2> /dev/null ) ] ; do sleep 1 ; done ; conky ) &' > .X.d/conky
tc@box:~$ echo '( while [ ! $( which tint2 2> /dev/null ) ] ; do sleep 1 ; done ; tint2 ) &' > .X.d/tint2
tc@box:~$ filetool.sh -b && sudo reboot
I believe that it is what you want: It starts the X server as early as possible and 'conky' and 'tint2' are showing up whenever they are available. I personally believe it's better to create in '~/.X.d' a file for each application, but I've tried to come up with a command sequence that should also work if you "squeeze" everthing into a single file.
As a side note: The test was conducted using a VM (i.e. QEMU without KQEMU). It was rather slow for 'the applications to start and I'm putting this down to being forced into loading 'wireless_tools.tcz' and the wireless kernel modules on a (virtual) system that will never have any such hardware.
-
[ -d ".X.d" ] && find ".X.d" -type f -print | while read F; do . "$F"; done
The last line of .xsession runs the scripts and programs in .X.d as shown above. The most universal solution would be a minor rewrite of the line, such that it starts all scripts and programs in the background. Which, obviously, is a suggestion for roberts, not for jur.
-
That would prohibit not backgrounding where a race condition may exist.
-
The present solution with its unpredictable order of script execution doesn't prevent race conditions either -- so we are no worse off with my suggestion.
A few posts further up in this thread someone showed the proper way of preventing race conditions. Oh, that was you ...
-
OK, I have taken all the good advice in this thread on board and come up with the following. Please feel free to point out any problems, as I am really still in early learning phase:
.X.d has a single file with
conky &
tint2 &
,profile has an extra inclusion
if [ -f "$HOME/.startup" ]; then
. "$HOME/.startup"
fi
.startup:
export GDK_NATIVE_WINDOWS=true
(
sleep 5
loadapp iptables && sudo basic-firewall noprompt
loadapp volumeicon && volumeicon &
loadapp iptables && sudo basic-firewall noprompt
loadapp hwmon-2.6.33.3-tinycore.tcz
loadapp pci-hotplug-2.6.33.3-tinycore.tcz
loadapp hicolor-icon-theme.tcz
loadapp gtk2_prefs
loadapp zenity
amixer sset 'Master' 64 >/dev/null
amixer sset 'Headphone' 55 >/dev/null
amixer sset 'Speaker' 55 >/dev/null
loadapp wine
updates
) &
The reason I am sticking with my loadapp script is that it contains all checking; I can start a new setup from scratch, all necessary extensions will be auto-downloaded.
-
iptables twice?
-
iptables twice?
Oops, thanks!
-
One more alternative: the waitForX binary waits for X to start. You could include this line (or something similar) in bootlocal.sh to wait for X to start before loading your extensions:
(/usr/bin/waitforX; [ -f /home/tc/.startup ] && home/tc/.startup) &
-
No, you cannot wait for X in bootlocal.sh, as this runs before the user is logged in.
-
Well, see, that's what I get for posting without actually trying it first. I typically include the caveat, "I haven't tried this ..." but I was lazy. It does work, however, if you change waitforX to: su -c "/usr/bin/waitforX" tc. :)
I'd not seen anyone examine methods for starting X quicker, though, so I found this topic interesting.