Tiny Core Linux
Tiny Core Base => TCB Bugs => Topic started by: GNUser on December 18, 2020, 06:43:58 PM
-
I was surprised today when I put a link to a shell script in ~/.X.d/ and there was no effect (i.e., the script did not run at boot).
It turns out the problem is this line in /etc/skel/.xsession (part of the Xlibs.tcz extension):
[ -d "$HOME/.X.d" ] && find "$HOME/.X.d" -type f | while read F; do . "$F"; done
The fix is to change the line to this:
[ -d "$HOME/.X.d" ] && find "$HOME/.X.d" -type f -o -type l | while read F; do . "$F"; done
I think the change adds some small value to the extension, so I'll submit my tweaked version for the repo.
-
Hi GNUser
~/.X.d/ is supposed to be used to launch processes after X has started, not execute scripts directly line by line. The
correct way to do this is create a file in ~/.X.d/ that contains:
MyScript &This way when your script is launched it can't block other processes from starting.
-
Thanks, Rich. I know. All my shell scripts are in a single directory (/opt/scripts), including ones called "jobs-startup" and "jobs-suspend". "jobs-startup" does indeed put most things in the background in order not to hold up the boot process.
I really appreciate TCL's "no scatter" philosophy and would rather keep all my scripts in one place, with symlinks everywhere else. Using a symlink in this specific case also makes my life easier because my /opt is persistent but /home is not (i.e., any time I change ~/.X.d/ I need to create a new backup).
The workaround I had been using was to create ~/.X.d/jobs-startup containing only "exec /opt/scripts/jobs-startup", but a symlink seems more elegant.
Anyway, I thought it wouldn't hurt for ~/.X.d/ to be more flexible and allow symlinks. If tweaked extension seems pointless or harmful, no need to post it. Sorry for the noise.
-
I think the change is fine.
-
Hi, Core people!
If You go tweaking .xsession I want to propose a small tweak too
[ -d "$HOME/.X.d" ] && find "$HOME/.X.d" -type f -o -type l | sort | while read F; do . "$F"; done
to make scripts execution order fully predictable.
Regards!
-
Hi, jazzbiker. Wow, that's a great idea, a simple way of making .xsession even more better :D I re-submitted tweaked Xlibs.tcz to include your idea.
-
Hi, GNUser!
I'm glad You enjoyed this tweak, I expect it rather natural and this is the very first patch I'm applying to every fresh install of TinyCore making use of X. But I've proposed it earlier in http://forum.tinycorelinux.net/index.php/topic,2588.0.html thread. Maybe it will be better to wait for Curaga's or Juanito's approval?
Hope You applied this sort to ~/.X.d only, not to /usr/local/etc/X.d?
-
Hi, jazzbiker. I added the sort command to both user and etc lines for consistency. Juanito can easily remove it from the etc line if he doesn't like it, or I'll be happy to revise the extension and resubmit it.
-
Oh, in the above mentioned thread I gave my considerations on /usr/local/etc/X.d scripts sorting. Look, "find" command lists files in the reverse creation order. So /usr/local/etc/X.d scripts wiil be executed according to the reversed extensions' order in Your onboot.lst, and if You need, You can adjust the sequence of scripts' execution swapping the extensions in Your onboot.lst. But "sort" in .xsession applied to /usr/local/etc/X.d will eliminate this possibility. Sorry for disturbing You, but I will kindly ask You to resubmit the extension without the "sort" in the /usr/local/etc/X.d chain, of course if You agree that the above mentioned effect can be harmful.
-
Haha, okay. I submitted three versions of tweaked Xlibs.tcz:
1. support for links in ~/.X.d
2. support for links in ~/.X.d, sort in $HOME/.X.d line, sort in /usr/local/etc/X.d line
3. support for links in ~/.X.d, sort in $HOME/.X.d line (no change to /usr/local/etc/X.d line)
Third option is most likely to add value without introducing any breakage.
-
Thanks!
Sorry, I'd underestimated Your submitting pace :-) I expected we are starting discussion, while You've already submitted the target twice :-) Hope my intervention in the process will not be the waste of time :-)
-
I'd underestimated Your submitting pace :-) I expected we are starting discussion, while You've already submitted the target twice :-)
Nobody has ever been able to accuse me of being the bottleneck in a team effort ;)
I wanted to make it as easy as possible for the developers to update the extension. Now they have three versions and can choose the one they think will work best for everybody. Hopefully we can have all the features we want with no breakage for anyone.
-
Consistent and predictable behavior is the key, and should be the goal. If we're going to sort (which would give us predictability), then sort both /usr/local/etc/X.d (which theoretically system should come before personal) and $HOME/.X.d second for consistency. ~/.X.d is usually the same as $HOME/.X.d, but not necessarily, so to avoid confusion should not be used. Otherwise you would need to determine if they are the same or not to prevent scripts from being executed twice.
-
Hi, andyj!
Predictability needs to be accompanied with controllability. The user can change the ~/.X.d/* names and achieve desirable execution order, using sort. But /usr/local/etc/X.d/* are coming from extensions, their names are defined by extensions' maintainers, and their execution order can be controlled by changing extensions loading order in onboot.lst, though invoking them through sorting will leave us with predictability ohne controllability.
-
Actually the files in /usr/local/etc/X.d won't be executed until .xsession is run when X starts which is after all extensions are loaded, not as the extensions are loaded so extension load order will not have any effect. It is also worth noting that an extension that has a file in /usr/local/etc/X.d will not be executed if it is loaded after X has started. The execution order in /usr/local/etc/X.d could be controlled by using names that start with numbers 00 through 99 like udev rules, but that is really only necessary if there is some kind of dependency between extensions. In that case it should fall on the extension maintainers to coordinate the order of their scripts through naming. How many scripts in /usr/local/etc/X.d are we talking about anyway? I know of one, because I started this way back when: http://forum.tinycorelinux.net/index.php/topic,22312.0.html (http://forum.tinycorelinux.net/index.php/topic,22312.0.html), so this apparently was not an issue until then.
-
Hi, andyj!
I didn't knew, that this /usr/local/etc/X.d chain in .xsession is Your contribution :-) Great thing, surely! It was added not long ago and that is why it is not widely used by maintainers, I think it needs to be promoted, but I don't know how - wiki is down :-( I must say, that some topics were dedicated exactly to troubles, which could be cured using this per-extension X startup scripts. One of the examples is terminus-fonts.tcz, I use it every day and it needs manually created script in ~/.X.d, however this manual action could be avoided by adding corresponding /usr/local/etc/X.d script to the extension. If I am not mistaken, /usr/local/etc/X.d is the right place for all the fonts caching calls. Probably there are another common cases for this directory use.
Of course I realize, that /usr/local/etc/X.d scripts will be executed only at X startup. I was mentioning that their invocation order (at X startup) will be the reverse load order in onboot.lst, because their links are created inside /usr/local/etc/X.d during tce-load and consequent "find ... " (in .xsession) will list them in reverse creation order. Maybe it is filesystem dependent behaviour, I'm using ext4 and didn't tested the others.
Regards!
-
In that case it should fall on the extension maintainers to coordinate the order of their scripts through naming.
Sorry, but I think it will seed the problems in the future. And if maintainers will use this fine possibility (per-extension X starup scripts) wider (I expect this the right way) all the possible future inconsistencies are inpredictable (projects are being updated and upgraded) and the user must have the ability to eliminate them with his own tools, I believe. Of course this is only an opinion.
Thanks!
-
If we "ls" /usr/local/etc/X.d we get one order of files, if we "find" we get another. One order is observable, the other is theoretically knowable but not without research. I was a little short sighted when I submitted the patch for /usr/local/etc/X.d in that I did not sort the results as my one file did not need sorting. Now I can see that if it was sorted using "sort" then when someone looked at a directory listing (using default sort) for /usr/local/etc/X.d they would see the files in execution order and there would be a lot less confusion and questions. Load order would be okay and would make sense if it was the order displayed by "ls", but one has to know to use a different command "find" to see that. I contend that showing one thing and doing something else will be perceived as wrong, as it apparently it is now by some. It should be intuitive, and I think that numbering files and sorting is how we can do that. Sadly, there just are not enough maintainers that this might be a problem. The fact that there are relatively few makes coordination a lot simpler than it might otherwise seem.
If this is really an issue, then try something like what I use in /opt/bootlocal.sh for files in /usr/local/etc/init.d instead:
for a in web-firewall openssh open-vm-tools php-fpm nginx; do
[ -x /usr/local/etc/init.d/$a ] && /usr/local/etc/init.d/$a start &
done
It should be easy enough to adapt this approach for /usr/local/etc/X.d as well.
-
One order is observable, the other is theoretically knowable but not without research.
Can the next be named "research" :-)
tc@box:~$ mkdir tmp; cd tmp
tc@box:~/tmp$ touch 1; touch 7; touch 3
tc@box:~/tmp$ find . -type f
./3
./7
./1
tc@box
?
I'm claiming, that Your patch is excellent, but You don't agree :-) Let's don't repair what's not broken!
-
Hi jazzbiker
Are you using busybox find or GNU find?
Do they both behave the same way?
tc@E310:~$ mkdir count
tc@E310:~$ cd count/
tc@E310:~/count$ touch 1; touch 7; touch 3
tc@E310:~/count$ find . -type f
./7
./3
./1
tc@E310:~/count$ which find
/usr/bin/find
tc@E310:~/count$ ls -l /usr/bin/find
lrwxrwxrwx 1 root root 17 Jun 9 2019 /usr/bin/find -> ../../bin/busybox
tc@E310:~/count$ tce-load -i findutils
findutils.tcz: OK
tc@E310:~/count$ which find
/usr/local/bin/find
tc@E310:~/count$ ls -l /usr/local/bin/find
lrwxrwxrwx 1 root root 40 Dec 20 11:58 /usr/local/bin/find -> /tmp/tcloop/findutils/usr/local/bin/find
tc@E310:~/count$ find . -type f
./7
./3
./1
tc@E310:~/count$
On my machine (TC10 32 bit), busybox find and GNU find behave the same way. However, they produce a different
result than your machine.
-
Hi, Rich!
This moment I use TC10 x86.
tc@box:~/count$ version
10.1
tc@box:~/count$ uname -a
Linux box 4.19.10-tinycore #1999 SMP Tue Dec 18 13:36:47 UTC 2018 i686 GNU/Linux
tc@box:~/count$ busybox | grep v1
BusyBox v1.29.3 (2020-07-04 15:39:28 EEDT) multi-call binary.
tc@box:~/count$ ls -l $(which find)
lrwxrwxrwx 1 root root 17 Jul 4 15:55 /usr/bin/find -> ../../bin/busybox
tc@box:~/count$
Thanks for checking. Can I please ask You to provide one more test?
tc@box:~$ mkdir count
tc@box:~$ cd count
tc@box:~/count$ touch 1; sleep 1; touch 7; sleep 1; touch 3
tc@box:~/count$ find . -type f
./3
./7
./1
P.S. In my opinion this aspect of "find", "ls" ... behaviour depends on libc.
-
Hi jazzbiker
tc@E310:~$ mkdir count
tc@E310:~$ cd count
tc@E310:~/count$ touch 1; sleep 1; touch 7; sleep 1; touch 3
tc@E310:~/count$ find . -type f
./7
./3
./1
tc@E310:~/count$ busybox find . -type f
./7
./3
./1
tc@E310:~/count$
tc@E310:~/count$ busybox | grep v1
BusyBox v1.29.3 (2018-12-19 15:29:37 UTC) multi-call binary.
tc@E310:~/count$ version
10.1
tc@E310:~/count$
My busybox has the same version number but a different date code.
-
Thanks!
But now I can not understand, what's the difference between our systems and the "find" behaviour. busybox build date is different because I was patching vi (if I'm not mistaken).
Does this looks like for Your system?
tc@box:~/count$ ls
1 3 7
tc@box:~/count$ ls -u
3 7 1
I use default TC ~ residing in memory. U2?
-
Hi jazzbiker
... Does this looks like for Your system? ...
Yes, I get the same results for both the busybox and the GNU versions of ls.
... I use default TC ~ residing in memory. U2?
No, I run persistent /home and /opt on all of my systems.
tc@E310:~/count$ cd /tmp/
tc@E310:/tmp$ mkdir count
tc@E310:/tmp$ cd count/
tc@E310:/tmp/count$ touch 1; sleep 1; touch 7; sleep 1; touch 3
tc@E310:/tmp/count$ find . -type f
./3
./7
./1
tc@E310:/tmp/count$ busybox find . -type f
./3
./7
./1
tc@E310:/tmp/count$
It seems the RAM file system behaves differently from one on a hard drive.
-
It looks like everyone is making my argument for me 8) Maybe we should use sort and get consistent results for all?
-
@Rich
So, how sad this appears to be, find's behaviour is fs-type dependent and can not be relied on :-( Looks like its output is already sorted by name on hd and unsorted for RAM. Thanks for testing. If You tracked this topic, what's Your opinion about these sort's in .xsession? Probably andyj is using persistent ~ too.
-
It looks like everyone is making my argument for me 8) Maybe we should use sort and get consistent results for all?
Yes, it looks like ;-( persistent home beats memory.
-
GNUser, sorry for disturbing and making so many noise.
-
Probably andyj is using persistent ~ too.
No persistence for any directory in /:
tc@box:~$ cd /mnt/sda1
tc@box:/mnt/sda1$ la
total 52
drwxr-xr-x 11 root root 4096 Dec 31 2019 ./
drwxrwxr-x 6 root staff 120 Dec 20 23:00 ../
drwxr-xr-x 3 tc staff 4096 May 1 2020 boot/
drwxr-xr-x 12 tc staff 4096 Sep 3 12:03 lamp/
drwxr-xr-x 13 tc staff 4096 Nov 7 15:08 lamp32/
drwsrwxrwx 13 tc staff 4096 Nov 6 19:30 lamp64/
drwx------ 2 root root 16384 Dec 21 2018 lost+found/
drwxrwxr-x 3 tc staff 4096 Feb 14 2020 tce32-10/
drwxrwxr-x 4 tc staff 4096 May 27 2020 tce32-11/
drwxrwxr-x 3 tc staff 4096 Feb 14 2020 tce64-10/
drwxrwxr-x 4 tc staff 4096 Nov 27 04:39 tce64-11/
tc@box:/mnt/sda1$ df
Filesystem Size Used Available Use% Mounted on
rootfs 7.0G 226.1M 6.8G 3% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/sda1 31.1G 22.8G 8.3G 73% /mnt/sda1
vmhgfs-fuse 1.8T 1.6T 182.7G 90% /mnt/hgfs/andy-home
vmhgfs-fuse 2.7T 1.8T 929.9G 66% /mnt/hgfs/bax2
tc@box:/mnt/sda1$
Just a lot of boot options for maintaining extensions across various releases.
-
I've made some testing on USB ext4 drive and must say that I can not describe how "find" is sorting its output. It acts independent on name, date, inode and so on. Without digging into sources nothing can be said. So andyj's proposal to sort the find's output is the way to bring predictability, otherwise the same extensions' set will behave differently in the different environments, which is very undesirable, so I vote for sorting both find's in .xsession.
P.S. Of course /usr/local/etc/X.d resides in memory unconditionally for TC, but seems that relying on unprocessed find's output is unacceptable.
-
Hi jazzbiker
@Rich
... what's Your opinion about these sort's in .xsession? Probably andyj is using persistent ~ too.
I'm a big fan of consistent behavior, so my opinion is include the sort command.
-
Using ls for creation time based sorting:
[ -d "/usr/local/etc/X.d" ] && ls -tc $(find "/usr/local/etc/X.d" -type f -o -type l) | while read F; do echo "$F"; done
-
GNUser, sorry for disturbing and making so many noise.
It's an interesting discussion, not noise, so nothing to be sorry about :)
My two cents on the thread so far:
1. I love how conservative TCL is, and how much care goes into making sure that changes cause no breakage. This is yet another of TCL's many outstanding qualities.
2. I agree that predictability and controllability are both important. It seems that having "sort" in the /usr/local/etc/X.d line would add predictability and, for extensions where it is important, would give the extension maintainer controllability. TCL11 x86_64 repo currently has only one extension (open-vm-tools-desktop.tcz) that puts something in /usr/local/etc/X.d, so it would not be difficult for extension maintainers to coordinate in the unlikely event that execution order matters.
-
Hi jazzbiker
Changing file names to control order of execution is more practical and intuitive than changing creation times.
-
I haven't updated Xlibs so far as it looks like we didn't come to an agreement on changes to the start-up script?
-
Hi, Juanito!
I feel guilty, that my attempt to propose sorting improvement leads to GNUser's proposal refusal. In my opinion we all agreed that adding "sort" to both ~/X.d and /usr/local/etc/X.d chains will provide predictability and controlability - this is GNUser's second submitted extension. If You expect this controversial, please post the first GNUser's submission with links added to ~/.X.d chain and without any sortings.
-
My impression was that we had reached agreement. To summarize:
1. Supporting links in $HOME/.X.d is an add-on feature that does not break anything; curaga gave the green light and there were no objections.
2. Adding sort to the $HOME/.X.d line would ensure predictable execution order of user's jobs; any breakage that arises from the change (going from unpredictable to predictable execution order) would be easy to fix by user renaming jobs so that names reflect desired execution order.
3. Adding sort to the /usr/local/etc/X.d line shouldn't break anything because it seems only rare extensions (I can only find open-vm-tools-desktop.tcz) use that directory. It seems advantageous to have sort in there going forward, to ensure predictability if/when more extensions use that directory.
Objectors and supporters, please speak up! If no objections are voiced, perhaps it would be reasonable to assume agreement and move forward with the changes.
P.S. My second Xlibs.tcz submission is the one that contains all of the proposed changes.
-
To avoid confusion, please post the startup script here.
-
Hi GNUser
My impression was that we had reached agreement. To summarize: ...
1. Agreed.
2. Agreed. It also removes the unpredictability between RAM and hard drive behavior mentioned here:
http://forum.tinycorelinux.net/index.php/topic,24585.msg156066.html#msg156066
3. Agreed. Since andyj approves, that's good enough for me.
-
Hi, Juanito. Here is /etc/skel/.xsession (part of Xlibs.tcz) with all three proposed changes:
Xvesa -br -screen 1024x768x32 -shadow -2button -mouse /dev/input/mice,5 -nolisten tcp -I >/dev/null 2>&1 &
export XPID=$!
waitforX || ! echo failed in waitforX || exit
"$DESKTOP" 2>/tmp/wm_errors &
export WM_PID=$!
[ -x $HOME/.setbackground ] && $HOME/.setbackground
[ -x $HOME/.mouse_config ] && $HOME/.mouse_config &
[ $(which "$ICONS".sh) ] && ${ICONS}.sh &
[ -d "/usr/local/etc/X.d" ] && find "/usr/local/etc/X.d" -type f -o -type l | sort | while read F; do . "$F"; done
[ -d "$HOME/.X.d" ] && find "$HOME/.X.d" -type f -o -type l | sort | while read F; do . "$F"; done
-
3. Agreed. Since andyj approves, that's good enough for me.
8)
-
To close any doubts in the full consensus, my answers to GNUser's questions put in the #36 post are:
1. Yes (links in ~/.X.d)
2. Yes (find $HOME/.X.d | sort)
3. Yes (find /usr/local/etc/X.d | sort)
-
Xlibs updated in 11.x x86/x86_64 repos and 12.x armv6/armv7/aarch64 repos
-
Hurray!
Thank you all for the interesting discussion and Juanito for posting :)