Tiny Core Linux
Tiny Core Extensions => TCE Talk => Topic started by: GNUser on February 03, 2025, 12:37:24 PM
-
For me, one of the very few pain points (possibly the only one) in using TCL is the fact that some extensions use /etc for config files while other use /usr/local/etc.
The pain arises when I'm trying out an extension for the first time: It takes some investigating and/or trial and error to figure out which etc directory to use.
I understand and agree with reasoning behind having /usr/local/etc but the reality is that many extensions do not use it--either because the extension contributor was unaware of our convention or because the build process for some applications makes it difficult/impossible to specify the configuration directory. The problem is not that the /usr/local/etc convention exists--the problem is that not all extensions follow the convention.
I'm not suggesting that the convention of using /usr/local/etc for extensions be abolished. But I would like to explore whether it's possible for a user who would rather have a single etc directory (e.g., me) to tweak his TCL system so as to achieve this.
I'm on TCL15 x86_64 and already tried remastering corepure64.gz by adding a symlink at /usr/local/etc that points to /etc. Booting with the remastered corepure64.gz did not go well.
Is this a fool's errand or is there an elegant way to achieve a working system where /usr/local/etc simply points to /etc? Many thanks for any insights/advice you can give.
-
For me, one of the very few pain points (possibly the only one) in using TCL is the fact that some extensions use /etc for config files while other use /usr/local/etc.
The pain arises when I'm trying out an extension for the first time: It takes some investigating and/or trial and error to figure out which etc directory to use.
I understand and agree with reasoning behind having /usr/local/etc but the reality is that many extensions do not use it--either because the extension contributor was unaware of our convention or because the build process for some applications makes it difficult/impossible to specify the configuration directory. The problem is not that the /usr/local/etc convention exists--the problem is that not all extensions follow the convention.
I'm not suggesting that the convention of using /usr/local/etc for extensions be abolished. But I would like to explore whether it's possible for a user (me) to tweak his TCL system so that everything goes into /etc.
I'm on TCL15 x86_64 and already tried remastering corepure64.gz by adding a symlink at /usr/local/etc that points to /etc. Booting with the remastered corepure64.gz did not go well.
Is this a fool's errand or is there an elegant way to achieve a working system where /usr/local/etc simply points to /etc? Many thanks for any insights/advice you can give.
My best suggestion would have been exactly what you already tried - symlink /usr/local/etc to point to /etc - although I would have, perhaps naively, tried a lazy way first without remastering. Remastering is clearly a better solution.
What were the details of "didn't go well"?
-
What were the details of "didn't go well"?
Hi Leee. Boot process completes and extensions are loaded, but instead of seeing my fluxbox desktop with custom desktop background I briefly see TCL's default desktop background and then screen turns black and I can't see the desktop. Hard to say exactly what's broken but the "fix" is to reboot from the console into my recovery partition and revert the remastered rootfs to stock rootfs.
So my conclusion is that the /usr/local/etc to /etc symlink in the rootfs is breaking one or more extensions.
-
What were the details of "didn't go well"?
Hi Leee. Boot process completes and extensions are loaded, but instead of seeing my fluxbox desktop with custom desktop background I briefly see TCL's default desktop background and then screen turns black and I can't see the desktop. Hard to say exactly what's broken but the "fix" is to reboot from the console into my recovery partition and revert the remastered rootfs to stock rootfs.
So my conclusion is that the /usr/local/etc to /etc symlink in the rootfs is breaking one or more extensions.
I haven't tried making any substantial mods to corepure64.gz (I'm just in the early stages of playing around with remastering) so sorry if this obvious or way out in left field but, even that early in the boot process... did you copy everything from /usr/local/etc/ into /etc/ before making the symlink?
Also, for simplicity in testing, I think you can create an additional .gz file (maybe corepure64_2.gz), containing only your modified files, and load it after the original and files in it will replace the original files from the original .gz. Then turning your mods on or off is just a matter of loading or not loading the extra .gz file at boot time. (Dead simple using grub but it's been ages since I used any other bootloader.)
FWIW, for me, the pain of having both /etc/ and /usr/local/etc/ is so minuscule that I would have never tried this particular mod. Far worse, for me, is the pain of trying to get the line spacing the way I want it here in the forum. :)
-
Hi Leee. Yes, using multiple initramfs archives and turning them on/off via bootloader is a nice trick.
I think I'm just going to give up on this mod. A symlink in the rootfs seems like the most promising approach but it's definitely breaking extension(s). I think having to troubleshoot the breakage is more painful than just living with the two etc directories.
Happy hacking!
-
Hi GNUser
Hi Leee. Boot process completes and extensions are loaded, but instead of seeing my fluxbox desktop with custom desktop background I briefly see TCL's default desktop background and then screen turns black and I can't see the desktop. Hard to say exactly what's broken ...
You could try saving some information to persistent storage (tce/).
Add syslog and loglevel=7 to your boot codes.
Create this script:
#!/bin/sh
# Allow time for the desktop to finish crashing.
sleep 10
# You may have to specify /mnt/sda... here if the path can't
# be retrieved from /etc.
TCE="Path/to/tce/directory"
cp /var/log/messages* "$TCE"
dmesg > "$TCE/dmesg.txt"
Then launch that script from the end of bootlocal.sh
My guess is this command that gets executed after mounting
an extension might be the issue:
COMMAND=/bin/cp -ais /tmp/tcloop/mirrors/usr /
The busybox cp looks like this:
tc@E310:~$ busybox cp --help
BusyBox v1.29.3 (2018-12-19 15:29:37 UTC) multi-call binary.
Usage: cp [OPTIONS] SOURCE... DEST
Copy SOURCE(s) to DEST
-a Same as -dpR
-R,-r Recurse
-d,-P Preserve symlinks (default if -R) < ### This probably stops it from overwriting your link.
-L Follow all symlinks < ### Adding this might make it work.
-H Follow symlinks on command line
-p Preserve file attributes if possible
-f Overwrite
-i Prompt before overwrite
-l,-s Create (sym)links
-T Treat DEST as a normal file
-u Copy only newer files
-
Hi Rich. Yes, the problem has something to do with how extensions are loaded.
When booting with the modded rootfs, /etc would be expected to contain all the files and directories normally found in /etc *plus* those normally found in /usr/local/etc. Actual result is that files and directories normally found in /usr/local/etc are totally absent from the system.
I will explore the cp command flags in tce-load and see if they are the reason for my mod not working as expected. Thanks for the tip.
-
Hi Rich. Changing this line in /usr/bin/tce-load:
yes "$FORCE" | sudo /bin/cp -ais /tmp/tcloop/"$APPNAME"/* / 2>/dev/nullto this:
yes "$FORCE" | sudo /bin/cp -aisL /tmp/tcloop/"$APPNAME"/* / 2>/dev/nulldid not help.
The problem is this: If an extension containing /path/to/parentdir/somefile is loaded on a system where parentdir is a symlink, /tmp/tcloop/<extension_name>/path/to/parentdir/somefile appears on the system as usual but /path/to/parentdir/somefile never appears on the system.
PS: Rich, I'll send you a PM with the logs you requested. There's too much data in there for this privacy-paranoid user to post here. If we find a solution I'll make sure to post it here.
PS2: Did we find a bug in tce-load? It seems so: When an extension is loaded, user expects the extension's files to show up on the system. It shouldn't matter whether the destination directory on the system is a real directory or a symlink to a directory.
-
Hi GNUser
Looking through the logs didn't reveal anything to me.
I was a bit surprised not to see any error messages from
the cp command.
Is it possible it is following the link and creating /etc/etc/ ?
Does adding this to the script reveal anything interesting:
ls -l /etc > "$TCE/lsetc.txt"
-
Hi Rich. The problem is nothing as fancy as /etc/etc being created and causing interference.
Please download the attached test.tcz. It contains a single, empty file: /usr/local/test/helloworld.
Then try this:
$ sudo mkdir /usr/local/test # this step is optional--if /usr/local/test doesn't exist, it gets created at the tce-load step
$ tce-load -i ./test.tcz
test.tcz: OK
$ ls /usr/local/test/helloworld
/usr/local/test/helloworld # helloworld gets created in the "test" directory as expected
Then try this:
$ sudo rm -rf /usr/local/test
$ sudo mkdir /test
$ sudo ln -s /test /usr/local/test
$ sudo rm /usr/local/tce.installed/test
$ tce-load -i ./test.tcz
test.tcz: OK
$ ls /usr/local/test/helloworld
ls: /usr/local/test/helloworld: No such file or directory # unexpected! tce-load doesn't like that "test" is a symlink, refuses to create any files in "test"
-
I understand and agree with reasoning behind having /usr/local/etc but the reality is that many extensions do not use it--either because the extension contributor was unaware of our convention or because the build process for some applications makes it difficult/impossible to specify the configuration directory. The problem is not that the /usr/local/etc convention exists--the problem is that not all extensions follow the convention.
My take has been that if an extension comes with default configuration files then it should go in /usr/local/etc because extension files shouldn't be outside of /usr/local. But I figured it was desirable to avoid this cludge if a default configuration isn't needed, and use /etc. Personally I'd like to only have symlinks in /usr/local and needing to put configuration files in there has caused me pain before, so the less of it the batter.
For your purposes, maybe just rsync /etc and /usr/local/etc after each edit or after running tce-load? Or a script that makes links to everything at /usr/local/etc in /etc after running tce-load? I started a script to do this but ran out of time (these things never just work), I might post back with it in a few hours/days/years...
-
Hi GNUser
Here's what it doesn't like:
tc@E310:~/YN$ /bin/cp -aisH /tmp/tcloop/test/usr /
cp: target '/usr/local/test' is not a directory
cp: can't preserve times of '/usr/local': Operation not permitted
cp: can't preserve ownership of '/usr/local': Operation not permitted
cp: can't preserve permissions of '/usr/local': Operation not permitted
cp: can't preserve times of '/usr': Operation not permitted
cp: can't preserve ownership of '/usr': Operation not permitted
cp: can't preserve permissions of '/usr': Operation not permittedIt's not a directory.
It wants a directory:
tc@E310:~/YN$ /bin/cp -rsfH /tmp/tcloop/test/usr /
cp: target '/usr/local/test' is not a directory
-
Hi Rich. Yes, I realize that the problem is that the cp step in tce-load wants a real directory and fails if it encounters a symlink to a directory instead. My question at this point is whether this behavior is fussy/unexpected enough to qualify as a bug.
Hi CNK. I considered some kind of wrapper script to tce-load but decided I don't want to complicate things. I'd like to either fix the problem at the root (i.e., in tce-load) or just leave this alone.
-
Hi GNUser
No, I don't think it qualifies as a bug. Based on the help
message from GNU cp, following links only applies to
source links:
----- Snip -----
-H follow command-line symbolic links in SOURCE
----- Snip -----
-L, --dereference always follow symbolic links in SOURCE
----- Snip -----
-
Hi Rich. I meant a bug in tce-load rather than a bug in cp . But perhaps it's best to leave this well enough alone: Tinkering with the relevant step in tce-load seems unwise given that it's a key step and the cp command with current flags is very extensively tested.
I guess my idea was a fool's errand after all ::)
Thanks for your help, as always :)
-
Hi GNUser
... Tinkering with the relevant step in tce-load seems unwise given that it's a key step ...
Agreed. But it might be possible to fix after tce-load.
Keep the link from /usr/local/etc->/etc
Add this to bootsync.sh before it calls bootlocal.sh:
for E in $(ls -1 /usr/local/tce.installed/)
do
if [ -e "/tmp/tcloop/$E/usr/local/etc" ]
then
sudo cp -ais "/tmp/tcloop/$E/usr/local/etc" /
[ -s /usr/local/tce.installed/"$E" ] && sudo /usr/local/tce.installed/"$E"
fi
done
/opt/Monitor.sh &It tests loaded extensions for usr/local/etc and links it to /etc
if it exists. It then runs the tce.installed script if it exists.
It then launches the Monitor.sh script:
#!/bin/sh
PIPE="/tmp/Load"
mkfifo "$PIPE"
inotifywait --quiet --monitor --recursive -e create --format '%f' /usr/local/tce.installed > "$PIPE" &
while true
do
IFS= read -r E <"$PIPE"
if [ -e "/tmp/tcloop/$E/usr/local/etc" ]
then
sudo cp -ais "/tmp/tcloop/$E/usr/local/etc" /
[ -s /usr/local/tce.installed/"$E" ] && sudo /usr/local/tce.installed/"$E"
fi
done
The next time tce-load installs an extension, inotifywait sends
the extension name to the FIFO. The read command retrieves
name and processes it if usr/local/etc exists.
I haven't tested it but I think it should work.
Check it for accuracy and typos.
-
Hi Rich. Thank you for such an interesting and thoughtful solution involving inotifywait . That should work just fine but I confess it's a bit more complex than the solution I was hoping to find.
I found something simpler that works, albeit with a caveat (which your solution does not suffer from). My solution is to add these lines to end of my /opt/bootlocal.sh:
rsync -a /usr/local/etc/ /etc
rm -rf /usr/local/etc
ln -s /etc /usr/local/etc
After boot is complete, /usr/local/etc is a symlink to /etc and everything is inside /etc , just as expected :)
Having just etc/ in /opt/.filetool.lst (rather than both etc/ and usr/local/etc/ ) works just fine, too :)
The caveat is that after boot process is complete, I need to refrain from loading any extensions that would try to put files in /usr/local/etc because the cp -ais step of tce-load cannot handle the symlink. In my opinion this as a minor bug in how tce-load works, but I cannot think of an elegant way to fix this at the moment.
-
Never mind my simple solution :'(
I looked in /etc/tc-config and found that bootsync.sh (and therefore bootlocal.sh) (line 641) runs user's backup is restored (tc-restore.sh on line 626). So to avoid the rsync from clobbering the files in etc restored in user's backup, we need this in bootlocal.sh:
rsync -a --ignore-existing /usr/local/etc/ /etc
rm -rf /usr/local/etc
ln -s /etc /usr/local/etc
The only problem with the above is that it breaks how /etc/sysconfig is built, and also seems to break fonts. What a can of worms.
I need a little break from this. Thank you all for your time and assistance.
-
I figured it out. The order of the steps is very important.
1. On system with default configuration (both /etc and /usr/local/etc exist, nothing etc-related in bootlocal.sh), do:
sudo rsync -a /usr/local/etc/ /etc
2. Look inside /etc and make sure fonts, sysconfig, and everything else looks good. Then, before rebooting, put etc/ in /opt/.filetool.lst and do a backup:
filetool.sh -b
3. Finally, add this to end of /opt/bootlocal.sh:
rsync -a --ignore-existing /usr/local/etc/ /etc
rm -rf /usr/local/etc
ln -s /etc /usr/local/etc
4. Reboot
After reboot everything is as expected.
As I mentioned before, the one issue is that any extensions that put files in /usr/local/etc must be loaded during boot. This is due to the cp -ais step in tce-load .
I think I may just live with the pain of having both /etc and /usr/local/etc after all. This may have been just a (worthwhile?) learning experience.
Thread may be marked as solved.
P.S. Come to think of it, the rsync command in bootlocal.sh is probably unnecessary, since everything the user needs is already in the backup created by filetool.sh .
-
Hi GNUser
... I looked in /etc/tc-config and found that bootsync.sh (and therefore bootlocal.sh) (line 641) runs user's backup is restored (tc-restore.sh on line 626). ...
These were all intentional.
Because cp -ais does not clobber files, running the restore first is safe.
bootsync.sh was chosen because it blocks and runs synchronously in
the foreground. This makes it possible to link local/etc / and run any
install scripts before any of those extensions get used.
bootlocal.sh runs asynchronously in the background allowing X, your
desktop, and other apps to start, potentially creating a race condition.
... Then, before rebooting, put etc/ in /opt/.filetool.lst and do a backup: ...
Generally you only want to backup files you've modified to minimize size
and time required to run your backup (and restore).
There may be some things in /etc that should not be backed up, possibly
things like fstab and udev.
Thread may be marked as solved. ...
Done.
-
Generally you only want to backup files you've modified to minimize size
and time required to run your backup (and restore).
There may be some things in /etc that should not be backed up, possibly
things like fstab and udev.
Hi Rich. I've gone back and forth between backing up only the files I've modified (to minimize size and so that .filetool.lst can serve as a record to remind me which files I've touched) and backing up all of /etc (for convenience). I never felt confident of which approach was superior until I read your reply today. Thank you very much for sharing your thoughts on this.