[Solved] /etc vs. /usr/local/etc pain

[GNUser]

For me, one of the very few pain points (possibly the only one) in using TCL is the fact that some extensions use /etc for config files while other use /usr/local/etc.

The pain arises when I'm trying out an extension for the first time: It takes some investigating and/or trial and error to figure out which etc directory to use.

I understand and agree with reasoning behind having /usr/local/etc but the reality is that many extensions do not use it--either because the extension contributor was unaware of our convention or because the build process for some applications makes it difficult/impossible to specify the configuration directory. The problem is not that the /usr/local/etc convention exists--the problem is that not all extensions follow the convention.

I'm not suggesting that the convention of using /usr/local/etc for extensions be abolished. But I would like to explore whether it's possible for a user who would rather have a single etc directory (e.g., me) to tweak his TCL system so as to achieve this.

I'm on TCL15 x86_64 and already tried remastering corepure64.gz by adding a symlink at /usr/local/etc that points to /etc. Booting with the remastered corepure64.gz did not go well.

Is this a fool's errand or is there an elegant way to achieve a working system where /usr/local/etc simply points to /etc? Many thanks for any insights/advice you can give.

[Leee]

For me, one of the very few pain points (possibly the only one) in using TCL is the fact that some extensions use /etc for config files while other use /usr/local/etc.

The pain arises when I'm trying out an extension for the first time: It takes some investigating and/or trial and error to figure out which etc directory to use.

I understand and agree with reasoning behind having /usr/local/etc but the reality is that many extensions do not use it--either because the extension contributor was unaware of our convention or because the build process for some applications makes it difficult/impossible to specify the configuration directory. The problem is not that the /usr/local/etc convention exists--the problem is that not all extensions follow the convention.

I'm not suggesting that the convention of using /usr/local/etc for extensions be abolished. But I would like to explore whether it's possible for a user (me) to tweak his TCL system so that everything goes into /etc.

I'm on TCL15 x86_64 and already tried remastering corepure64.gz by adding a symlink at /usr/local/etc that points to /etc. Booting with the remastered corepure64.gz did not go well.

Is this a fool's errand or is there an elegant way to achieve a working system where /usr/local/etc simply points to /etc? Many thanks for any insights/advice you can give.

My best suggestion would have been exactly what you already tried - symlink /usr/local/etc to point to /etc - although I would have, perhaps naively, tried a lazy way first without remastering.  Remastering is clearly a better solution.

What were the details of "didn't go well"?

[GNUser]

What were the details of "didn't go well"?

Hi Leee. Boot process completes and extensions are loaded, but instead of seeing my fluxbox desktop with custom desktop background I briefly see TCL's default desktop background and then screen turns black and I can't see the desktop. Hard to say exactly what's broken but the "fix" is to reboot from the console into my recovery partition and revert the remastered rootfs to stock rootfs.

So my conclusion is that the /usr/local/etc to /etc symlink in the rootfs is breaking one or more extensions.

[Leee]

What were the details of "didn't go well"?

Hi Leee. Boot process completes and extensions are loaded, but instead of seeing my fluxbox desktop with custom desktop background I briefly see TCL's default desktop background and then screen turns black and I can't see the desktop. Hard to say exactly what's broken but the "fix" is to reboot from the console into my recovery partition and revert the remastered rootfs to stock rootfs.

So my conclusion is that the /usr/local/etc to /etc symlink in the rootfs is breaking one or more extensions.

I haven't tried making any substantial mods to corepure64.gz (I'm just in the early stages of playing around with remastering) so sorry if this obvious or way out in left field but, even that early in the boot process... did you copy everything from /usr/local/etc/ into /etc/ before making the symlink?

Also, for simplicity in testing, I think you can create an additional .gz file (maybe corepure64_2.gz), containing only your modified files, and load it after the original and files in it will replace the original files from the original .gz.  Then turning your mods on or off is just a matter of loading or not loading the extra .gz file at boot time.  (Dead simple using grub but it's been ages since I used any other bootloader.)

FWIW, for me, the pain of having both /etc/ and /usr/local/etc/ is so minuscule that I would have never tried this particular mod.  Far worse, for me, is the pain of trying to get the line spacing the way I want it here in the forum.   

[GNUser]

Hi Leee. Yes, using multiple initramfs archives and turning them on/off via bootloader is a nice trick.

I think I'm just going to give up on this mod. A symlink in the rootfs seems like the most promising approach but it's definitely breaking extension(s). I think having to troubleshoot the breakage is more painful than just living with the two etc directories.

Happy hacking!

[Rich]

Hi GNUser

Hi Leee. Boot process completes and extensions are loaded, but instead of seeing my fluxbox desktop with custom desktop background I briefly see TCL's default desktop background and then screen turns black and I can't see the desktop. Hard to say exactly what's broken ...

You could try saving some information to persistent storage (tce/).
Add  syslog  and  loglevel=7  to your boot codes.
Create this script:

Code: [Select]

#!/bin/sh

# Allow time for the desktop to finish crashing.
sleep 10

# You may have to specify  /mnt/sda...  here if the path can't
# be retrieved from /etc.
TCE="Path/to/tce/directory"

cp /var/log/messages* "$TCE"
dmesg > "$TCE/dmesg.txt"
Then launch that script from the end of bootlocal.sh

My guess is this command that gets executed after mounting
an extension might be the issue:

Code: [Select]

COMMAND=/bin/cp -ais /tmp/tcloop/mirrors/usr /
The busybox cp looks like this:

Code: [Select]

tc@E310:~$ busybox cp --help
BusyBox v1.29.3 (2018-12-19 15:29:37 UTC) multi-call binary.

Usage: cp [OPTIONS] SOURCE... DEST

Copy SOURCE(s) to DEST

        -a      Same as -dpR
        -R,-r   Recurse
        -d,-P   Preserve symlinks (default if -R) < ### This probably stops it from overwriting your link.
        -L      Follow all symlinks < ### Adding this might make it work.
        -H      Follow symlinks on command line
        -p      Preserve file attributes if possible
        -f      Overwrite
        -i      Prompt before overwrite
        -l,-s   Create (sym)links
        -T      Treat DEST as a normal file
        -u      Copy only newer files

[GNUser]

Hi Rich. Yes, the problem has something to do with how extensions are loaded.

When booting with the modded rootfs, /etc would be expected to contain all the files and directories normally found in /etc *plus* those normally found in /usr/local/etc. Actual result is that files and directories normally found in /usr/local/etc are totally absent from the system.

I will explore the  cp  command flags in  tce-load  and see if they are the reason for my mod not working as expected. Thanks for the tip.

[GNUser]

Hi Rich. Changing this line in /usr/bin/tce-load:

Code: [Select]

yes "$FORCE" | sudo /bin/cp -ais /tmp/tcloop/"$APPNAME"/* / 2>/dev/nullto this:

Code: [Select]

yes "$FORCE" | sudo /bin/cp -aisL /tmp/tcloop/"$APPNAME"/* / 2>/dev/nulldid not help.

The problem is this: If an extension containing /path/to/parentdir/somefile is loaded on a system where parentdir is a symlink, /tmp/tcloop/<extension_name>/path/to/parentdir/somefile appears on the system as usual but /path/to/parentdir/somefile never appears on the system.

PS: Rich, I'll send you a PM with the logs you requested. There's too much data in there for this privacy-paranoid user to post here. If we find a solution I'll make sure to post it here.

PS2: Did we find a bug in tce-load? It seems so: When an extension is loaded, user expects the extension's files to show up on the system. It shouldn't matter whether the destination directory on the system is a real directory or a symlink to a directory.

[Rich]

Hi GNUser
Looking through the logs didn't reveal anything to me.
I was a bit surprised not to see any error messages from
the cp command.

Is it possible it is following the link and creating  /etc/etc/ ?
Does adding this to the script reveal anything interesting:

Code: [Select]

ls -l /etc > "$TCE/lsetc.txt"

[GNUser]

Hi Rich. The problem is nothing as fancy as /etc/etc being created and causing interference.

Please download the attached test.tcz. It contains a single, empty file: /usr/local/test/helloworld.

Then try this:

Code: [Select]

$ sudo mkdir /usr/local/test # this step is optional--if /usr/local/test doesn't exist, it gets created at the tce-load step
$ tce-load -i ./test.tcz
test.tcz: OK
$ ls /usr/local/test/helloworld
/usr/local/test/helloworld # helloworld gets created in the "test" directory as expected
Then try this:

Code: [Select]

$ sudo rm -rf /usr/local/test
$ sudo mkdir /test
$ sudo ln -s /test /usr/local/test
$ sudo rm /usr/local/tce.installed/test
$ tce-load -i ./test.tcz
test.tcz: OK
$ ls /usr/local/test/helloworld
ls: /usr/local/test/helloworld: No such file or directory # unexpected! tce-load doesn't like that "test" is a symlink, refuses to create any files in "test"

[CNK]

I understand and agree with reasoning behind having /usr/local/etc but the reality is that many extensions do not use it--either because the extension contributor was unaware of our convention or because the build process for some applications makes it difficult/impossible to specify the configuration directory. The problem is not that the /usr/local/etc convention exists--the problem is that not all extensions follow the convention.

My take has been that if an extension comes with default configuration files then it should go in /usr/local/etc because extension files shouldn't be outside of /usr/local. But I figured it was desirable to avoid this cludge if a default configuration isn't needed, and use /etc. Personally I'd like to only have symlinks in /usr/local and needing to put configuration files in there has caused me pain before, so the less of it the batter.

For your purposes, maybe just rsync /etc and /usr/local/etc after each edit or after running tce-load? Or a script that makes links to everything at /usr/local/etc in /etc after running tce-load? I started a script to do this but ran out of time (these things never just work), I might post back with it in a few hours/days/years...

[Rich]

Hi GNUser
Here's what it doesn't like:

Code: [Select]

tc@E310:~/YN$ /bin/cp -aisH /tmp/tcloop/test/usr /
cp: target '/usr/local/test' is not a directory
cp: can't preserve times of '/usr/local': Operation not permitted
cp: can't preserve ownership of '/usr/local': Operation not permitted
cp: can't preserve permissions of '/usr/local': Operation not permitted
cp: can't preserve times of '/usr': Operation not permitted
cp: can't preserve ownership of '/usr': Operation not permitted
cp: can't preserve permissions of '/usr': Operation not permittedIt's not a directory.

It wants a directory:

Code: [Select]

tc@E310:~/YN$ /bin/cp -rsfH /tmp/tcloop/test/usr /
cp: target '/usr/local/test' is not a directory

[GNUser]

Hi Rich. Yes, I realize that the problem is that the  cp  step in  tce-load  wants a real directory and fails if it encounters a symlink to a directory instead. My question at this point is whether this behavior is fussy/unexpected enough to qualify as a bug.

Hi CNK. I considered some kind of wrapper script to  tce-load  but decided I don't want to complicate things. I'd like to either fix the problem at the root (i.e., in tce-load) or just leave this alone.

[Rich]

Hi GNUser
No, I don't think it qualifies as a bug. Based on the help
message from GNU cp, following links only applies to
source links:

Code: [Select]

----- Snip -----
  -H                           follow command-line symbolic links in SOURCE
 ----- Snip -----
  -L, --dereference            always follow symbolic links in SOURCE
 ----- Snip -----

[GNUser]

Hi Rich. I meant a bug in  tce-load  rather than a bug in  cp  . But perhaps it's best to leave this well enough alone: Tinkering with the relevant step in  tce-load  seems unwise given that it's a key step and the  cp  command with current flags is very extensively tested.

I guess my idea was a fool's errand after all

Thanks for your help, as always