Pedagogy -- Scripts instead of backups

[alu]

i need opinions

what do you think about making a page in the wiki in order to point users to another big functionality of TC -- i.e. load your personal TC with scripts instead of using a backup. why? because lot of issues can be avoided this way, and you can personalized TC as you like by just saving config. and pref. files onto the hd and/or an external device and call them on demand on boot or right after boot.

(capitals to make it clear) THIS IS NOT A WAY TO THROW BACKUP MODUS OF OPERATION AWAY AS ONE OF THE POSSIBILITY OF USING TC

but just a way to show that everything can be achieve in TC by using (very) simple scripts which play the same role as backups without drawbacks (solving an issue is a matter to change a script). 

that said, what do you think of a page with examples of scripts for users in order for them to understand the basics of TC customization by scripts only? hints: start TC with a given background, a given wm/DE, with symbolic links in order to restore config. files/users' directories for applications (ooo, firefox and other browsers etc.).

'what do you think' means here: useful or just plain non sense ?
tia al

[tinypoodle]

Nothing would prevent a user from having scipts of their choice as only content of their backup.

For such purpose I still see the existing backup mechanism as ideal.

[gerald_clark]

You can already use custom backup files and onboot lists.

lst=user1.lst mydata=user1data.

Now you have your custom bootlist and custom startup scripts in your private bootlocal.sh.

[Guy]

alu

You make comments like

why? because lot of issues can be avoided this way

and

simple scripts which play the same role as backups without drawbacks

I don't understand what issues and drawbacks you are talking about. It would be good if you gave examples.

I don't use backup. I think this is the best way to run tinycore on a hard drive.

You could have a page with examples of scripts. I suggest putting it under the heading "For Advanced Users" so new users don't get confused, and think this is how you have to run Tinycore. Anyone can contribute to the wiki. So you can write such a page if you want.

[alu]

thanks all for the feedback

Guy, i did such comments regarding my own experience -- i have made reports sometimes on the forum believing that something was wrong with tc, and had to face the fact that this was not the case, what was wrong was my backup; deleting the backup and restarting a fresh tc solves my problems. I think that a user has a similar problem with flwm (last posts on the forum) and -- maybe i am wrong -- i suspect that something is wrong with his backup. If you have a small backup, deleting it and reconstruct your own tc is not much time consuming, but if you have a big one, this could take a lot of time.

tinypoodle -- actually yes, but i see advantages by using scripts and symbolic links for config files and directories because you keep your /home/tc directory very light and tc untouched. for example, i have my sylpheed config files and my config files for ooo 2.3 on another device and i link them to the appropriate place into /home/tc; my /home/tc is not altered by these big files, and portability is secured because my mails and my preferences will be saved on my other device.

gerald_clark -- yes, but with my proposal, you even don't have to use bootlocal.sh and filetool.lst; everything can be written down in a start script that you run in a terminal after boot in order to restore your configuration and prefered apps. one drawback still remains -- you have to type one line per hand with the default us keyboard right after boot -- so this is not an all purpose solution (not for people using tc in kiosk/remote/server-only mode, for example). 

[tinypoodle]

tinypoodle -- actually yes, but i see advantages by using scripts and symbolic links for config files and directories because you keep your /home/tc directory very light and tc untouched. for example, i have my sylpheed config files and my config files for ooo 2.3 on another device and i link them to the appropriate place into /home/tc; my /home/tc is not altered by these big files, and portability is secured because my mails and my preferences will be saved on my other device.

Is your claim that avoiding backup would increase portability, compared to using backup? If so, that could make no sense whatsoever to me, without being able to understand how you would mean that - as I could not think of any way to provide more portability than with current backup mechanism as shipped in base.

[Lee]

I wonder if there's not some problem with the backup function that not everyone experiences?  I use the standard tc backup functionality all the time and it never gives me any trouble but perhaps it is not so for everyone.

What are the issues others are having with backup?  Why try to avoid using it?

[alu]

Lee -- the backup functionality in tc works well, this is not my purpose to criticize it, but to provide an alternate use of tc which could be as flexible as with backup -- and maybe more flexible than with backup.

tinypoodle -- partly yes, i think that you can get (maybe increase) the same functionality with scripts and symbolic links as with backup. Example: i have a usb device (sda1) which i am using to boot tc (it could be another device, but this is my setup). i also have a TC directory in which i save my scripts, config files and directories for some apps.  in this directory i have customized .xsession, .Xmodmap and a /user directory for Ooo 2.3, as well as a directory for .sylpheed. i am booting tc with the option 'text'. at the command line, i want to recover my usual configuration, so i run the following script (call it START):

#!/bin/bash
sudo cp /mnt/sda1/.xsession /home/tc/
ln -s /mnt/sda1/.Xmodmap /home/tc/
tce-load -i /mnt/sda1/tce/optional/kmaps.tcz
sudo loadkmap < /usr/share/kmap/qwertz/fr_CH-latin1.kmap
tce-load -i /mnt/sda1/tce/optional/mc.tcz
tce-load -i /mnt/sda1/tce/optional/links.tcz
tce-load -i /mnt/sda1/tce/optional/sylpheed.tcz
ln -s /mnt/sda1/.sylpheed-2.0 /home/tc/

the 4 first lines are explicit -- configuration of display, and keyboard. the lines 5 to 7 install the minimal apps i always need. the last line create a symbolic link to /home/tc in order for sylpheed to run with my usual preferences and with my mail-boxes.

additionally, i embed in the script above some symbolic links from scripts on sda1 to /home/tc. with these scripts, i can run other services or apps on demand. for example, the following lines link a script  to run LaTeX, to install cups and configure my printer, to get my wifi connection up and running:

ln -s /mnt/sda1/Tex /home/tc/
ln -s /mnt/sda1/Print /home/tc/
ln -s /mnt/sda1/Wifi /home/tc

when i need to be online, i can type sh Wifi, and i am connected.

now, i can shut my computer down without doing a backup, and restart a fresh tc.

i see following benefits over backup:

1. you can customize scripts for different machines without making different backups -- for example, i have configuration files related to display and keyboard configurations corresponding to 3 different computers; based on the script START, i have a STARTA script, a STARTB and a STARTC script with the corresponding config. files related to the appropriate computer. when i am moving from home to work -- i.e. from computer A to computer B, i run tc as usual, and at the prompt i run STARTB instead of STARTA.

2. using symbolic links for custom config files and user's directories make /home/tc remaining very light, which is good as we are running in ram.

3. tc boot time and shutdown time remain very quick because you don't load a backup at boot, and you don't do one at shutdown.

4. changing a preference (f.ex.: background image, wbar configuration and the like) is just a matter of editing a file which -- if the appropriate file has been linked from /sda1 to /home/tc -- can be done on the fly, and kept saved on our usb sda1 without making a new backup for each change one would like to do.

[tinypoodle]

alu,

My question was very specifically regarding portability.

It seems to me that you try to avoid to reply about my rather simple question "as is" - besides from making what is provided for with a very userfriendly mechanism in TC base sound extremely complicated.

Having uncompressed files reduces portability.
Having scattered files per definition reduces portability.
Having scattered files of an UNIX filesystem residing anywhere else than on a UNIX filesystem reduces portability even more (e.g. attributes, symlinks).

Mentioning devices as examples means to me that your alternative is dependent on specific devices which defeats every purpose of portability.

i am not claiming that the backup mechanism as is at current would be perfect and not be subject to improvement, but that the approach is ideal when it comes to providing portability and economy of resources usage.



P.S.: I can not understand how "pedagogy" would be related to the subject in any way.

[alu]

tinypoodle,

pedagogy -- because i thought that it could be interesting for other users to know/learn an alternate way to run a custom tc install besides backup.

portability

1. "uncompressed files reduces portability": if you say that the files/directories to be linked to /home/tc are not compressed, that's true, but still, i can not see the reason why it would reduce portability;

example with backup: you have a mydata.gz in your hd to be loaded/uncompressed on boot; now, you have to use another pc, so you take for example a usb-stick with tc installed on it, and you'll copy mydata.gz onto it in order to boot with tc and your backup on this other pc;

example with the alternate usage: you have a TC directory on your hd with config. files/prefs. files/directories for some apps; you have to use another pc, you do the same as above, and instead of copying mydata.gz onto your usb-stick, you copy your TC directory; basically, this won't make a difference towards the backup usual way;

example with the alternate usage 'bis': you are not device dependent, since you can remotely link/mount your config. files/directories; with mydata.gz, you have to have it with you since it will be uncompressed on boot (or you would use pxe boot, which is a very specific case)

2. "scattered files": this is rather on the opposite, this is an alternate way to use tc and to benefit from its speed without sacrificing portability (maybe you won't agree on this), assuming that each one has to organize their files/directories their way. in my case, files and directories are in one TC directory and i call them with scripts regarding the pc i am using.

my purpose is not to promote another mechanism which should replace the backup mechanism in the base. it is another way to get the same effects than with the backup mechanism, and maybe a way which could be of advantage for some users over the backup way as it is for me.

[tinypoodle]

tinypoodle,

pedagogy -- because i thought that it could be interesting for other users to know/learn an alternate way to run a custom tc install besides backup.

Well, not sure that the terminology is appropriate here, but that would be an issue of semantics.

portability

1. "uncompressed files reduces portability": if you say that the files/directories to be linked to /home/tc are not compressed, that's true, but still, i can not see the reason why it would reduce portability;

Most simply put: size is crucial for portability.

e.g. I have an instance of my backup stored in a router and one on a floppy. Both cases require recompression with a high performance compressor in order to fit.

Also, many providers of free shell accounts grant rather limited storage space where every MB might matter.

example with the alternate usage 'bis': you are not device dependent, since you can remotely link/mount your config. files/directories; with mydata.gz, you have to have it with you since it will be uncompressed on boot (or you would use pxe boot, which is a very specific case)

Umm, I see that quite exactly the other way round. If you have to link or mount then you lose portability, you could not do that with location and device independent storage on the internet.
You do not have to have mydata.gz with you but could retrieve it from any kind of internet storage from any location.
Restoring backup after boot has always worked fine for me.
Not sure why pxe boot should be regarded as a specific case, when TC base even includes a server out of the box.

2. "scattered files": this is rather on the opposite, this is an alternate way to use tc and to benefit from its speed without sacrificing portability (maybe you won't agree on this), assuming that each one has to organize their files/directories their way. in my case, files and directories are in one TC directory and i call them with scripts regarding the pc i am using.

Have you ever tried what you suggest on a FAT* filesystem? If so, was it free of any complications?
Also, it is way more simple to transfer a single file over the net compared to a file tree, though latter is not an impossibility.

[alu]

Most simply put: size is crucial for portability.

agree, and this is exactly what the proposed alternate way wants, e.g. keep things at a minimum size; just another example -- with my user profile for Ooo 2.3 and my sylpheed config files, i am at about 50 mb; if i would use a backup, i can use the TC utility or a high performance compressor; but still, i will use 50 mb in ram once uncompressed on boot; this might be not a big issue if you have plenty of ram, but it could be an issue if you don't have much;

now if the point for you is to use tc with a backup that you can retrieve over the internet, the alternate way is maybe not the best one. this said, i don't trust providers enough to let them store my backup/my files and directories, and i won't advice user to do so. the simplest solution to me is to have your backup or (in the proposed alternate way) your files/directory on an external device to carry with you where you want to use a pc with TC. or to have a private server.

If you have to link or mount then you lose portability, you could not do that with location and device independent storage on the internet.

you don't have to, but you can do it -- and not by using independent storage on the internet which is (in my opinion) risky and which has limitations, but by using your own private server for example. but i agree with you that in this case, the proposed alternate way is maybe not the best one ~ which might also depend on other factors like bandwidth etc.

Not sure why pxe boot should be regarded as a specific case, when TC base even includes a server out of the box.

just to point out that users with pxe boot can take advantage of both solutions, even if pxe boot is probably not the preferred way to use tc by most users -- specific in this sense

Have you ever tried what you suggest on a FAT* filesystem? If so, was it free of any complications?

obviously, working with file systems that does not cope with permissions will be a source of complications for anything, but i don't use these file systems, and won't advice other users to use them.

also, this does not mean to transfer trees over the internet; i mean, if you have a huge backup of (say for our discussion) 200 mb, it does not make much sense to retrieve it remotely, too. if you have a little one (4-5 mb), there is not much difference to retrieve a directory or a compressed backup.

again, this is neither a replacement alternative for the backup mechanism in the base, nor an all-purpose solution, but another way to use and custom tc of which everyone could benefit given some implication in scripting.

[tinypoodle]

Have you ever tried what you suggest on a FAT* filesystem? If so, was it free of any complications?

obviously, working with file systems that does not cope with permissions will be a source of complications for anything, but i don't use these file systems, and won't advice other users to use them.

That's where I and by my perception TC documentation disagree with you.
I am using TC in "mount mode" with backup/restore on FAT32 since 21 months and have never ever noticed the smallest complication. That is certainly an issue most relevant to portability.

also, this does not mean to transfer trees over the internet; i mean, if you have a huge backup of (say for our discussion) 200 mb, it does not make much sense to retrieve it remotely, too. if you have a little one (4-5 mb), there is not much difference to retrieve a directory or a compressed backup.

So, to be able to talk based on facts, I inspected my current backup and am providing some numbers here:

mydata.tgz 2083 KB - recompression could shrink the size to nearly half if so required.
untarred backup: 8213 KB (1425 files, 312 directories) 8 top level directories.
I see a difference of retrieving 1 directory versus a tree of 8 top level directories containing a total of 312 directories.
Also, I couldn't figure how to do the equivalent of storing backup as an e-mail attachment with a file tree.

[alu]

fat32 -- TC documentation does not disagree with the fact that fat32 file system is a file system which does not support permissions; your are lucky with your file system and i am happy for you, i have had TC and all my docs on a 8 gb usb-stick with a fat32 file system and after a year, i have had a crash disk, and have lost files, something that never happened to me with ext2 or ext3 file system on usb-stick.

your facts -- i am not speaking of trees, i am speaking of config files, and users' directories for applications like Oooo 2.3, web browser, email client and the like (examples in posts above); besides, it would be very surprising to me that you would have 1425 files and 312 directories of that sort, and i don't know if this can be compared with what i say -- because if you would use bon echo, open office and sylpheed, and if you would make a backup in order to restore their related files and directories in /home/tc, you would be above 2 MB quickly. i have at the moment bon echo (with no cache), openoffice 2.3, sylpheed 3.1.1, geany running (also other apps, but they are not big) which save their config files in /home/tc, and with my way to do it, i have /home/tc at 6.4 MB with this (important) precision: i apply my alternate proposal only to sylpheed and Ooo 2.3 (e.g. not for geany and bon echo).

[cosmin_ap]

I use this approach myself, if somewhat more modular and portable than yours, in two respects:

1) I made a script /opt/mount <disk-label> which mount disks by label to /mnt/<label> and I symlink from that point.

2) I made a script /opt/require <extension> which runs /opt/<extension>/install.sh if found, otherwise tries to download and/or install a TC extension; you can see this as a post-install hook where you can customize your extensions as you want by copying over (or symlinking) your own files which would conveniently reside in /opt/<extension>; /opt/require also touches tce.installed so that another call /opt/require of the same extension silently does nothing -- this has a neat consequence: you can use /opt/require to ask for dependencies and not care about load order.

Example (from my head, not tested):

bootlocal.sh:

Code: [Select]

/opt/require timezone
/opt/require ntpclient
/opt/require tftp
/opt/require cosmin   #user "extension"

/opt/cosmin/install.sh:

Code: [Select]

adduser -u 5000 cosmin
addgroup cosmin staff
cp -af /opt/cosmin/home/. /home/cosmin/.
chown cosmin:cosmin -R /home/cosmin
/opt/require samba
/opt/require msmtp
/opt/require ntpclient

Btw, I totally see the benefits over mydata.tgz:
- you want to symlink files from mounted drives into /srv -type locations like samba shares, tftp or www root dirs.
- you want to fix permissions on each reboot so that messing them up won't persist between reboots
- mydata.tgz is getting big
- your "persistent" files don't survive power failure and backup is not done upon calling poweroff, a mounted /opt would (if you disable the write cache)
- you have extensions that add files to /etc/.skel so you want these extensions already loaded by the time you issue adduser.

Unfortunately, dismissive replies like "you can already do this with x, y or z" or arguments over semantics of portability are easier to issue than understanding of the problem and/or solution.