trouble with persistence

[jjacobs]

I need help with persistence.

I have added a couple of users, enabled dropbear and samba. I followed the wiki but have failed to have settings restored on reboot.

Here are the steps I took.

I added to /opt/shutdown.sh the line
filetool.sh backup

I added to .filetool.lst
etc
home/user1
home/user2
usr/local/etc/samba
usr/local/etc/samba/private
var/lib/samba

I ran the filetool.sh script in order to set the back up device which happens to be /dev/hda3.

I obtained the UUID of the device via:
blkid -s UUID /dev/hda3

I added to /opt/bootlocal.sh the line

restore=UUID=the uuid obtained earlier

When I reboot, the usual tinycore X background appears, but nothing is functional.

I am then forced to reboot from the CD sometimes twice in text mode in order to restore functionality.

After functionality is restored all user and samba settings have been lost.

First:
Does anyone see something glaringly wrong in my attempt to save settings?  Did I miss a step or a chapter?

Next:
Does the UUID need to be quoted in the /opt/bootlocal.sh restore line?

Finally:
Do I need to add to /opt/bootlocal.sh the line:
filetool restore
?

As always all help is greatly appreciated.

[gerald_clark]

If your tce directory is found, it should include mydata.tgz, and be automatically restored at boot.
What is the filesystem type of hda3?  It should be ext2 or ext3.

[bmarkus]

If your tce directory is found, it should include mydata.tgz, and be automatically restored at boot.
What is the filesystem type of hda3?  It should be ext2 or ext3.

/tce can be on FAT partition too.

[jjacobs]

fdisk -l shows:

Disk /dev/hda: 41.1 GB, 41110142976 bytes
255 heads, 63 sectors/track, 4998 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/hda1   *           1          39      313236  83 Linux
/dev/hda2              40         170     1052257+ 82 Linux swap
/dev/hda3             171        4997    38772877+ 83 Linux

hda1 and hda3 are ext3

After I reboot in text mode a couple of times I fear the mydata.tgz no longer has the directories listed in my previous post backed up.

[gerald_clark]

You are makeing  things too complicated.
All you need is a tce directory on /dev/hda3.

Boot once with the boot option "tce=hda3"
Install your applications, make your changes, do a backup.

[jjacobs]

I don't think gerald_clark understood my original post.  I made users and enabled samba. In order to maintain those users and my samba settings through reboot, I need to save more than the defaults that appear in /opt/.filetool.lst.

In the wiki under the title Persistence it says

"By default, all files and folders in /home/tc are saved to mydata.tgz when the filetool is run."

Clearly I need to back up more than what is in /home/tc. One example would be the directories of the users I created.  Another would be the passwd file in /etc.  There are others as well.

What is not clear to me is the section in the wiki titled "Restoring Automatically at Startup"

Do I need to add to /opt/bootlocal.sh the line:
filetool.sh restore
?

At the bottom of this section it states:
"Because of this, a kernel argument is used to ensure that the customized /opt/bootlocal.sh file is extracted before the startup process completes.

Using kernel argument
restore=UUID="

I took this to mean that I needed to add the line
restore=UUID=uuid of the location of my backup

Is this correct?

Am I correct in assuming that by adding to /opt/shutdown.sh
filetool.sh backup

the files and directories appearing in /opt/.filetool.lst will be backed up upon shutdown?

gerald_clark suggests
Boot once with the boot option "tce=hda3"

In my setup I believe this is taken care of in my grub menu.lst file in the line:
kernel /boot/bzImage quiet maxloop=255 waitsub=10 tce=UUID="uuid of my hard drive" tz=PST,M10.1.0/2,M4.2.0/2

[maro]

@jjacobs: First off I fully agree with gerald_clark, don't make it too complicated for yourself. I'd suggest you follow his advice.

Furthermore here is my take of what might have gone wrong or where you've confused yourself:

(1) '/opt/bootlocal.sh' is to start system wide (i.e. not user specific) processes at boot time. An entry like restore=UUID=... is certainly wrong in this file. That is a boot code that you either manually enter at the "boot:" prompt, or include in the boot loader configuration (e.g. 'menu.lst' for GRUB, 'grub.cfg' for GRUB2,  'extlinux.conf' for EXTLINUX, ...)

(2) During the boot process TC scans the available hard disks (and USB pen drives) in an attempt to find a '/tce' directory. IIRC it takes the first one it finds. So if you only have one across all available devices you won't need a boot code to guide TC. If you have several 'tce' directories (e.g. to boot different versions of TC and / or using different sets of extensions) you'll need to supply this "guidance".

(3) Likewise the 'mydata.tgz' file is expected to be located in the 'tce' directory. It will be restored automatically provided it can be found. Again if you've got only one in your one 'tce' directory you should not need the guidance via a boot code. To avoid restoration you'd use boot code 'norestore', but that appears to be the opposite of what you try to achieve. Check the content of '/opt/.backup_device'. If it is empty your 'mydata.tgz' file was not found. Similarly the content of '/opt/.tce_dir' tells you where TC found the 'tce' directory. If it still reads '/tmp/tce' you are in "cloud" mode, because no 'tce' directory was found. Don't try to alter the content of those files yourself, I'm just trying to give you some idea for troubleshooting.

(4) Adding an explicit backup instruction into '/opt/shutdown.sh' is in my view not required. AFAIK this file is called by '/usr/bin/exittc', which in itself is the GUI "logout" tool. And this tool has the default setting to execute a backup at shutdown, so you are just forcing TC to perform two backups immediately following each other. BUT this does not protect you from the "risk" of not having done a shutdown before you use something like sudo poweroff or sudo reboot. In this case you'd still have to execute a backup yourself if you want one (via filetool.sh backup).

(5) Using a UUID instead of a device name is recommended mostly when the 'tce' directory is on a movable device like a USB pen drive. Taking it from PC to PC might mean that the respective device name is different on the different PCs. Therefore the UUID (or a LABEL) is a good way to still identify the "location" of the 'tce' directory. A fixed hard disk will most likely not be moved around, so one could safely use the device name (or rather the partition name like 'hda3').

So again, do what gerald_clark suggested: boot ONCE with boot code 'tce=hda3' and TC will prepare the backup file and set things up to store your extensions in '/mnt/hda3/tce'. Afterwards you should not need the 'tce' boot code any more since TC should be able to find it automatically.

As alternative to backup / restore you could use persistent 'home' and / or 'opt' (e.g. also on 'hda3'). But this should better be another phase after you've become more familiar with TC. So don't try it all at once. The thought of "crawling before walking" comes to mind ...

[jjacobs]

It seems I've generated some unintentional emotion.  Emotion is good makes one feel alive.  

Back to my problem.   Maro have you or gerald-clark ever created a user, enabled samba and dropbear for those users then retained those settings through reboot?

Have either of you ever created a user then retained that users settings through reboot?  

If you had you would know that you need to maintain the user's home directory
/home/useryoucreated

You also need to retain the user's password.  If you added that user to a group
you need to preserve that user's group.

The files that have user and group settings occur in
/etc

Since I also enabled samba I would like to retain the samba settings through reboot.

I know at least the smb.conf file and the samba password data base would need to be preserved through reboot. I believe these files appear in
/usr/local/etc/samba
/usr/local/etc/samba/private
/var/lib/samba

I'm not certain about /var/lib/samba.

If the filetool shell script is unable to save and restore such settings I'll write my own script.  Does anyone know if filetool can accomplish this use case?

maro do you know more than the author of the wiki? The wiki under the section titled Saving Automatically at Shutdown says

To save your settings and files automatically at shutdown, add the line:
filetool.sh backup
to /opt/shutdown.sh.

Yet you advise
Adding an explicit backup instruction into '/opt/shutdown.sh' is in my view not required. AFAIK this file is called by '/usr/bin/exittc', which in itself is the GUI "logout" tool. And this tool has the default setting to execute a backup at shutdown, so you are just forcing TC to perform two backups immediately following each other. BUT this does not protect you from the "risk" of not having done a shutdown before you use something like sudo poweroff  or sudo reboot. In this case you'd still have to execute a backup yourself if you want one (via filetool.sh backup).

Which is correct?  Should one add the line filetool.sh backup to /opt/shutdown.sh or not.  If not could you update the wiki?

I appreciate the advice of not using my hard drive's UUID in the boot code.
I also appreciate the information on when opt/shutdown.sh is called.

I have been using reboot as su from a terminal window.

Have you or gerald_clark ever edited /opt/.filetool.lst?  Do you know what it's for? Maybe you can explain it better than the wiki.

[Juanito]

Adding these to .filetool.lst seems to enough for me with samba and tc/samba users for printer sharing for windows machines:

Code: [Select]

usr/local/etc/samba/smb.conf
usr/local/etc/samba/private
etc/passwd
etc/shadow

[jjacobs]

If one had created and placed a user in a group I suppose adding
etc/group

would also be needed.  

Can the contents of entire directories be restored?  In my original post I showed that I had added to /opt/.filetool.lst
etc

I got the idea that by placing the directory name all its contents would be saved from the wiki otherwise there would be no use for the file /opt/.xfiletool.lst.

I'll try on Monday my tc box is at the office.

Maybe my troubles stemmed from adding to /opt/.filetool.lst
var/lib/samba

or maybe as maro suggested placing the UUID of the hard drive in the boot code is causing of my troubles.

Juanito Is the wiki correct?  Or is maro correct?

Does the line filetool.sh backup

need to be added to /opt/shutdown.sh?

If the wiki is incorrect it should be updated.

Does anything need to be added to /opt/bootlocal.sh?

[Juanito]

Can the contents of entire directories be restored?

Yes

Maybe my troubles stemmed from adding to /opt/.filetool.lst
var/lib/samba

Why?

Does the line filetool.sh backup

need to be added to /opt/shutdown.sh?

To be honest, I never checked - on each machine I've used tc on, I edit .filetool.lst, use the backup/restore button in the panel and things "just work"

[jjacobs]

I appreciate the honesty.  Again because of everyone's help I have a few things to try on Monday.

I tried to save /var/lib/samba because I thought some part of the samba settings resided there. I was probably mistaken.

When you say things just work how are you doing the reboot?  Are you rebooting via sudo reboot in a terminal or are you exiting though the gui?

In my use case I would like to be able to reboot remotely using dropbear.

[maro]

@jjacobs: Just a quick remark to your comments in reply #7: The Wiki is a community page and it's content is not managed by the Core team. So a certain amount of the information is (potentially) wrong or outdated (due to the fast moving pace of TC/MC). I consider it the least authoritative source of TC/MC related information.

I have my reasons why I choose not to participate via the Wiki but rather offer my (limited) knowledge via the forum here. Rest assured that I tend to make my statements after having read the code and / or trying things out. At least I attempt to do that most of the time and I'm therefore having a high degree of confidence in the correctness of those statements (at the point in time of writing them). But due to the fast changing nature of TC/MC (and the fact that I even don't trust myself) I surround them with a lot of qualifiers (like "AFAIK" or "IIRC") to avoid the impression that they remain always correct.

You are right that I have not done your specific setup and my response was limited to your reported more "generic" persistence issues. IMHO you would gain most by using a persistent "home" instead of backup/restore for the user data, and limit the scope of backup/restore to the "areas" of '/opt' and '/etc' (which you'll need to manage accordingly).

I guess that is as far as I'd like to go in my response ("OVER AND OUT").

[Juanito]

When you say things just work how are you doing the reboot.  Are you rebooting via sudo reboot in a terminal or are you exiting though the gui.

From the gui and also from a terminal using "sudo exitcheck.sh reboot"

[jjacobs]

Thanks gerald_clark, maro, and Juanito!!  Perhaps the greatest thing I've found about tc is the repsonsiveness in the Forum.

maro you're right about gaining much from using persistent home. I read about it in the wiki after your first reply but have yet to give it a try.

I now have my use case completely working.  The use case is to have a couple of users, dropbear, and samba restored on remote reboot.

To do this I added this line to my grub menu.lst file
kernel /boot/bzImage quiet maxloop=255 waitsub=10 tce=hda3 home=hda3 tz=PST,M10.1.0/2,M4.2.0/2

Previously this line read
kernel /boot/bzImage quiet maxloop=255 waitsub=10 tce=UUID="uuid of my hard drive" tz=PST,M10.1.0/2,M4.2.0/2

The new line makes my home directory persistent and I no longer use the UUID of my hard drive in the boot code.

I added these lines to /opt/.filetool.lst:

etc/passwd
etc/group
etc/shadow
usr/local/etc/samba/smb.conf
usr/local/etc/samba/private

Finally I added these lines to /opt/bootlocal.sh

/etc/init.d/dropbear start
/usr/local/etc/init.d/samba start

Likely there is a more proper way to start these services up at boot, but this works.

Any and all suggestions are always greately appreciated.