filetool.sh: Why not using rsync as backup utility?

[soundcheck]

Hi folks.

I just had an idea, while waiting for the backup to be finished.

The backup takes quite some for obvious reasons. 

The main reason (as far as I can see): you cp/tar the entire persistent stuff from scratch all the time.

It might be worth to spend a thought on rsync.
This way you'd only save the deltas. And the backups where done in a second.

Please apologize if this is a nonsene idea. (I am still getting into TC)

[helander]

I see two potential problems with the proposal compared to the current solution:

• Can you rsync to FAT volumes and retain all file properties correctly (ownership, permissions, etc)?
• The current solution compresses the data. This might be of benefit for some users. For one it speeds up the restore.

/Lars

[linus72]

If your backup is taking a long time to "backup"
you should check your trash can(if using XFE), clear your firefox cache, etc
and also I noticed that firefox sooner or later makes a 30-50mb "database"
or something, which can only be resolved by uninstalling firefox and re-installing anew.

I no longer use Firefox because of this:)
Opera with flash10 works and is light.

My backup takes about 5-8sec at reboot,etc
that's with opera, flash10, xfe, xfw, mtpaint, and OSS
a few wallpapers and my dragon jwmthemes

[mcnalu]

Good tip on ff. Opera on tc is amazingly fast and renders pages well.

5-8 secs is an eternity in tc though!

Fat or ntfs can't handle symlinks which rules out rsync for me.

Best solution at present would be persistent home.

Andrew

[linus72]

Yes, to clarify, that's on USB without home or opt.

I have alot of wallpapers and themes, which slows it a little too.

Opera is the ticket though.

I discovered the firefox thing when trying to make a small tinycore cd
after doing bookmarks, etc the damn thing would somehow build a database or something in /home/tc/.mozilla and my little cd would balloon to over 100mb.

so, no-go firefox

I checked and without my wallpapers and themes the backup is like 2 sec

[roberts]

Minefield / Firefox /... and those pesky sqlite databases have been discussed. They can really slow down your backup and use up much space. See: http://forum.tinycorelinux.net/index.php?topic=492.0

Other tips to reduce your backup time, is to factor out static data into personal tce.
Just use tar command to make, for example, mybackgrounds.tce, which could be the wallpapers and themes.

As for rsync, I use it as the primary tool to backup all the development files for TC and MC. But for reasons stated it is not the default. However, rsync is included in the base system.

[linus72]

I didn't realize I could make a tce of my wallpapers and themes?
what is the process to do this Roberts?

simply tar the opt and jwmthemes folders and rename it .tce or what?
I have alot of themes
( http://multidistro.com/tinycore-shots/tc-scrnshots.html )

and each wallpaper, jwmtray, jwmtheme, etc are all different for each theme
so how would I make sure when I want blackdragon theme+wallpaper that it would work?

would I have to seperate each theme/wallpaper into a different tce?
right now I use grub menu entry to boot into theme I want

this is for my 4gb ext3 kingston usb tc entry

Code: [Select]


title <<<<---Back to Main Menu
root (hd0,0)
configfile /boot/grub/menu.lst

title TinyCore_2.1-Amethyst Dragon
root (hd0,0)
kernel /boot/bzImage quiet desktop=jwm restore=sda1/colors/amethyst tce=sda1/tce waitusb=5 max_loop=255 noicons embed
initrd /boot/tinycore.gz

title TinyCore_2.1-Black Dragon
root (hd0,0)
kernel /boot/bzImage quiet desktop=jwm restore=sda1/colors/black tce=sda1/tce waitusb=5 max_loop=255 noicons embed
initrd /boot/tinycore.gz

title TinyCore_2.1-Blue Dragon
root (hd0,0)
kernel /boot/bzImage quiet desktop=jwm restore=sda1/colors/blue tce=sda1/tce waitusb=5 max_loop=255 noicons embed
initrd /boot/tinycore.gz

title TinyCore_2.1-Gold Dragon
root (hd0,0)
kernel /boot/bzImage quiet desktop=jwm restore=sda1/colors/gold tce=sda1/tce waitusb=5 max_loop=255 noicons embed
initrd /boot/tinycore.gz

title TinyCore_2.1-Red Dragon
root (hd0,0)
kernel /boot/bzImage quiet desktop=jwm restore=sda1/colors/red tce=sda1/tce waitusb=5 max_loop=255 noicons embed
initrd /boot/tinycore.gz

title TinyCore_2.1-Silver Dragon
root (hd0,0)
kernel /boot/bzImage quiet desktop=jwm restore=sda1/colors/silver tce=sda1/tce waitusb=5 max_loop=255 noicons embed
initrd /boot/tinycore.gz

title TinyCore_2.1-White Dragon
root (hd0,0)
kernel /boot/bzImage quiet desktop=jwm restore=sda1/colors/white tce=sda1/tce waitusb=5 max_loop=255 noicons embed
initrd /boot/tinycore.gz

[roberts]

Yikes, so many backups, how do you keep track of your real data?
In concept based on your grub menu.lst, I would make each wallpaper/theme a tce with a startup script to copy them to their proper locations. I would store them in the optional directory under the tce directory. I would use a custom boot option of say, theme=BlackDragon. Then in .xsession test for the contents of theme boot code and tce-load the result.Finally, adjust your grub with the new theme boot option. Now you will once again have one common backup file and it will be much faster. There are obviously other ways to approach this, but this is one. Have fun with it.

[Juanito]

I didn't realize I could make a tce of my wallpapers and themes?
what is the process to do this..

Load the advcomp extension, then something like

Code: [Select]

$ tar -czvf my.tce --numeric-owner -T files
or
$ tar -czvf my.tce --numeric-owner file1 file2 ...
then
$ advdef -z4 my.tce
..where "files" is a file containing the names of the files you would like to include in your tce

Edit: I'm assuming the files are owned by tc:staff otherwise you need to use "sudo"

[linus72]

OK, thanks very much for that info Roberts and Juanito!

I'm gonna try it now.

Yikes, so many backups, how do you keep track of your real data?

lol, there actually is no "real" data there, I make multidistro stuff so I try to keep everything real clean.
meaning, I just installed opera, flash, oss, xfe and xfw and my wallpaper/theme stuff.

I use it mostly for jammin down some heavy tunes, and web-surfing

I will say that tinycore with just OSS is louder than Puppy, or my Ultimate-edition 2.2 (ubuntu 9.04), on my 300w speakers, and that's good too

I'm still a little confused on all you said there Roberts, but I will try to muddle thru.

[soundcheck]

I see two potential problems with the proposal compared to the current solution:

• Can you rsync to FAT volumes and retain all file properties correctly (ownership, permissions, etc)?
• The current solution compresses the data. This might be of benefit for some users. For one it speeds up the restore.

/Lars

Hi.

1.
For fat32 you need to use the --modify-window option

man-page:
--modify-window
When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than the modify-window value. This is normally 0 (for an exact match), but you may find it useful to set this to a larger value in some situations. In particular, when transferring to or from an MS Windows FAT filesystem (which represents times with a 2-second resolution), --modify-window=1 is useful (allowing times to differ by up to 1 second).

Code: [Select]

rsync -av  --modify-window=2 /srcdir /destdir

2. Compression

Since I work with different builds and storage space is no issue I could easily live with uncompressed trees on my media. I could also move data
from one tree to another.


With rsync I can even easily implement incremental backups, in case I messed something up I can easily rollback to an earlier stage.
The incremental backups just include the delta data. space is no subject.


-------------

The above comments "Take Opera" -- keep your system lean. Come on. That's not the way it works.

I have a build environment , with quite some sources and kernel-headers, etc. Backup gets quite annoying.





 

[Guy]

Backup will be quicker if you add

home/tc/.xfe/trash
home/tc/.mozilla

to .xfiletool.lst

If you use those programs.

Also save any large files, such as downloads, in a folder other than /home/tc

If you use home you can save files in /home rather than /home/tc

If you use encrypted home, you can save files in /mnt/sda1

[alu]

just tried to use rsync in order to backup folders and data from tc to my home server with:

rsync -a -e ssh /mnt/sda1/MyfolderonTC/ ssh me@myhomeserver:/home/me/folder/backupdirectoryonmyserver

impossible to perform the backup even if i can connect to my server; rsync ends up saying:

rsync: connection unexpectedly closed (0 bytes read so far)
rsync error: error in rsync protocol data stream ...

[vitex]

Use

Code: [Select]

rsync -av /mnt/sda1/MyfolderonTC/  me@myhomeserver:/home/me/folder/backupdirectoryonmyserver


[Guy]

I use Tiny Core with persistent home and opt, and make extensions for anything with personal settings, such as printer setup, and don't use backup at all.

People are asking, how can we improve backup?

I think a better question is, how can we set up Tiny Core to not use backup at all?


The only situation where backup may be an advantage is when running from a USB drive, as it will result in the USB drive lasting longer.

If you run from a USB drive and browse the internet, backup takes too long, so it is easier not to use it (or don't backup internet browser cache). It is probably easier to buy a new USB drive when it fails, than to waste too much time backing up.

I would only use backup, is if I ran Tiny Core from a USB drive and did not use it for browsing the internet.


The other place where backup is useful (when set up this way) is to backup documents (not Tiny Core files and settings).


These are personal preferences. I know others have different opinions.