Backup as a background task (that commits at shutdown)

[Ulysses_]

During a reboot the backup can take an annoyingly long time.  

Is it possibly a good idea to make backup a background task that:

1. only runs when the cpu and drives are idle
2. only transfers modified files
3. does not commit until you explicitly select Backup at shutdown

Also,

4. Vmware filing systems reside inside a .vmdk file and there is a tool where the host can mount this file and modify its contents. If a similar concept (mounting read-write a fs that resides in a single file) is available in linux, and maybe "tarfs" is it, then this might save cpu cycles as keeping it up to date would only require modified files to be written to it, rather than creating the entire mydata.tgz.

[curaga]

Google "loop mounts". Keep a persistent home to avoid long backup times.

[roberts]

Use control panel -> System Stats -> bigHomeFiles
Adjust .xfieltool.lst to eliminate backing up unnecessary files,
and/or factor out static data to personal tcz extensions
or mount and use a persistent store.

[Ulysses_]

factor out static data to personal tcz extensions
or mount and use a persistent store.

The persistent store solution is not according to the TC philosophy that makes it super-fast by having the filing system in memory.

Are files from personal tcz extensions kept in memory if you load the extension but do not access the files?

[curaga]

Depends on whether those extensions are copied to fs (ram) or not.

[Rich]

Hi Ulysses_
While background monitoring and backup may be appealing, there is a simpler way that could result
in considerable time savings if a lot of your files never get changed. Create a file called staticfiles.lst
containing the names of paths and filenames that don't change. Next, set up a script updatestaticfiles.sh
that tars and gzips that list of files and saves it to staticfiles.tgz on your backup media. The third step
could be one of the following methods:

1. Append your list to the xfiletool.lst file or
2. Modify the backup script to use your list in addition to xfiletool.lst.

Then add a line to bootlocal.sh to gunzip and untar the files to restore them upon start up.

Once set up your backups should run faster. When your backups start slowing down again add the
offending files to staticfiles.lst and run updatestaticfiles.sh. If you use method 1 above you will also
need to update xfiletool.lst.

One thing to be careful of is that you DO NOT add files required by the system during startup, stick
to personal files.

[Ulysses_]

Alright.

Looking at the big picture, the idea is to try and limit the backup to as little modifiable data as possible, if I understand correctly.  

And therefore one must avoid using too much of the functionality in the repo, as too much writeable data would make the backup painfully slow?  

Instead, what if the backup happened in the background and therefore size was not an issue?  Then even caches could be backed up, painlessly and automatically, eg web server or browser caches.  While sticking to the philosophy of doing everything in ram for maximum performance.

[Guy]

Ulysses

I am not sure if you are running Tiny Core from a usb drive, or a hard drive.

If you are running it from a usb drive, or other solid state memory, using backup will prolong the life of the drive.

If you are running Tiny Core from a hard drive, you are better off with persistent home and opt. Any difference in performance will only be fractions of a second. You will not notice it. The only thing you will notice is the time taken to start up and shut down when using backup.


I was never concerned about the time taken to shut down. You can shut down Tiny Core and walk out of the room. After backup is complete, the computer will be turned off.

What annoyed me, was the time taken to start up. You want to be able to turn the computer on and use it. Not turn it on, and wait and wait and wait.

I have not used backup for a long time. I use persistent home and opt, and make new extensions for personal settings.

[Ulysses_]

So flash wear is the reason for the backup scheme, not performance?  

And having home in the ram fs does not make a noticeable difference in performance?  Any benchmark results to back this up?  

One expects many applications such as a caching web server to be faster with the cache in the ram fs, than with persistent home and opt.

I was never concerned about the time taken to shut down.

Sure, it was the Reboot that was annoying.

Running TC from a hard drive, but from a fat32 partition (for easy access to the data from the vmware host using the vmware mount tool).

[Guy]

Why don't you do an experiment. Use persistent home and opt for awhile, and see what you think.