Multiple mydata.tgz files

[jerramy]

Something that I need TinyCore to do, that it doesn't, is to allow multiple mydata.tgz type files, with independant .filetool.lst files.  Additionally, it would need to verify that files have changed before updating the file, and only update the parts that have changed.

TAR files would probably be better, so that the whole file doesn't have to be re-written due to a small change.  This would really help out when using a flash drive as the boot+storage medium.  Here's what I'm thinking:

# /opt/.master.filetool.lst 
#contains paired entries:
/opt/.filetool.lst mydata.tar
/opt/.filetoolA.lst mydataA.tar
/opt/.filetoolB.lst mydataB.tar

Each file listed first in each pair lists files that will get backed up to the second file in the pair.  If no files have changed in a given fileset, then there's no need to re-write the data. 

My problem is that I'm using a TinyCore flash drive to do development for a small linux controller, and it's SDK only supports linux.  But most of the system development (other systems attached to the controller) has been done in Windows, and my laptop has Windows7 on it.  I know I should just install TinyCore in a virtual machine, but had issues getting the shared networking set up...

So I've been swapping back and forth between booting to TinyCore and Windows.  The boot process is fast enough, but saving my entire home directory (where the SDK has installed itself) takes forever, and is probably wearing out my flash drive.  I've been forced to just make note of my changes to my development code (usually just small changes in the field) and re-do them every time, after shutting down TCL without saving.

Does anyone know if such a system (or something similar) already exists in TinyCore?  I know I could create an extension from the SDK, but was hoping for a system to conditionally (but automatically) back up file sets, independant of eachother.

[gerald_clark]

Boot option: mydata=othername
will result in your backup file being othername.tgz

Boot option lst=other.lst
will result in the use of other.lst for your onboot list

[jerramy]

Is the loading of mydata.tgz done in the kernel, or is it something that is in a script somewhere?  If it's in a script, I could probably change it to load more than one restore file.

I assume that the process of /opt/.filetool.lst getting used to create mydata.tgz is in a script somewhere.  I imagine that this half of the the backup/restore process is probably the easiest to hack at.

[gerald_clark]

Do not load your SDK in your home directory.
Install it someplace else.
You can create your own extension containing your SDK, or create a tar file you load manually or from bootlocal.sh.

[hiro]

jerramy, you're right, try vi `which filetool.sh`.
But I wouldn't hack around in there, instead we create our own extensions, which is more flexible. You can read more about it on the web site.

[jerramy]

Yeah, I know I should bite the bullet, and make an extension for the SDK.  And I'm sure I eventually will.

This isn't the first time I've installed something, and thought, "Hey, how can I make this section load every time, without having to add it to mydata.tgz?"

And the answer is of course, to chug through the creation of an extension.  I just thought a method by which I only had to touch two files (the one I'm adding, and the one I'm adding the listing of it) would be a simpler way of taking care of such matters.

What is the simplest method to make an extension, in five commands or less?

[jerramy]

Sorry, I know I sound flippant.

The SDK kinda installs itself where it installs itself, with all its scripts dependant upon that install folder.  It wants you to create a special user, just for it to install into it's user's home.  I would have to change it's install script, which while doable, would break the current work instructions (for subsequent programmers at my position) to grab the originals from the repository and run them.  It's not that that's impossible, or even all that hard, except for the paperwork.

The amount of work to create an extension would be much less than to create a system that did what I want.  I recognise this.  It would however add the feature that different areas were updated automatically and independantly, reducing wear and tear on flash devices.

I'll look into how to do this myself (perhaps some shutdown script that autogenerates extensions), and report any positive findings.

I have another project involving getting CUDA development to work repeatedly in TCL, which has gone frustratingly.  I got it to work once, but I lost it on a reboot, and haven't managed to regain it yet...  It also has an install script (actually one huge file, half script, half encoded binary), that again, pretty much does whatever it wants.  There's three flavors, SUSE, Ubuntu, and RedHat.  I would have prefered Slackware or Debian, but you gets what you gets.  Ubuntu's has installed with various fiddling and modification to half baked instructions for various forum posts, though not permanently.
 

[curaga]

With big chunks of data like that, I usually just place them somewhere on the hd & create a symlink. The link is either stored in the backup, or created in bootlocal (if the device naming can change, it should be created by the label/uuid), and then you could have say /sdk as a path. With quite fast boots nonetheless.

This would work inside the home dir too.

[jerramy]

Hmmm.... so instead of creating a directory for the SDK, I create a symlink to a directory on the flashdrive.

My current problem with that is my flashdrive is FAT (so I can grab and drop files from Windows), but the SDK installs symlinks.  But I could make dual partitions on it, with one of them an EXT3 for support for symlinks.

Thanks Curaga, that's the best idea I've heard yet.

[maro]

I could be wrong, but to me it looks like instead of setting up one (or several) VM for your development task you make your life much more complicated then it needs to be. Depending on how confident you are of finding a solution down the road you appear to be travelling on now I'd like to suggest to revisit this alternative approach.

If you would describe your requirements and the exact issue that you were facing when you decided to abandon the VM approach I'd be surprised if myself (or someone else here) would not able to help you to overcome it. I'd like to think that I have a fair bit of experience with operating TC (and other guests) in VMs (mostly QEMU and VirtualBox) on Windows hosts and might be able to help.

[jerramy]

Hmmm...  My issue was that I couldn't get networking set up between Tiny Core and Windows 7, in VirtualPC2007, on a machine that had only one ethernet port.  You're probably right, that getting it working in a virtual machine will end up being possible (if perhaps a different VM).  And since the SDK's home directory will be backed up to mydata.tgz in the virtual drive file, which would be on a spinning hard disk, I won't have the issue of killing my flash (and writes will be faster).

I have another project I'm working on however, to have a NVIDIA CUDA development Tiny Core Linux, which needs to be on a flash drive.  I'm fairly sure that CUDA won't work in most any virtual machine, and really need to boot to Tiny Core directly.  I also want the ability to take my handy demonstration system and plug it into other peoples computers, to show them the ease of getting it up and running.  In this case, creating an extension will be necessary (if I can't get some alternate system set up).