[iso remaster] Storing TCE extensions for transport in TC v3.3

[goossbears]

Hi,
I downloaded, remastered, and burned to a 180mb+ miniCD, TinyCore v3.3 through the default /boot + /boot/isolinux hierarchy.

Within the 'Tiny Core Linux File Architecture' diagram at http://tinycorelinux.com/architecture.html, the middle 'Files on CDROM / HDD' teal box lists the files from the CD (in RAM when loaded) as follows:

/boot/bzImage
/boot/tinycore.gz
/boot/isolinux/  (for CD)
/tce/mydata.tgz
/tce/onboot.lst (list of extensions to auto load)
/tce/optional/screen.tcz (extension)
/tce/optional/rsync.tcz (extension)
.... Additional Extensions ....

I've already included on my remastered CD, everything before /tce/mydata.tgz

Am very unclear how I could download and keep as many .tcz extensions as I can fit on remastered and reburnt 180mb mini CDs  
Even with pics, wallpapers, pdf's, other doc files, ...etc., there is still 80mb+ wasted free space
Shall downloaded extensions be downloaded into a populated /tce directory sitting besides the default /boot on the remastered ISO?? Into solely a /tce/optional directory as the above Architecture diagram depicts??

According to the section 'The Mount Mode of Operation: TCE/Install' at http://tinycorelinux.com/concepts.html :

Tiny Core uses the Appbrowser to place application extensions in this tce/ and flag them as either "OnBoot" (mount at boot) or "On Demand" (do not mount at boot, but create a special menu section for easy access and display an icon if available).

I wish to download these application extensions onto a remastered ISO and burned CD before Appbrowser is ever invoked through TC!
There are some maintenance .tcz apps it would handy to always keep around without always having to rely on an Internet connection.

Any TCE Tips & Tricks pointers here or is there a better forum for this??
TYIA,
-goossbears

[Guy]

http://wiki.tinycorelinux.com/Integrating+Extensions

You can set up a cd basically the same as a normal installation, so it runs the same.

Be aware: The cd is read only, so you can't save any changes to it. If you have the tce directory on the cd, and boot using it, you cannot download additional extensions using the normal method, as the cd is read only. If you include everything you want, and set it up how you want it, that is not a problem.

I suggest including everything you would use to install Tiny Core on another computer, as it would be useful for that.

[goossbears]

If you have the tce directory on the cd, and boot using it, you cannot download additional extensions using the normal method, as the cd is read only. If you include everything you want, and set it up how you want it, that is not a problem.

Yep, set up as the /tce/optional directory on the CD with as many extensions saved here as possible for before&after burning.
Major problem is that with seemingly thousands! of possible extensions (a*.tcz - z*.tcz) to choose from, it is next to impossible to limit the final choice by including everything I could possibly want (and have it all fit on a single CD).

[tinypoodle]

There are several ways of remastering.

I have created an additional initrd with extensions in /opt/tce/optional according to instructions in wiki for dynamic remastering.

Neither is the boot medium needed any longer after boot - in my case that is harddisk on the network - but I don't think it would be of any difference if it would be a CD, as the extensions get extracted from the initrd to tmpfs like any other files on initrd.
I did not meet any issues to then use /tmp/tce as tce dir to install other extensions when needed.

Be aware that in this case, the size of total of extensions you add to the remaster will have an equivalent "cost" of RAM.

[Guy]

If you use a regular /tce directory on the cd, you cannot have another tce directory that you can use at the same time.

I think, if you use /opt/tce you can have a regular /tce directory on the hard drive or usb drive as well. So you can download additional applications.

Someone correct me if I am wrong.

[goossbears]

Thanks for replies and hints!
First of all, I see that this thread was moved from the TC Extensions Tips & Tricks section to this (current) General TC Talk section, along with an appended [iso remaster] in the subject line. Okay.

Second of all, I wish to re-iterate that my goal was (and is) to store TCE extensions for portable use.
Storage could be via a CD-R (a "mini" 180mb one in my case) or a USB drive for the future; not necessarily on hard disk drives.
Storage for such portability is through using TC 3.x TCE extensions in .tcz format .

Third of all, the remastering I wish to carry out is at the isolinux stage (for CD-R's), and not at the kernel-remastering or tinycore.gz unpacking/packing stages. In other words, for me, this would mean carrying out all steps mentioned at  http://wiki.tinycorelinux.com/Remastering up to the steps  "extract tinycore.gz for adding/removing something:" yet continuing with the "Creating an iso" section.
Using TC itself for this , I would probably at least download and store the cdrtools.tcz and mkisofs-tools.tcz extensions for CLI usage instead of using the suggested ISO Master GUI tool.

Fourth, tinypoodle wrote:

I have created an additional initrd with extensions in /opt/tce/optional according to instructions in wiki for dynamic remastering.

This wiki is at http://wiki.tinycorelinux.com/Dynamic+root+filesystem+remastering
No desire to use the EXTLINUX bootloader along with the "additional initrd with extensions in /opt/tce/optional".....there might be FAT-based filesystems requiring the SYSLINUX bootloader and don't want the RAM cost of being forced to using any given .tcz extensions that I've downloaded and that get compiled into a heavier initrd.
Besides, the ISOLINUX bootloader is typically used for "El Torrito" CD-Rs and the SYSLINUX bootloader the same for USB drives. ---- see my second point above.
One option that might make some sense here, though, is to create a top-level /syslinux directory for the remastery-level I'm attempting and download the latest syslinux tarball from http://www.kernel.org/pub/linux/utils/boot/syslinux/ into this /syslinux directory for future unpacking and usage. Yes, this would have a RAM "cost" penalty, but this does give me the helpful flexibility to extract and use whichever SYSLINUX, PXELINUX, ISOLINUX, and/or EXTLINUX lightweight bootloader(s) I would require for future 2nd-level remasterings.

Fifth, Guy wrote

if you use /opt/tce you can have a regular /tce directory on the hard drive or usb drive as well. So you can download additional applications

I have reviewed http://wiki.tinycorelinux.com/Integrating+Extensions , and Guy's suggestion would be ideal if correct. As this wiki directly states for the operation of adding extensions to a new ISO image into /tmp/newiso/tce/optional :

Doing this operation might prevent using a tce directory outside of the CD; if it is not explicitly specified, the boot process uses the first tce directory found.

Essentially, incorporating these two methods could enable the desired dual application-loading capability:
1. Maintaining the default RAM-loading /tce directory of the boot process for downloading+running .tcz extensions from the Internet repositories on an as-needed basis through Appbrowser, and with no further files necessarily added outside of the base. The boot process would first use this /tce directory.
2. Creating a top-level /opt/tce/optional directory which would store onto an ISO CD or USB only those .tcz extensions that could not be downloaded+run otherwise.
Yes,

An ISO created this way can not achieve application persistence for the user (no extensions can be loaded by the user, beyond re-loading those on the ISO)

And yet this second capability without persistence is precisely useful for my desired purposes of portability, e.g., for those times when an Internet connection remains unavailable.
I could then copy needed extensions on-the-fly from /opt/tce/optional into the default /tce/optional directory

In this latter case of 2 when remastering with a /opt/tce/optional directory, I could then store a much more limited subset of .tcz apps.
E.g. off the top-of-my-head,

• the previously mentioned cdrtools.tcz and mkisofs-tools.tcz
• the .tcz extensions for full bash and perl scripting capability
• tcz. extensions for other small and/or CLI-specific apps
• the module for my e100 and epro100 NICs and then insmod (or modprobe) either one when needed, rather than the non-loading default

Would need to purposefully omit browser and ftp app extensions under this concept.

Corrections or further feedback?
-goossbears

[tinypoodle]

You seem to confuse certain things...

I created an additional initrd.gz to be loaded, and that does basically what you describe at the end of your post, extract extensions into /opt/tce/optional, while I then could still download extensions to /tmp/tce/optional (which is the default location of TC for extensions).

I never mentioned anything about bootloaders.
Personally I use mostly linld to boot from FAT* or pxelinux to boot over net. With both I just specified both initrd's instead of one.

Not sure if you really need any modules not in base for your NICs, but firmware perhaps.

[goossbears]

Thanks for helping to clear up any confusion.
I think that I was unclear about my fifth point above.
Maybe this will help clear things up.

1. Pair of key TC websites

• the File Architecture Diagram, http://tinycorelinux.com/architecture.html
• the Concepts page, http://tinycorelinux.com/concepts.html

2. As you can see from these two key websites,

a. In the Mount Mode of Operation:

   * Applications are stored locally in a directory named tce on a persistent store, .e..g, a supported disk partition (ext2,ext3.ext4,vfat, or fat)
    * Applications are optionally mounted on reboot (see onboot.lst in forum and wiki).
    * Mounting applications saves RAM for other uses.
 applications are stored locally in a directory named /tce on a persistent store.  

Although stored .tcz extensions are only optionally mounted on reboot --- thus saving RAM for other uses -- still, this persistent extensions' storage mode using the default /tce verifiably requires a writable partition of type ext2,ext3.ext4,vfat, or fat.
Typical writable partitions include those ext2,ext3.ext4,vfat, or fat types found on hard-disk drives and on USB drives, but exclude the read-only partition found on iso9660-type CD-Rs.

b. In the Copy Mode of Operation, "Application Extensions are copied into RAM instead of mounted."
The "cost" of this extra RAM-copying thus rises as more and more application extensions are preloaded into RAM, whether these extensions are files on CDROM / HDD /tce directories or whether these are squashfs-compressed extension files residing in /usr/local.
In all cases here, the Copy Mode of Operation copies and merges these extension directories into the overall filesystem structure upon booting a Running Tiny Core System [tinycore.gz, initrd], as depicted in the File Architecture Diagram.

3. Well, I really don't need either of these two options; 2a or 2b above.
Instead, I prefer to store a subset of downloaded and unmounted .tcz extensions onto a read-only, iso9660-type CD-R (for now).
This specifically excludes the Mount Mode with its "Applications [that] are stored locally in a directory named tce on a persistent store".
I also wish to avoid any RAM-cost performance-hits from placing any downloaded .tcz application extensions into top-level /tce or squashfs-compressed /usr/local directories before remastering.
This specifically excludes the Copy Mode with its resulting action of all extensions "copied into RAM instead of mounted".

Maybe an extraneous tce directory excluded from a Running Tiny Core System's file structure is what I need for separately storing such .tcz's and copying them to the /opt/tce/optional or /tmp/tce/optional directories on the revised initrd once these latter directories are loaded in RAM (and yes, I did review the prior post!   )
Maybe something like a default-unmounted but populated /supertce that lies outside any directory listed in the File Architecture Diagram ??

-goossbears

[Guy]

Storage could be via a CD-R (a "mini" 180mb one in my case) or a USB drive

The simplest method is to use USB Install, to install on a USB drive. Then install extensions on the USB drive.

You can run Tiny Core from the USB drive.

The USB is writable, so you can install additional extensions.

You can use the USB drive to install Tiny Core on other computers.


If you want to use a CD. The simplest way, is as I said above:

http://wiki.tinycorelinux.com/Integrating+Extensions

After writing the CD, you cannot install additional extensions, or save changes.

You can write to other CDs, with additional extensions.

[Jason W]

If the tce directory is on the base of the cd image, then you can install extra extensions for the current session by using this command after booting:

sudo echo "/tmp/tce" > /opt/.tce_dir

But any installed extensions of course get lost on a reboot, but it does allow you to add more as needed for per session use.

A cd made this way does break some of the base functions regarding extensions, but I have found it is useful for special purposes. 

[tinypoodle]

goossbears,

Ok, now you are more clear.

What I described is basically what you mention under 2.a. with the difference that the extensions are extracted from the (additional) initrd on the cd into /opt/tce/optional which is on a tmpfs and therefore writable.
In this case a choice of extensions can be loaded onboot by a onboot.lst predefined in the remaster, otherwise the extensions can be mounted on demand from /opt/tce/optional.

However, reading your 3., I think what you want is to store extensions in an .iso but totally seperated from the system and selectively manually copy them on first need to the default tce dir (/tmp/tce), as this would not have any cost of RAM at all.

Search forum and wiki for such, the link Guy posted in upper post is one way, or you might even find a remastering script suitable. I guess you might even be able to avoid to remaster the initrd, but just remaster the .iso instead.

BTW, you might want to consider using CD-RW rather than CD-R, unitl you are sure you got exactly what you desire.