Why do we need both .tce and .tcz ?

[samedirection]

This is a slight expansion of this thread, but I think it's relevant to the discussion.  I'd be very glad for a flexible mechanism for loading different collections of extensions at boot time.  Such a mechanism could also be made to specify which extensions get loaded to ramdisk and which get mounted as a loop file.  This could kill multiple birds with one stone and really help those who use their tinycore installs for different purposes, or on different machines with different memory amounts and hardware requirements.

Imagine a plain text config file in which you can list 'modules'---user created collections of extensions. You can have any number of such modules, and each can have any number of .tcz's (assuming that we settle on a single extension type each of which can be either be loaded to ramdis or mounted).  For each tcz, you also optionally specify if you want it loaded to ramdisk.  By default it gets mounted and symlinked tcz-style.

For example, I have one setup on a low ram computer where I need everything possible mounted as a loop file, but don't need development files or my gtk2 stuff.  I'll call this 'skinny.'

Sometimes I'm on a newer machine. If I find that it speeds things up, I'd be glad to load more core functionality to RAM. So I have my basic set of modules, including the gtk2 stuf, with some critical parts loaded to ramdisk.  I call this 'fat'.

In either case I might want some functionality which I rarely use, provided by extension X and its dependencies.  I set up a new module with those extensions, call it wizbang.  And another collection, with, say the wireless drivers, programs and utilities, which I call 'wireless.'  I also would have modules called 'devel+x' 'mediaplayer' etc.
   
Then at boot time, I can specify any number of 'modules' to load with a boot option that takes a comma-separated list

tce=sda1 modules=fat,wireless,wizbang,mediaplayer  OR: modules=skinny,somemodule

Later, if I discover I need a module that I didn't load at boot time, that should be possible as well.  If I need a group of extensions I haven't yet set up, I can edit the modules configuration file and create the module.  Then load it.  Ideally of course there should be a script that checks for dependencies.  Even more ideally it would let me download dependencies  if a connection is available and prompt me to add them to my module-configuration file, or let me force-ignore them if I know what I'm doing.

This should allow all extensions to be kept in a single directory (even the optional ones), and be of a single type.  It could allow on-demand use of pre-defined groups of extensions (keeping memory free if you don't need those extensions).  It would also minimize the current difficulties associated with keeping different configurations available.  Currently I either keep a copy of an extension for each configuration I want (easier, since dependencies are managed, but wastes bandwidth and it's harder to keep multiple configurations up-to-date), or else I create symliked collections from a central repository.  I have to manage dependencies manually, which is error-prone.

Thanks for entertaining this rather long-winded idea.  It would have some issues to be worked out, but it could be a huge leap forward in usability and flexiblity, and simplify extension provision as well.

[roberts]

I already provide the ability to have named tce directories.
You can boot with tce=dev for a development environment, or tce=offrice whereas the office directory contains tce and tcz for office extensions, or tce=games, etc.

[bigpcman]

I already provide the ability to have named tce directories.
You can boot with tce=dev for a development environment, or tce=offrice whereas the office directory contains tce and tcz for office extensions, or tce=games, etc.

I would also like to note that it is possible to use your own custom boot codes. This was discussed way back in January. http://forum.tinycorelinux.net/index.php?topic=460.msg2914#msg2914

I do like the idea of user control  "to specify which extensions get loaded to ramdisk and which get mounted as a loop file".

[tclfan]

If I can voice my opinion as end user, I strongly vote for one repository, which means tcz only. This is to maximize efficiency and memory utilization as well as streamlining maintenance. This makes it also simpler for end user, who does not need to understand the difference to make a choice.
My opinion brings nothing new to this discussion, I am just emphasizing what has been said already and just want to express my end user perspective.

[Lee]

I like the idea of a single extension type that could easily be either loaded or mounted.   Perhaps an extra button on appbrowser - one for "load selected" and one for "mount selected"?   (And equivalent command line operation, of course.)

From an extension builders point of view, my own (very limited) experience has been:

1) I put together the gtm5.tce extension
2) There's no reason (that I know of) why it couldn't be a .tcz as well but
3) I just never got around to making a .tcz out of it.

That's lack of time (or just laziness) on my part.  I'm sure its not hard to make a .tcz extension - its just that I've never done it and now it has been so long since I even read about it that it will take me a while when I do.  With a single extension type, I'd be done and wouldn't be worrying about it.


As for the idea of "modules" or sets of extensions, that can be done with a pseudo-extension listing the extensions to be included as deps, though I suppose there's plenty of room for increased ease of use and/or other improvements.  And isn't this roughly (really roughly) like what runlevels do?


Once the tce directory has been set at boot time (either by scanning or by an explicit "tce=whatever" boot code), is there a convenient way to change it?

[Jason W]

There are a couple of issues related to the possibility of a 'copy to the system' use of the tcz.  One is that the user.tar.gz is being deprecated to using the startup scripts.to copy config files.  The user.tar.gz was used to hold writable config files and others that must exist as real files in the system.  But by definition the user.tar.gz overwrote any files in it's way, which is not a good idea for config files that already exist in the system.  If you have a config file in your backup, and you have not yet loaded the extension the file belongs to, that backed up file will be clobbered when you load the extension.  Here is a way that will help avoid that by use of the startup script.  Say you have an extension extension.tcz and a config file /usr/local/etc/extension.conf that is part of it, as well as /home/$USER/extension.txt.  I would place the config files under /usr/local/share/extension and copy it to /usr/local/etc/ using a test case:

Code: [Select]

#!/bin/sh

TCUSER=`cat /etc/sysconfig/tcuser`

[ -f /usr/local/etc/extension.conf ] || cp /tmp/tcloop/extension/usr/local/share/extension/extension.conf /usr/local/etc/

[ -f /home/"$TCUSER"/extension.txt ] || cp /tmp/tcloop/extension/usr/local/share/extension/extension.txt /home/"$TCUSER"/



This would prevent clobbering of config files and allow the possible 'copy to system' use of tcz's if that is implemented.  This works as well with tce's to prevent the clobbering of writable configs, of course then you would be copying from /usr/local/share/extension.  If you are installing _anything_ to the /home directory, it is a must to use the TCUSER variable to allow the script to work both during boot and runtime loading, as well as not to clobber files in the /.home directory.

[roberts]

Don't expect the tce repository to go away in the 2.x series, as that would break eariler 2.x versions.
However, I will be introducing new options to "test the waters" in next RC of 2.3.

Please take heed to Jason's advice, as many tczs will likely need to be adjusted during this phase.

[samedirection]

I already provide the ability to have named tce directories.
You can boot with tce=dev for a development environment, or tce=offrice whereas the office directory contains tce and tcz for office extensions, or tce=games, etc.

I like that feature and use it a lot. I was just musing about a way to expand that idea which might be easier for the user to _maintain_ multiple configurations, and which might also work for modular collections of apps and dependencies which could be combined at boot time or loaded after boot.  Presently you have to treat each of your 'tce=xxxxyyy' setups as separate systems and maintain them separately.  (Either you keep your extensions in one place and symlink to your 'office and 'dev' directories or you just maintain and update separate systems.  The latter is really easier, since you don't have to manage dependencies manually, but it uses more drive space and bandwith).  I thought if we could use something else besides the filesystem to group and specify such configurations we might get more flexibility (including the flexibility to specify whether to load a file to the ramdisk or to loop mount it).     

Thanks Jason for your helpful summary.

[jpeters]

There are a couple of issues related to the possibility of a 'copy to the system' use of the tcz.  One is that the user.tar.gz is being deprecated to using the startup scripts.to copy config files.  The user.tar.gz was used to hold writable config files and others that must exist as real files in the system.  But by definition the user.tar.gz overwrote any files in it's way, which is not a good idea for config files that already exist in the system.  If you have a config file in your backup, and you have not yet loaded the extension the file belongs to, that backed up file will be clobbered when you load the extension.  Here is a way that will help avoid that by use of the startup script.  Say you have an extension extension.tcz and a config file /usr/local/etc/extension.conf that is part of it, as well as /home/$USER/extension.txt.  I would place the config files under /usr/local/share/extension and copy it to /usr/local/etc/ using a test case:

Code: [Select]

#!/bin/sh

TCUSER=`cat /etc/sysconfig/tcuser`

[ -f /usr/local/etc/extension.conf ] || cp /tmp/tcloop/extension/usr/local/share/extension/extension.conf /usr/local/etc/

[ -f /home/"$TCUSER"/extension.txt ] || cp /tmp/tcloop/extension/usr/local/share/extension/extension.txt /home/"$TCUSER"/



This would prevent clobbering of config files and allow the possible 'copy to system' use of tcz's if that is implemented.  This works as well with tce's to prevent the clobbering of writable configs, of course then you would be copying from /usr/local/share/extension.  If you are installing _anything_ to the /home directory, it is a must to use the TCUSER variable to allow the script to work both during boot and runtime loading, as well as not to clobber files in the /.home directory.

It seems that restore will overwrite files installed by the tcz extension, so if the extension loads the config file to /home/tc, I can overwrite it and it will remain after rebooting.

[Jason W]

Restoring from backup will overwrite any existing file.  But loading the tcz will not overwrite an already restored file (or simply a preexisting file) with a new copy using the above approach, and that is the desired behavior.

[jpeters]

Restoring from backup will overwrite any existing file.  But loading the tcz will not overwrite an already restored file (or simply a preexisting file) with a new copy using the above approach, and that is the desired behavior.

That's probably a cleaner approach.  My point was that restore seems to occur after tcz loads, so the end result should be the same.  (ie, the loaded tcz config file gets overwritten with the desired copy saved in backup).