What is the practical difference in tce vs tcz ?

[pema]

Gentlemen, some confusion about extensions.
I have used both extensions and often I find both extension types for same package. In wiki it is only mentioned that tce draws more ram. When do I need to have tce's ? When will tcz's not work ? Are the extensions once loaded backed up to my local .tgz and then removable ?

[roberts]

See http://www.tinycorelinux.com/getting_started.html
Basically extensions are store in a persistent partition not in your backup.
tcz are mounted and therefore easily unmounted and use considerably less ram.
Both types usually load at boot time, but can be "upon demand" (optional directory), or "installed" in a persistent partition or loop file (boot code local=hdXY).
Modularity, factorization of static content (extensions) from dynamic (user files), and quick booting are the concepts of Tiny Core.

[mcewanw]

I had a tce only setup and decided to try [EDIT] tcz only for a while in order to experience the benefits of low-RAM usage. However, the amount of memory used (according to "free" at any rate) seemed to be very similar, but every program took considerably longer to load when tcz's were used (and the system thus appeared sluggish in comparison to using tce's only).

I noted that the tcz file is mounted to /tmp/local.

Is that in tempfs or RAM as I imagine it is?

On the above basis I researched a bit on tempfs and though I'm still pondering the matter doubt that mounting tcz's onto any RAM filesystem ends up providing any RAM saving (the view supported by the actual performance of my system). For example the following link provides a short discussion which seems to also suggest that tempfs (and thus, I imagine, any method of mounting tcz's onto RAM) does not save RAM at all:

http://ozlabs.org/pipermail/linuxppc-embedded/2005-February/016885.html

I naturally then searched the tiny core forum to see if there was anyone else doubting the benefits of tcz in terms of their saving RAM and came across the following older thread in which user Curaga seems to suggest that tcz extensions do not in fact save much if anything in terms of RAM when mounted onto RAM (as they seem to be).

http://forum.tinycorelinux.net/index.php?&topic=231.0

I thus question that the default tiny core is using tcz's in a way that is as useful as suggested in Core Concepts, and wonder if Curaga's comment from the thread above doesn't point the way to how tcz's should better be used?

Using tcz is only ram-efficient when they are not in RAM themselves; if they are for example on a hd or usb stick. That's because they are mounted instead of extracted.

If I have this wrong I would be grateful if you could suggest some reliable means of measuring the advantages gained, since using "free" at least, reveals nothing advantageous (and the systems actual performance can't help but leave me doubtful about using tcz's).

[^thehatsrule^]

It is up to the user how they want to run their system, and there will be differences depending on which extensions are used.

As for actual measurements... I haven't done any formal tests, but they seem to work for those who use them.

I think there's some confusion in your post.  The main point is that you'd save the most amount of memory if you do not store tcz's in RAM.  You can choose whether or not to do so.


Note that tce is a tgz but not a tcz.
[fixed post a bit]

[mcewanw]

Note that tce is a tgz but not a tcz.

oops... accidentally wrote tgz when I of course meant tcz. Hope my edits to original post have now fixed that.

It is up to the user how they want to run their system, and there will be differences depending on which extensions are used . . . The main point is that you'd save the most amount of memory if you do not store tcz's in RAM.  You can choose whether or not to do so.

Yes, I accept that it is up to the user. However, my point is that TC "Core Concepts" appears to me to say that using tcz's instead of tce's simple will save RAM, when in practice, simply replacing tce's with tcz's doesn't seem to offer that advantage at all. If you simply populate your tce folder with tcz's and on grub kernel line have a tce=... option then there seems to be little (if any) RAM gain at all; without definite instructions I think many users may thus be mislead by the tcz "Core Concepts" description/claim.

Of course a user might use tclocal with tce's; in which case tce's will no longer be sitting in RAM but instead reside on a persistent location (so no advantage there in using tcz's either surely?)

So if Core Concepts doesn't indicate how to save RAM with tcz's, where is the possibility currently documented anywhere?

I presume one can loop mount a tcz from a persistent location, rather than it being mounted on [EDIT] /tmp/tcloop as occurs when using tcz's with grub kernel line option tce=... and not using tclocal.

[EDIT]
Perhaps I am simply stupid, but it seems to me that others continue to have inadequately answered misconceptions, brought about by the current TC documentation, regarding the advantages or otherwise of using tcz extensions, as in Kagashe's original post ("inadequate" purely in the sense of not being clarified in terms of tcz installation documentation):

http://forum.tinycorelinux.net/index.php?topic=231.0

[Jason W]

It's essentially kind of like this.  If you use tce with tclocal, or tcz's that reside on permanent storage the RAM use will be about the same.  If you run in cloud mode, tcz will save some ram over the tce.  Let's say the tce is 2mb compressed, but when installed into ram it is 5mb worth of files.  So the tce takes up 5mb as the tce itself is deleted after loading or else it would take up 7mb total.  The tcz is a little larger than the compressed tce, so it is about 2.3mb and sits in /tmp/tce.  When mounted, it still takes up 2.3mb.  So the ram savings are basically 5mb - 2.3mb which = 2.7mb RAM savings. 

Each symlink in RAM is about 50 bytes, which is normally smaller than the file that it points to so RAM savings are there.  Especially when the tcz resides on storage media. 

I initially used PPI/tclocal mostly, but I have grown to like tcz's in that it affords ram savings coupled with rebooting to a known pristine state. 

One thing about TC is it like Burger King, you can have it your way.  And each way makes sense in different situations. 

[mcewanw]

If you use tce with tclocal, or tcz's that reside on permanent storage the RAM use will be about the same.

I'm not using tclocal.

I do have my tcz's on permanent storage (currently in a folder I've called TClinux/tce on my /dev/hda4 partition as it happens, which appears as /mnt/hda4/TClinux/tce in the filesystem)

On my grub menu.lst kernel line I use: tce=hda4/TClinux/tce

I currently populate my /mnt/hda4/TClinux/tce/ directory with tcz's (having previously tried using tce's in there instead).

In that scenario (which seems to me to be the one suggested by Core Concepts as the way to use tgz's) I appear to gain no great RAM saving at all [EDIT: except that I accept that the tgz will be in compressed form, which saves a small amount of RAM but not a lot after you have actually run the application and its size appears to fill RAM and stay there even on closing the application]; everything generally just takes longer to load.

So what, precisely, is the recommended way to use tcz's?

[Jason W]

Tcz's residing on storage media take up virtually no RAM by being loaded.  That is the most efficient way to use tcz's.

And tcz's load much quicker than tce's with the advent of the tcz-symlinker.  Tce's are slow to load in comparison.

[mcewanw]

Tcz's residing on storage media take up virtually no RAM by being loaded.  That is the most efficient way to use tcz's.

I'm sorry, but I just don't get anything out of that answer.

I detailed how I am using tcz's in my last post, and I can assure you, the RAM saving on running the applicaions end up being negligible.

I believe the real problem is that the method of using PPR above, results in the tcz being mounted to tmpfs, so once you run the application it fills up memory pages (or is swapped out by tempfs if you are low on actual RAM) and stays there even when you close the app down. Certainly the symlinks take up virtually no space, but as soon as you run the application and then close it, the RAM used by the app pretty much remains.

So, if you don't mind detailing it, how do you usually set up your system to use tcz's in the "efficient way"?

[Jason W]

One more time.  As I said, runtime RAM is the same between extension types.  It is the fact that the tcz does not store files in the system (RAM that is) but rather the files exist on storage.  Tce/ppr all the files exist in RAM as well as the runtime RAM used.  That is overall more RAM usage.  Say 20mb runtime ram is used, the tce files take up 10mb in the system (RAM) but the tce is mounted on storage.  The tcz takes virtually no RAM space on it's own aside from runtime.  So tce takes up 30mb including runtime and the tcz takes up 20mb virtually all being runtime. 

One more proof of concept, do a "du -h" on /tmp/tcloop with many tcz's loaded.  Here is my reading on my 256mb RAM machine:

Code: [Select]

root@box:/home/tc# du -h /tmp/tcloop | tail
241.5K  /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share/mplayer/skins/default
243.5K  /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share/mplayer/skins
290.0K  /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share/mplayer
4.0K    /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share/pixmaps
298.5K  /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share
2.5K    /tmp/tcloop/MPlayer-svn-gtk2/usr/local/tce.menu
4.1M    /tmp/tcloop/MPlayer-svn-gtk2/usr/local
4.1M    /tmp/tcloop/MPlayer-svn-gtk2/usr
4.1M    /tmp/tcloop/MPlayer-svn-gtk2
277.0M  /tmp/tcloop
root@box:/home/tc#

Whoa, let me repeat that.  277mb ram of system files on a 256mb ram machine.  There is -21mb for the runtime of apps if I was using tce.  Yet I am happily chugging along with this amount of system RAM used:

Code: [Select]

root@box:/home/tc# cat /proc/meminfo
MemTotal:         255512 kB
MemFree:           29968 kB
Buffers:           12092 kB
Cached:           146124 kB
SwapCached:            0 kB
Active:            42992 kB
Inactive:          64736 kB
Active(anon):      18436 kB
Inactive(anon):    27164 kB
Active(file):      24556 kB
Inactive(file):    37572 kB
Unevictable:       95960 kB
Mlocked:               0 kB
HighTotal:             0 kB
HighFree:              0 kB
LowTotal:         255512 kB
LowFree:           29968 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                20 kB
Writeback:             0 kB
AnonPages:         45472 kB
Mapped:            20440 kB
Slab:              17884 kB
SReclaimable:      14480 kB
SUnreclaim:         3404 kB
PageTables:          356 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      127756 kB
Committed_AS:     125440 kB
VmallocTotal:     770040 kB
VmallocUsed:        1296 kB
VmallocChunk:     756680 kB
DirectMap4k:        4096 kB
DirectMap4M:      258048 kB
root@box:/home/tc#

In other words I have memory to spare when I have 277mb worth of files installed on a 256mb RAM machine.  PPR/TCE would have locked my machine with the same apps.  I am not knocking PPR/TCE but just showing how tcz saves ram.

[Jason W]

One more consideration.  If you are using a usb key to store your tcz's on, then they will by nature be slower to operate than tce's or tcz's that exist in RAM due to the slowness of the usb drive.  If you have tce's and tcz's located on a usb drive then the tce's will operate faster since they load to  RAM and not mounted to the usb key.  Each method has it's advantages. 

[mcewanw]

One more time.  . . .

One more proof of concept, do a "du -h" on /tmp/tcloop with many tcz's loaded.  Here is my reading on my 256mb RAM machine:

Code: [Select]

277.0M  /tmp/tcloop
root@box:/home/tc#

Whoa, let me repeat that.  277mb ram of system files on a 256mb ram machine.  There is -21mb for the runtime of apps if I was using tce.  Yet I am happily chugging along with this amount of system RAM used:

Okay, I'll do some further tests myself. Your meminfo stats are interesting. The du -h figures don't necessarily mean so much: tmpfs uses swap if it runs out of RAM proper so it would keep "happily chugging along". The system would certainly freeze up if overloaded with tce's though.

[Jason W]

mcewanw,
My defensive tone is due to a bad day at work (12hrs today).  Nothing personal.  But really, I think tcz;s are kind of neat and that is the normal mode I run in the last couple of months.

[jls]

I also have low memory pc s and using tcz my system is slow (using 1.4, not tested 2.x).
PPI is good

[roberts]

Practical? Perhaps using oo2.tcel in a PPR vs oo2.tczl will demonstrate.
Don't even run it. Just compare load times and before and after 'free'
On some machines oo2.tcel is not an option whereas oo2.tczl is.
Try it with 256M, oo2.tcel crashes, oo2.tczl runs albeit slowly.

Perhaps there is confusion with PPR/tces together with tclocal. It was never intended that the two should meet. Why would one want to repeatedly 'install' the same static code into the same persistent store upon evey boot, especailly with flash or other limited write cycle devices? On ther otherhand tczs were never intended to run in cloiud mode. PPI being a hybrid scatter mode install is the fastest but has the same limitiations of a traditional installation. TC is all about choices. YMMV