Tiny Core Linux
Tiny Core Extensions => TCE Q&A Forum => Topic started by: pema on January 14, 2009, 10:24:20 AM
-
Gentlemen, some confusion about extensions.
I have used both extensions and often I find both extension types for same package. In wiki it is only mentioned that tce draws more ram. When do I need to have tce's ? When will tcz's not work ? Are the extensions once loaded backed up to my local .tgz and then removable ?
-
See http://www.tinycorelinux.com/getting_started.html (http://www.tinycorelinux.com/getting_started.html)
Basically extensions are store in a persistent partition not in your backup.
tcz are mounted and therefore easily unmounted and use considerably less ram.
Both types usually load at boot time, but can be "upon demand" (optional directory), or "installed" in a persistent partition or loop file (boot code local=hdXY).
Modularity, factorization of static content (extensions) from dynamic (user files), and quick booting are the concepts of Tiny Core.
-
I had a tce only setup and decided to try [EDIT] tcz only for a while in order to experience the benefits of low-RAM usage. However, the amount of memory used (according to "free" at any rate) seemed to be very similar, but every program took considerably longer to load when tcz's were used (and the system thus appeared sluggish in comparison to using tce's only).
I noted that the tcz file is mounted to /tmp/local.
Is that in tempfs or RAM as I imagine it is?
On the above basis I researched a bit on tempfs and though I'm still pondering the matter doubt that mounting tcz's onto any RAM filesystem ends up providing any RAM saving (the view supported by the actual performance of my system). For example the following link provides a short discussion which seems to also suggest that tempfs (and thus, I imagine, any method of mounting tcz's onto RAM) does not save RAM at all:
http://ozlabs.org/pipermail/linuxppc-embedded/2005-February/016885.html
I naturally then searched the tiny core forum to see if there was anyone else doubting the benefits of tcz in terms of their saving RAM and came across the following older thread in which user Curaga seems to suggest that tcz extensions do not in fact save much if anything in terms of RAM when mounted onto RAM (as they seem to be).
http://forum.tinycorelinux.net/index.php?&topic=231.0
I thus question that the default tiny core is using tcz's in a way that is as useful as suggested in Core Concepts, and wonder if Curaga's comment from the thread above doesn't point the way to how tcz's should better be used?
Using tcz is only ram-efficient when they are not in RAM themselves; if they are for example on a hd or usb stick. That's because they are mounted instead of extracted.
If I have this wrong I would be grateful if you could suggest some reliable means of measuring the advantages gained, since using "free" at least, reveals nothing advantageous (and the systems actual performance can't help but leave me doubtful about using tcz's).
-
It is up to the user how they want to run their system, and there will be differences depending on which extensions are used.
As for actual measurements... I haven't done any formal tests, but they seem to work for those who use them.
I think there's some confusion in your post. The main point is that you'd save the most amount of memory if you do not store tcz's in RAM. You can choose whether or not to do so.
Note that tce is a tgz but not a tcz.
[fixed post a bit]
-
Note that tce is a tgz but not a tcz.
oops... accidentally wrote tgz when I of course meant tcz. Hope my edits to original post have now fixed that.
It is up to the user how they want to run their system, and there will be differences depending on which extensions are used . . . The main point is that you'd save the most amount of memory if you do not store tcz's in RAM. You can choose whether or not to do so.
Yes, I accept that it is up to the user. However, my point is that TC "Core Concepts" appears to me to say that using tcz's instead of tce's simple will save RAM, when in practice, simply replacing tce's with tcz's doesn't seem to offer that advantage at all. If you simply populate your tce folder with tcz's and on grub kernel line have a tce=... option then there seems to be little (if any) RAM gain at all; without definite instructions I think many users may thus be mislead by the tcz "Core Concepts" description/claim.
Of course a user might use tclocal with tce's; in which case tce's will no longer be sitting in RAM but instead reside on a persistent location (so no advantage there in using tcz's either surely?)
So if Core Concepts doesn't indicate how to save RAM with tcz's, where is the possibility currently documented anywhere?
I presume one can loop mount a tcz from a persistent location, rather than it being mounted on [EDIT] /tmp/tcloop as occurs when using tcz's with grub kernel line option tce=... and not using tclocal.
[EDIT]
Perhaps I am simply stupid, but it seems to me that others continue to have inadequately answered misconceptions, brought about by the current TC documentation, regarding the advantages or otherwise of using tcz extensions, as in Kagashe's original post ("inadequate" purely in the sense of not being clarified in terms of tcz installation documentation):
http://forum.tinycorelinux.net/index.php?topic=231.0
-
It's essentially kind of like this. If you use tce with tclocal, or tcz's that reside on permanent storage the RAM use will be about the same. If you run in cloud mode, tcz will save some ram over the tce. Let's say the tce is 2mb compressed, but when installed into ram it is 5mb worth of files. So the tce takes up 5mb as the tce itself is deleted after loading or else it would take up 7mb total. The tcz is a little larger than the compressed tce, so it is about 2.3mb and sits in /tmp/tce. When mounted, it still takes up 2.3mb. So the ram savings are basically 5mb - 2.3mb which = 2.7mb RAM savings.
Each symlink in RAM is about 50 bytes, which is normally smaller than the file that it points to so RAM savings are there. Especially when the tcz resides on storage media.
I initially used PPI/tclocal mostly, but I have grown to like tcz's in that it affords ram savings coupled with rebooting to a known pristine state.
One thing about TC is it like Burger King, you can have it your way. And each way makes sense in different situations.
-
If you use tce with tclocal, or tcz's that reside on permanent storage the RAM use will be about the same.
I'm not using tclocal.
I do have my tcz's on permanent storage (currently in a folder I've called TClinux/tce on my /dev/hda4 partition as it happens, which appears as /mnt/hda4/TClinux/tce in the filesystem)
On my grub menu.lst kernel line I use: tce=hda4/TClinux/tce
I currently populate my /mnt/hda4/TClinux/tce/ directory with tcz's (having previously tried using tce's in there instead).
In that scenario (which seems to me to be the one suggested by Core Concepts as the way to use tgz's) I appear to gain no great RAM saving at all [EDIT: except that I accept that the tgz will be in compressed form, which saves a small amount of RAM but not a lot after you have actually run the application and its size appears to fill RAM and stay there even on closing the application]; everything generally just takes longer to load.
So what, precisely, is the recommended way to use tcz's?
-
Tcz's residing on storage media take up virtually no RAM by being loaded. That is the most efficient way to use tcz's.
And tcz's load much quicker than tce's with the advent of the tcz-symlinker. Tce's are slow to load in comparison.
-
Tcz's residing on storage media take up virtually no RAM by being loaded. That is the most efficient way to use tcz's.
I'm sorry, but I just don't get anything out of that answer.
I detailed how I am using tcz's in my last post, and I can assure you, the RAM saving on running the applicaions end up being negligible.
I believe the real problem is that the method of using PPR above, results in the tcz being mounted to tmpfs, so once you run the application it fills up memory pages (or is swapped out by tempfs if you are low on actual RAM) and stays there even when you close the app down. Certainly the symlinks take up virtually no space, but as soon as you run the application and then close it, the RAM used by the app pretty much remains.
So, if you don't mind detailing it, how do you usually set up your system to use tcz's in the "efficient way"?
-
One more time. As I said, runtime RAM is the same between extension types. It is the fact that the tcz does not store files in the system (RAM that is) but rather the files exist on storage. Tce/ppr all the files exist in RAM as well as the runtime RAM used. That is overall more RAM usage. Say 20mb runtime ram is used, the tce files take up 10mb in the system (RAM) but the tce is mounted on storage. The tcz takes virtually no RAM space on it's own aside from runtime. So tce takes up 30mb including runtime and the tcz takes up 20mb virtually all being runtime.
One more proof of concept, do a "du -h" on /tmp/tcloop with many tcz's loaded. Here is my reading on my 256mb RAM machine:
root@box:/home/tc# du -h /tmp/tcloop | tail
241.5K /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share/mplayer/skins/default
243.5K /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share/mplayer/skins
290.0K /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share/mplayer
4.0K /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share/pixmaps
298.5K /tmp/tcloop/MPlayer-svn-gtk2/usr/local/share
2.5K /tmp/tcloop/MPlayer-svn-gtk2/usr/local/tce.menu
4.1M /tmp/tcloop/MPlayer-svn-gtk2/usr/local
4.1M /tmp/tcloop/MPlayer-svn-gtk2/usr
4.1M /tmp/tcloop/MPlayer-svn-gtk2
277.0M /tmp/tcloop
root@box:/home/tc#
Whoa, let me repeat that. 277mb ram of system files on a 256mb ram machine. There is -21mb for the runtime of apps if I was using tce. Yet I am happily chugging along with this amount of system RAM used:
root@box:/home/tc# cat /proc/meminfo
MemTotal: 255512 kB
MemFree: 29968 kB
Buffers: 12092 kB
Cached: 146124 kB
SwapCached: 0 kB
Active: 42992 kB
Inactive: 64736 kB
Active(anon): 18436 kB
Inactive(anon): 27164 kB
Active(file): 24556 kB
Inactive(file): 37572 kB
Unevictable: 95960 kB
Mlocked: 0 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 255512 kB
LowFree: 29968 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 20 kB
Writeback: 0 kB
AnonPages: 45472 kB
Mapped: 20440 kB
Slab: 17884 kB
SReclaimable: 14480 kB
SUnreclaim: 3404 kB
PageTables: 356 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 127756 kB
Committed_AS: 125440 kB
VmallocTotal: 770040 kB
VmallocUsed: 1296 kB
VmallocChunk: 756680 kB
DirectMap4k: 4096 kB
DirectMap4M: 258048 kB
root@box:/home/tc#
In other words I have memory to spare when I have 277mb worth of files installed on a 256mb RAM machine. PPR/TCE would have locked my machine with the same apps. I am not knocking PPR/TCE but just showing how tcz saves ram.
-
One more consideration. If you are using a usb key to store your tcz's on, then they will by nature be slower to operate than tce's or tcz's that exist in RAM due to the slowness of the usb drive. If you have tce's and tcz's located on a usb drive then the tce's will operate faster since they load to RAM and not mounted to the usb key. Each method has it's advantages.
-
One more time. . . .
One more proof of concept, do a "du -h" on /tmp/tcloop with many tcz's loaded. Here is my reading on my 256mb RAM machine:
277.0M /tmp/tcloop
root@box:/home/tc#
Whoa, let me repeat that. 277mb ram of system files on a 256mb ram machine. There is -21mb for the runtime of apps if I was using tce. Yet I am happily chugging along with this amount of system RAM used:
Okay, I'll do some further tests myself. Your meminfo stats are interesting. The du -h figures don't necessarily mean so much: tmpfs uses swap if it runs out of RAM proper so it would keep "happily chugging along". The system would certainly freeze up if overloaded with tce's though.
-
mcewanw,
My defensive tone is due to a bad day at work (12hrs today). Nothing personal. But really, I think tcz;s are kind of neat and that is the normal mode I run in the last couple of months.
-
I also have low memory pc s and using tcz my system is slow (using 1.4, not tested 2.x).
PPI is good
-
Practical? Perhaps using oo2.tcel in a PPR vs oo2.tczl will demonstrate.
Don't even run it. Just compare load times and before and after 'free'
On some machines oo2.tcel is not an option whereas oo2.tczl is.
Try it with 256M, oo2.tcel crashes, oo2.tczl runs albeit slowly.
Perhaps there is confusion with PPR/tces together with tclocal. It was never intended that the two should meet. Why would one want to repeatedly 'install' the same static code into the same persistent store upon evey boot, especailly with flash or other limited write cycle devices? On ther otherhand tczs were never intended to run in cloiud mode. PPI being a hybrid scatter mode install is the fastest but has the same limitiations of a traditional installation. TC is all about choices. YMMV
-
On ther otherhand tczs were never intended to run in cloiud mode.
???!!!!!!!!!! but they're great for that! i mean sometimes they're a lot better... and not only because a tce extension in cloud mode is permanently installed until you use tce-uninstall or reboot, and in cloud mode when you reboot you have to download and reinstall all the tce's again. tcz's are trivial to unmount, delete and update (sometimes a tcz is updated twice in one day) in cloud mode. no potentially messy (actually it's a nice util) tce-uninstall's needed.
-
I'm running TCL on a Compaq box using PPR/TCE mode, booting off a read-only CD and saving to a USB stick. I've arrived at a set of apps that does what I want (Opera + Flash, XMMS (using ALSA), and epdfview). Thinking of trying out the TCZ mode. Would this have a big impact on performance since I'm using a USB stick? And would converting to TCZ just be a matter of deleting everything out of the TCE directory and downloading the TCZ versions of the apps/dependencies?
-
would converting to TCZ just be a matter of deleting everything out of the TCE directory and downloading the TCZ versions of the apps/dependencies?
oh i wouldn't do that. tcz's are often converted tce's, and although i love and use tcz when possible, it doesn't always work. wpa-supplicant.tcz still doesn't work for me, it closes with an error, but switching to the tce worked. you have the right idea, delete a tce and replace it with a tcz, but don't do all of them at once, unless you're only using a few of them. then again if you're feeling crazy, go for it. note that you can usually mix tce and tcz's, and many people do. also note that a few extensions do not exist in tcz.
-
Thanks for the advice. A few decades ago I WAS known as a wild and crazy guy, but now I'd have to think about it.
Something I'm not clear on with TCZ's. Are they also stored in the TCE directory and just mounted rather than being loaded to RAM, or do they end up somewhere else? If I try changing them over a few at a time I'll need to know how to keep track of where everything is.
-
Something I'm not clear on with TCZ's. Are they also stored in the TCE directory and just mounted rather than being loaded to RAM...
yes. it's possible (for you) to load them from other folders than the tce folder, but no one recommends it. (the exception being tce/optional, that's technically part of the tce folder.)
-
The warning from tobiaus was timely. Apparently there's no TCZ version of Flash, so I assume not only Flash but also its dependencies have to be tce? If that's the case then there's no point in using the TCZ version of epdfview since the dependencies are almost the same.
Opera, alsa, and xmms do have TCZ versions (and few dependencies) so I may try those.
-
I assume not only Flash but also its dependencies have to be tce? If that's the case then there's no point in using the TCZ version of epdfview since the dependencies are almost the same.
i think you misunderstood (or missed, but no worries) at least part of what i said. you should be able to mix tcz and tce, sometimes. what do i mean by this? two things:
1. that just because you're using some tcz extensions doesn't mean you can't use tce (and vice versa.)
2. that although you can't tell for example, ff's tce to get its deps in tcz, if you already have the deps (for example, gtk2) as tcz's, it will consider those deps "already installed" and use them.
i realize that's a mouthful, but i hope it makes sense. if the deps for flash are already installed by edfview's tcz, you won't need doubles of them as tce.
-
Thanks, that's the info I needed. I was under the impression that since Flash itself is TCE only, all its dependencies had to be too.
I'm using a USB stick for storage and it's stated below that may impact TCZ performance. Since I find I'm actually using Opera (with Flash off) and XMMS/ALSA all the time I plan to leave them as TCE for now. epdfview and Flash I use much less often so I'll try them (or in the case of Flash its dependencies) as TCZ.
-
Thanks, that's the info I needed. I was under the impression that since Flash itself is TCE only, all its dependencies had to be too.
nope :)
I'm using a USB stick for storage and it's stated below that may impact TCZ performance.
that's true it may, and i can tell you why but someone already did and you'll figure it out anyway. it's worth saying that just because it can affect performance doesn't mean it always will (noticeably.)
ram only holds so much, and i'm happier using tcz whenever possible, including on usb. my experience with ppr/tcz is that overall it's the best thing for everything i do, but i spend as much time using cloud mode, where i also find tcz helps even if it should/shouldn't. there are situations that people have mentioned, where tce is simply better. they exist! i've seen them, though for me they are rare. for those rare situations, and for testing new extensions, i too use tce. heed warnings, but use what suits you.
-
Thanks, that's the info I needed. I was under the impression that since Flash itself is TCE only, all its dependencies had to be too.
In practice, it is usually quite easy to make a tcz out of a tce. The wiki tells you how to make a tcz. I haven't done so for a few days but I think (without being able to double check at this second) the procedure goes like this for making a flashplayerX.tcz from a flashplayerX.tce (where X is the flashplayer version):
You probably need to install extensions advcomp and cramfs-utils for the following to work.
1. Make a temporary folder in which to extract your flashplayerX.tce contents. For example: mkdir pkg2. Put a copy of your flashplayerX.tce into folder pkg.
3. cd into folder pkg and extract flashplayerX.tce by entering: tar xvf flashplayerX.tce4. Delete the copy of flashplayerX.tce in folder pkg.
5. cd out of folder pkg by entering: cd ..6. You should now be able to create flashplayerX.tcz by simply entering: mkfs.cramfs pkg flashplayerX.tcz7. You can now move flashplayerX.tcz into your tce/optional folder (or put it in your main startup tce folder ready to go on next boot after removing your previous flashplayerX.tce version from there).
[I did in fact make a tcz version of flashplayer9 for myself some while back (after making the tce from an old libflashplayer.so I had lying about on my system) and it worked fine; just double check the tiny core wiki incase I've missed anything out. Note that the above assumes the cramfs method rather than the zisofs method].
-
The same method but a little simpler is to be in the same directory as your tce, and then:
mkdir pkg
tar xzvf extension.tce -C pkg
mkfs.cramfs pkg extension.tcz
-
I changed everything except Flash9 over to TCZ -- only took a few minutes since I didn't have many apps installed. Not seeing much obvious difference in performance or RAM usage, but it boots faster.
As far as making Flash9 a TCZ, when you say be in the same directory as your tce, I assume you mean (starting from root) to cd /mnt/sda1/tce and then run the commands?
-
Yes, in your case:
cd /mnt/sda1/tce
mkdir pkg
tar xzvf extension.tce -C pkg
mkfs.cramfs pkg extension.tcz
rm -r pkg
-
Well, it worked. Not all that sure it made much practical difference, but interesting.
The pkg directory with a number of sub-directories is still in the tce directory. I assume that's supposed to stay there? The libflashsupport.so and libflashplayer.so files are in there.
Also, is it necessary to keep the md5.txt files after download?
-
No, both can be removed.
-
how to remove it ?
there no guide to do it,
how to remove it if installed as PPR/TCE,PPR/TCZ or PPI/TCE ?
maybe all tce can be converted to tcz ,
is it possible ppr/tce converted to ppi/tce ?
-
tcz can be removed with no problems. this is because tcz is barely ever installed. (it installs almost nothing.) tce can be removed also. you would think it's a bad idea, but the mechanism that performs the uninstall is really exceptional. both have tools called tce-uninstall and tcz-uninstall, and sometimes you need them to remove a package. you just open the term, type "tce-uninstall" (or tcz for tcz) and the rest should be easy.
for ppr/tce and ppr/tcz removing a package is easier: delete the actual tce or tcz file from the tce folder and reboot. depending where its kept that may not remove everything (like flashplayer) that was not really part of a package, but downloaded by a download script (flashplayer is not technically part of any package.) you could try to treat that as a bug in tce-uninstall and see if flashplayer can be added, even though i believe tce-uninstall downloads .dep files or .list files, and neither file will list flashplayer itself.
-
I am getting interested in Linux after doing a project at school on operating systems. Some of my friends age getting interested too and we are looking into different distributions. Windows is sometimes described as "bloatware" and even Ubuntu, which I see is the most popular Linux, is quite big. That's what attracted me to Tinycore. I have been given an old laptop and would like to put Tinycore on and perhaps, when I've learned more, put Microcore on. I'd like to start learning about the command line too.
I have read through the topic on the difference between TCE & TCZ but I must be thick cos I haven't really understood. Could someone kind please explain in language suitable for kids, just what the difference is please and which I should use to put a few extra packages onto my laptop? Thanks :-[
-
tce format for extensions - as opposed to tcz - is deprecated since a while.
"TCE" is used as a general acronym for Tiny Core Extensions.
-
tce is a tarball but tcz is a compressed read-only file system (squashfs)
tce can only be extracted to root file system while tcz is mounted to /tmp/tcloop and symlinks created in root file system, tcz can also be copied to file system
root file system is a ram disk so extracting files to ram is not very good for low ram computers instead mounting extension from a persistent storage device (hard disk, usb memory etc.) is a better idea
tce is deprecated since TCL 2.7 and tcz is the unique extension type of TCL