Tiny Core Linux

Tiny Core Base => TCB Talk => Topic started by: TraperJohn on May 22, 2009, 09:05:08 PM

Title: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 22, 2009, 09:05:08 PM
Hi I'm new to TC Linux and I have a number of questions.
First I come from Slax and I see that you have some posts about squashfs-lzma format and I was wondering if that works on TC yet?
Second I'd like to know if TC uses “linux-live-script” like Slax, Backtrack and Puppy do?
If the above are true then I can provide a wide number of scripts to make the most of them.
I know that other people may have asked this before but I didn't see any defined answer.
Title: Re: squashfs-lzma “linux-live-script”
Post by: roberts on May 22, 2009, 10:54:49 PM
No. We don't use the linux-live scripts from SLAX.
We also rejected squashfs-lzma as too big an impact on boot times, especially with older and/or embedded systems. We use the busybox/initramfs as the boot structure. As described at the busybox and embedded linux sites. Everything else is either tarred over or mounted & symlinked into place. We support ziofs/cramfs and soon, next release, squashfs. A very simple structure results in very fast boot times and with great extensibility.
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 23, 2009, 04:15:45 PM
OK simple solution and response but I believe that squashfs is already on the mainline kernel so we could use “.sfs” at least if we wonted.
I know that mounting squashfs-lzma files takes longer then tar, initramfs and ziofs/cramfs do but if we only used the  squashfs-lzma with the activation script off linux-live-script then boot time would not be affected only the individual application mount time would be.
I think that the benefit of having such a small size for applications/modules to be great enough to give it a try.
I'd definitely suffer for an additional 2 or 3 seconded per application of extra waiting time if I had even only 25% more application to choose from.
As for linux-live-script needed to use activation/deactivation I don't see it having that much affect on boot time, I have set it up under other distros before and its not even as complexed to build as the kernel patches for lzma,
So if anyone can get TC to use squashfs-lzma I will gladly build linux-live-script into a TC-core iso.
Title: Re: squashfs-lzma “linux-live-script”
Post by: Jason W on May 23, 2009, 05:34:32 PM
I built squashfs-lzma for TC 1.x, and it could be integrated into a person remaster to use squash-lzma extensions.  It is in the extension area.  Same as regular squashfs for TC 1.x, all one would need to do include the module in the iso and modprobe it in tc-config.  As stated, squashfs will be a supported tcz format along with cramfs and zisofs in TC 2.x since squashfs is in base TC 2.x.
However, also as stated, squashfs-lzma will not be a supported extension format as it incurs a performance loss that is not found with standard squashfs. 
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 23, 2009, 06:17:51 PM
As stated, squashfs-lzma will not be a supported extension format as it incurs a performance loss that is not found with standard squashfs.
Not supported, but will it work?
Title: Re: squashfs-lzma “linux-live-script”
Post by: Jason W on May 23, 2009, 06:21:23 PM
An extension could be made to be used on one's own system, simply by replacing the standard squashfs module with the squashfs-lzma one.  Then one can convert their persoanl collection of tcz's in to squashfs-lzma tcz's. 
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 23, 2009, 06:30:29 PM
simply by replacing the standard squashfs module with the squashfs-lzma one.
OK where do I find the standard squashfs.tcz module in the iso or do you refer to a kernel module?
Title: Re: squashfs-lzma “linux-live-script”
Post by: Jason W on May 23, 2009, 06:48:19 PM
squashfs-2.6.26.tcem is the kernel module and squashfs-tools.tce is the tools, they are both in the 1.x tce extension area for TC 1.x. 

squashfs-tools-4.0.tce is the tools for TC 2.x, and the kernel module is included in the base system.
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 23, 2009, 07:18:53 PM
So replace squashfs-tools-4.0.tce with the squashfs-lzma one, modprobe it in tc-config then start converting tcz's in to squashfs-lzma tcz's.
Sounds good.
Title: Re: squashfs-lzma “linux-live-script”
Post by: Jason W on May 23, 2009, 07:51:18 PM
Actually there is not a squashfs-lzma-tools.tce for TC 2.x, only for 1.x.  And the 1.x version will not work in 2.x. 
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 25, 2009, 12:56:09 PM
All right so how do we build one for TC 2.x?
Title: Re: squashfs-lzma “linux-live-script”
Post by: Jason W on May 25, 2009, 01:20:19 PM
Squashfs-lzma is not supported for squashfs 4.0, but when lzma makes it into the kernel there may be patches made for it:

http://www.nabble.com/squashfs-4.0-lzma-support-td22269948.html

Squashfs 4.0 is the version in the kernel and the one used by TC.  A squashfs-lzma 3.4 module could be built, but that would preclude the use of regular squashfs-4.0 tcz's, which is now a supported extension format, once that module is loaded. So it would be better not to have a squashfs-lzma extension in the download area.

Of course, even when or if lzma makes it into the mainline kernel gzip will still be the compression method preferred by TC.
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 25, 2009, 02:10:50 PM
Any other high compression algorithms that will work with squashfs 4.0 in a simullar fashen to lzma with 7z like results?
Title: Re: squashfs-lzma “linux-live-script”
Post by: jpeters on May 25, 2009, 04:44:12 PM
Interesting comparison study

http://odzangba.wordpress.com/2009/03/25/gzip-vs-bzip2-vs-lzma/

My vote is for speed, although bzip2 seems like a good compromise. lzma is in a league all it's own for
slowness.     
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 25, 2009, 04:59:20 PM
http://odzangba.wordpress.com/2009/03/25/gzip-vs-bzip2-vs-lzma/ is right but We wouldn't need to compress applications more then once as we only are mounting them with activate.
O and by the way in my own mount tests with zip to lzma I see only a small diff about 3-4 seconds on my last gen P4 system.

I found some 7zip compression algorithms written to work with gzip on sourceforge.net but 0 that exist in the kernel.
I'm not sure as to how to search this one out.
Any one know what to look for?
Title: Re: squashfs-lzma “linux-live-script”
Post by: curaga on May 26, 2009, 04:18:39 AM
The 7zip algorithm for gzip only affects compression, increasing both time and ratio (but still being about as fast as bzip2 to compress, and with no speed decrease decompressing). Because the stream is standard, 7zip compressed gzip streams can be decompressed with any compliant implementation.

Oh, and we already have such :) advdef in advcomp.tce can be used to recompress gzipped files with the 7zip algorithm for smaller files. It's used both for extensions and the base image.
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 26, 2009, 04:24:05 PM
I thought that lzma used 7zip compression with a slightly different format.
 ???Then I don't understand: how come lzma is not up to squashfs 4 yet if 7zip is ????
I guess that 7zip is the way to go 8).
Thanks.
Title: Re: squashfs-lzma “linux-live-script”
Post by: curaga on May 27, 2009, 04:26:56 AM
This is a bit confusing, I agree :)

7zip is a program, and a format. When talking about the program 7zip, they have a more efficient gzip algorithm there than in gzip. The 7zip format instead uses lzma compression.
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 27, 2009, 02:00:24 PM
O
Title: Re: squashfs-lzma “linux-live-script”
Post by: jpeters on May 28, 2009, 01:25:37 AM
What is tc using for its backup.tgz ?  I manually tried cpio and tar ...neither worked.   
Title: Re: squashfs-lzma “linux-live-script”
Post by: curaga on May 28, 2009, 04:10:45 AM
backup.tgz is a regular gzipped tarball, perhaps yours is corrupted?
Title: Re: squashfs-lzma “linux-live-script”
Post by: jpeters on May 28, 2009, 08:27:25 AM
backup.tgz is a regular gzipped tarball, perhaps yours is corrupted?

It unpacked okay with tar, but after repacking with 'tar -czf', the file would disappear when booting and load norestore,  I did this several times, with the same result, and had to add files and backup from within tc for it to work.   Also, when I moved  a 'myfiles.tgz' it created a circular dir structure (link to ".") that appears to be impossible to delete.  

edit: I'm thinking maybe there's an issue with the file or file location being in use, since I was also getting "Stale NFS file handle" issues when simply trying to move "tinycore.tgz.bak" to "tinycore.tgz"  I think I had to do something like first move it to another folder,  
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on May 31, 2009, 04:20:39 PM
I got back from vacation ;D and back to work :-\.
After are discussion I mounted the tinycore iso, decompressed the tinycore.gz and expanded the CPIO'ed tinycore.
Now I see the typical linux fs but where is the tce, tcz and other images I expected ????
I thought TC used compressed  images in tce, tcz formats on the iso.
Don't get me wrong I am used to one big fs to edit like in the core lzm of Slax ;) but when you were talking about replacing applications you noted the  tce, tcz extensions.
 ???So now I'd like to know what you were talking about?
Title: Re: squashfs-lzma “linux-live-script”
Post by: jpeters on May 31, 2009, 06:40:38 PM
Extensions are kept in a separate folder that can be pointed to in  boot options  (eg, tce=/hdax/tcz).  That way, they'll load even if you change to a an updated boot folder.  Check out the intro help docs.  
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on June 01, 2009, 11:55:29 AM
But are the applications installed in to the fs or mountable during boot from tce...?
If they are the latter then I'd expect to see some of them in the iso already.
Title: Re: squashfs-lzma “linux-live-script”
Post by: jpeters on June 01, 2009, 12:06:00 PM
But are the applications installed in to the fs or mountable during boot from tce...?
If they are the latter then I'd expect to see some of them in the iso already.
The iso contains grub, the kernel and pristine setup that can be rebooted at any time using
the "norestore, base" bootoptions. All the rest is modular, using extensions in a separate folder.    
See wiki, remaster, for info regarding opening tinycore.gz.
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on June 01, 2009, 07:18:13 PM
What's the name of the separate folder?
Title: Re: squashfs-lzma “linux-live-script”
Post by: jpeters on June 02, 2009, 12:31:48 AM
your choice (see my previous post) . Please read the intro help docs  :)
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on June 02, 2009, 07:45:20 PM
The iso contains
boot with sub directories isolinux, bzImage (the kernel), and tinycore.gz (the compressed CPIOed fs/linux-stuff).

boot options...
Quote
"norestore, base" bootoptions.

Quote
All the rest is modular, using extensions in a separate folder.
???separate folder??
Title: Re: squashfs-lzma “linux-live-script”
Post by: jpeters on June 02, 2009, 10:12:35 PM
http://www.tinycorelinux.com/concepts.html
Title: Re: squashfs-lzma “linux-live-script”
Post by: TraperJohn on June 03, 2009, 02:30:10 PM
What I have to guess is that the iso has no extra folder but rather adds everything from mirror, or your own specified directories to the fs by mounting, extracting, or installing based on what you pick at boot.
So what I need to edit is either in the CPIO/compressed fs or on the mirror.

Em I wright?