Can it boot faster? Let's try :)

[pioj]

Ok. I just learned to unpack, modify, and repack .tcz extensions this morning.

So, I'm trying to figure how could I speed some things a bit more. I have some more questions to ask:


1. Which approach do you guys like most?    (on a common modern machine, let's say 4GB RAM)

  A) SquashFS with -comp lzo.   LZO extensions will be bigger than gzip in size, but they will uncompress faster than gzip at boot time.

  B) SquashFS with -comp xz.   XZ extensions are smaller in size and they will load faster to RAM before they have to uncompress.


2. Are the  extensions first copied into Ram then uncrompressed into it, or are just directly uncompressed? 

3. Wouldn't be faster to pack some related extensions (iex. Xorg) into one big .tcz if you're going to load the whole extension anyway?



Note: I'm not talking about submitting anything or chaning the way the Repos are.   I'm just colleting some opinions...

[pioj]

Ah I forgot..

Is it mandatory to use '-b 4096' when packing to tcz?  I found this to result in a bigger file...

[curaga]

For the first question, XZ squashfs is not supported in our current kernel config.

2. Are the  extensions first copied into Ram then uncrompressed into it, or are just directly uncompressed?

They aren't uncompressed at all, unless you choose so (via copy2fs). If they are, it happens directly from the storage device, without a copy to RAM first.

3. Wouldn't be faster to pack some related extensions (iex. Xorg) into one big .tcz if you're going to load the whole extension anyway?

Yes, it would. That's why we used to have some big packs (compiletc and Xorg), though it's much better for maintenance to have modularity.

Is it mandatory to use '-b 4096' when packing to tcz?  I found this to result in a bigger file...

Yes, as it greatly decreases RAM usage. For your own use, you can of course use whatever you like; but 4k for the repo.

[pioj]

Thx a lot , curaga.

Yes, as it greatly decreases RAM usage. For your own use, you can of course use whatever you like; but 4k for the repo.

I now understand what it does, thx. I'll be testing a lot with block sizes, so I may post my results on a separate thread

[pioj]

Are the extension files compressed using UPX? 

I know by experience this tool works great with executables and libraries.

[Rich]

Hi pioj

Are the extension files compressed using UPX?

No, they are squashed into the extension. See:
http://wiki.tinycorelinux.net/wiki:creating_extensions#creating_a_tcz

[tinypoodle]

You can check that yourself:

Code: [Select]

upx -t file
I never noted any.

Are you sure about libraries? That would be news to me...

[pioj]

It works for exes and dll at Windows, but I'm afraid it only works for executables on Linux.

Anyways, we have a good example in Xorg-7.6.tcz, which has up to five different executables, 1 MB each. (Xnest, Xephyr, Xdmx, Xvfb, Xorg and Xfbdev).

I'll try to do some tests with them. Who knows...

[tinypoodle]

I'm afraid it only works for executables on Linux.

plus kernel

[neonix]

You should guys start pack your executables in tczs with "upx -9". TC is getting bigger every year. You can also compile some programs with static libraries that will be compressed both with executables.

kernel could be compuked with uncomoressed core.gz and then upx with biggest compression should be used.
I am using mobile internet with limited megabites. Installing new version is still fast becouse TC is not bloatware.

[hiro]

upx is using some kind of lzma.
i disagree with your goal. compression and decompression with upx -9 is way too slow.
also, we would be compressing twice, that's not a good idea.

[curaga]

Upx comes with a delay on startup (the decompression), and it makes it harder to analyze the binary (readelf, etc). It may not make sense for the repo, but if you have a specific deployment where the size benefit is worth it, it's quick for you to script-loop over your extensions.

[neonix]

I'm not sure (I read source code of this program many years ago) but upx use lzo when you use -1 argument, gzip with -9 argument, and lzma/lzma2 with --ultra-brute argument.

On old x86_32 processors lzma may not work good, but gzip will always work and decompression takes miliseconds or 1 second.
On x86_64 all processors can handle lzma. Decompression time is less than 1 second, 64-bit programs are bigger than 32-bit programs.
There are more pros than cons. With upx you decrease free RAM space, speed up HDD speed, and minimalize internet transmition. You protect TC from bloatware.

If you want to read executibles you can decompress files with upx -d argument.

There could be also two versions of kernel+core.gz (compresed and normal).


Some people boot TC from LAN or from the Internet. I plan to have router with pxe booting and no HDD, as a testing platform.

Compression in tcz is very inefficient. Its good for text and libraries, but not for executables.

[hiro]

i don't think you understand how tc works.
you don't do a batch decompression, you do it on the fly.