WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Can it boot faster? Let's try :)  (Read 4038 times)

Offline pioj

  • Jr. Member
  • **
  • Posts: 76
Can it boot faster? Let's try :)
« on: April 26, 2013, 11:24:59 AM »
Ok. I just learned to unpack, modify, and repack .tcz extensions this morning.

So, I'm trying to figure how could I speed some things a bit more. I have some more questions to ask:


1. Which approach do you guys like most?    (on a common modern machine, let's say 4GB RAM)

  A) SquashFS with -comp lzo.   LZO extensions will be bigger than gzip in size, but they will uncompress faster than gzip at boot time.

  B) SquashFS with -comp xz.   XZ extensions are smaller in size and they will load faster to RAM before they have to uncompress.


2. Are the  extensions first copied into Ram then uncrompressed into it, or are just directly uncompressed? 

3. Wouldn't be faster to pack some related extensions (iex. Xorg) into one big .tcz if you're going to load the whole extension anyway?



Note: I'm not talking about submitting anything or chaning the way the Repos are.   I'm just colleting some opinions...

Offline pioj

  • Jr. Member
  • **
  • Posts: 76
Re: Can it boot faster? Let's try :)
« Reply #1 on: April 26, 2013, 01:21:24 PM »
Ah I forgot..

Is it mandatory to use '-b 4096' when packing to tcz?  I found this to result in a bigger file...

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11041
Re: Can it boot faster? Let's try :)
« Reply #2 on: April 26, 2013, 02:37:33 PM »
For the first question, XZ squashfs is not supported in our current kernel config.

Quote
2. Are the  extensions first copied into Ram then uncrompressed into it, or are just directly uncompressed?

They aren't uncompressed at all, unless you choose so (via copy2fs). If they are, it happens directly from the storage device, without a copy to RAM first.

Quote
3. Wouldn't be faster to pack some related extensions (iex. Xorg) into one big .tcz if you're going to load the whole extension anyway?

Yes, it would. That's why we used to have some big packs (compiletc and Xorg), though it's much better for maintenance to have modularity.

Quote
Is it mandatory to use '-b 4096' when packing to tcz?  I found this to result in a bigger file...

Yes, as it greatly decreases RAM usage. For your own use, you can of course use whatever you like; but 4k for the repo.
The only barriers that can stop you are the ones you create yourself.

Offline pioj

  • Jr. Member
  • **
  • Posts: 76
Re: Can it boot faster? Let's try :)
« Reply #3 on: April 27, 2013, 05:18:05 AM »
Thx a lot , curaga.

Quote
Yes, as it greatly decreases RAM usage. For your own use, you can of course use whatever you like; but 4k for the repo.
I now understand what it does, thx. I'll be testing a lot with block sizes, so I may post my results on a separate thread :)








Offline pioj

  • Jr. Member
  • **
  • Posts: 76
Re: Can it boot faster? Let's try :)
« Reply #4 on: April 27, 2013, 02:06:53 PM »
Are the extension files compressed using UPX? 

I know by experience this tool works great with executables and libraries.

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11576
Re: Can it boot faster? Let's try :)
« Reply #5 on: April 27, 2013, 02:46:46 PM »
Hi pioj
Quote
Are the extension files compressed using UPX?
No, they are squashed into the extension. See:
http://wiki.tinycorelinux.net/wiki:creating_extensions#creating_a_tcz

Offline tinypoodle

  • Hero Member
  • *****
  • Posts: 3857
Re: Can it boot faster? Let's try :)
« Reply #6 on: April 27, 2013, 02:47:56 PM »
You can check that yourself:
Code: [Select]
upx -t file
I never noted any.

Are you sure about libraries? That would be news to me...
"Software gets slower faster than hardware gets faster." Niklaus Wirth - A Plea for Lean Software (1995)

Offline pioj

  • Jr. Member
  • **
  • Posts: 76
Re: Can it boot faster? Let's try :)
« Reply #7 on: April 27, 2013, 08:23:17 PM »
It works for exes and dll at Windows, but I'm afraid it only works for executables on Linux.

Anyways, we have a good example in Xorg-7.6.tcz, which has up to five different executables, 1 MB each. (Xnest, Xephyr, Xdmx, Xvfb, Xorg and Xfbdev).

I'll try to do some tests with them. Who knows...

Offline tinypoodle

  • Hero Member
  • *****
  • Posts: 3857
Re: Can it boot faster? Let's try :)
« Reply #8 on: April 27, 2013, 08:48:54 PM »
I'm afraid it only works for executables on Linux.

plus kernel ;)
"Software gets slower faster than hardware gets faster." Niklaus Wirth - A Plea for Lean Software (1995)

Offline neonix

  • Wiki Author
  • Sr. Member
  • *****
  • Posts: 391
Re: Can it boot faster? Let's try :)
« Reply #9 on: May 16, 2024, 01:26:24 PM »
You should guys start pack your executables in tczs with "upx -9". TC is getting bigger every year. You can also compile some programs with static libraries that will be compressed both with executables.

kernel could be compuked with uncomoressed core.gz and then upx with biggest compression should be used.
I am using mobile internet with limited megabites. Installing new version is still fast becouse TC is not bloatware.

Offline hiro

  • Hero Member
  • *****
  • Posts: 1229
Re: Can it boot faster? Let's try :)
« Reply #10 on: May 16, 2024, 05:26:37 PM »
upx is using some kind of lzma.
i disagree with your goal. compression and decompression with upx -9 is way too slow.
also, we would be compressing twice, that's not a good idea.

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11041
Re: Can it boot faster? Let's try :)
« Reply #11 on: May 17, 2024, 03:55:22 AM »
Upx comes with a delay on startup (the decompression), and it makes it harder to analyze the binary (readelf, etc). It may not make sense for the repo, but if you have a specific deployment where the size benefit is worth it, it's quick for you to script-loop over your extensions.
The only barriers that can stop you are the ones you create yourself.

Offline neonix

  • Wiki Author
  • Sr. Member
  • *****
  • Posts: 391
Re: Can it boot faster? Let's try :)
« Reply #12 on: May 17, 2024, 09:59:11 AM »
I'm not sure (I read source code of this program many years ago) but upx use lzo when you use -1 argument, gzip with -9 argument, and lzma/lzma2 with --ultra-brute argument.

On old x86_32 processors lzma may not work good, but gzip will always work and decompression takes miliseconds or 1 second.
On x86_64 all processors can handle lzma. Decompression time is less than 1 second, 64-bit programs are bigger than 32-bit programs.
There are more pros than cons. With upx you decrease free RAM space, speed up HDD speed, and minimalize internet transmition. You protect TC from bloatware.

If you want to read executibles you can decompress files with upx -d argument.

There could be also two versions of kernel+core.gz (compresed and normal).


Some people boot TC from LAN or from the Internet. I plan to have router with pxe booting and no HDD, as a testing platform.

Compression in tcz is very inefficient. Its good for text and libraries, but not for executables.
« Last Edit: May 17, 2024, 10:08:59 AM by neonix »

Offline hiro

  • Hero Member
  • *****
  • Posts: 1229
Re: Can it boot faster? Let's try :)
« Reply #13 on: May 17, 2024, 10:19:14 AM »
i don't think you understand how tc works.
you don't do a batch decompression, you do it on the fly.